I am Flynn, I am from DataWire, everything he said about me is mostly true. We're gonna be talking today about service meshes, what, why, and how. And I realized, while putting these together, that we actually had left off a really, really important word in that title, what, who, why, and how. Talking about this sort of thing is always important to get into who cares about it, why they care about it, in addition to how you're gonna use it. Just for my own notification starting off, let me get a, well, it's hard to see a show of hands, show of hands or shout out or something. How many of you would say you're primarily developers? Not a lot, uh-oh. How about operations? Okay, a little more off-centric today. How many of you have no clue you're just trying to get stuff done? That's me most of the time anyway. How many of you have run across service meshes before? Anybody use them? I see a couple of hands, that's good. Anybody care to shout out what they use or if they're still looking? Swarm? Okay. Anything else? Sorry, say again? Sir, it's back. Ah, okay. All right, so we've mostly been working with Kubernetes and I'm mostly gonna be using Kubernetes-specific terminology to the extent that it fits anything. But everything that I'm talking about here with the mesh, with the patterns that you can use with it, it's pretty independent of how you actually get things running. So really quickly, we're gonna hit the what of a service mesh and then we'll dig into who and why and then we'll hit how at the end. Depending on how we're doing at time, we'll have a really, really brief demo, but I'm basically gonna be prioritizing questions over the demo. If you have questions, let's hear them. The point of a service mesh is that you're taking a bunch of services and collecting them all into something that can work together as a single application. It kind of extends the idea of containers and Kubernetes and things like that. Containers take a bunch of processes and give them a standard sort of interface so that the ops guys can work with them more easily. Kubernetes is about a sort of standardized, sane experience of getting these things to run together, likewise swarm and the other things like that. Service mesh is kind of extending that idea. Another thing that's interesting about this is that one of the tricks with the microservices world is that they are distributed applications and sadly they come with the complexities of distributed applications. One of the other things that a properly done mesh can arrange is it can give both development and operations a little bit more of the experience that we had working with a single process monolithic thing where yeah, they were a pain to develop. It was slow developing them, but we at least tended to have a decent amount of visibility. You had a single process, you could attach a debugger to it, you could poke at it and again, a well-designed mesh can let you recover a little bit of that sort of feeling, maybe not all of it, but some. And that leads us to the question of why does this stuff matter? Who cares about it? As we talk about that, I'd like you to meet Jane. Jane is the microservices development lead for the Evil Genius Cupcake Company where the cupcakes are so good, they're evil. She's working on a shiny new application, she doesn't have anybody using it yet. It's just something that she's just started off with. Specifically, the marketing folks have noticed that some of the people who buy Evil Genius Cupcakes really, really like them and they're off doing all manner of evil things hoping they get more cupcakes. So they have a user, the user wants to go through and keep track of their evil deeds, the evilness tracker has to save these things in a database and then eventually somebody, maybe the user, maybe the marketing, will want to go through and read some information about these things. This is kind of the extent that Jane is gonna be thinking or the level at which Jane's probably gonna be thinking about this design. She will probably at some point have to be aware of the fact that it is off running in a container so she has to think about how people get access to it but she doesn't really want to go and dig too far down into the details of it, into the technical stuff. She doesn't want to get involved in yak shaving. Her counterpart Felicia is the operations lead for Evil Genius Cupcakes. Felicia has a very different view of the world. Her job is wrangling all these dozens of microservices plus a legacy model that's been trying to make some sense of it. I should point out since we're at a DevOps conference that in any reasonable organization these days these are really roles, not necessarily people. Felicia may be doing her own development when she's not on call, Jane may be on call but at any given point these people will have a specific role in mind. If Felicia gets called in the middle of the night she's not gonna be thinking about life as a developer. She's going to be thinking about it as an ops person. This is Felicia's view of her world. She's got a cloud, inside the cloud there are a bunch of services running, there's a monolith and they all talk to each other in ways that may or may not be terribly obvious. When life is good these things talk to each other, they play nicely, everything is great. When they stop Felicia's phone rings and she has a very bad day. So what does Jane want out of this world? Like we mentioned she's thinking about this as a business problem. She wants to let people track their evil deeds, she wants that to be straightforward and easy for her. She wants it to be elegant to the sense that when she looks at the code later she doesn't want to be horribly embarrassed by the hackeries she's thrown together, she'd like it to be something she can feel good about. She would like to think about the business problem and not have to go dive down into hideous technical stuff every single minute of the day. Felicia mostly just wants her phone not to ring. But when it does ring, and it will, she wants to be able to quickly figure out what's going on, she wants to quickly get a sense of where the problem is and who she needs to bring in on it. The question is, can a service mesh help them with this? Any guesses, any opinions? I'm gonna vote yes. Can it always help? Ha ha ha ha! No. Not just no, but the heck no. To get a little bit of a sense of why we can dig into a little bit of what Jane is gonna be doing as she's working on this thing. She's gonna start off with something that she's pretty much just beating on, trying to make changes to, trying to get running. She doesn't really have any users yet. What does it mean to successfully deploy this thing at this stage of the game? If it doesn't crash, we can call it good at this stage of the game. If it does crash, she just restarts it, no big deal. DataWire calls this one stage one. Probably she'll fairly quickly get over to a second stage where she actually has a real users. Real people out in the world trying to use this service. What does a successful deployment mean at this stage? Say again? It can handle a load. It can handle a load? Presumably it works. Basically, something that doesn't disrupt her existing users, or at least hopefully doesn't disrupt them very much. She may, depending on what the service is, end up at a third stage where instead of users depending on it, it's users but also other services within the application depending on it. So-called foundational services. What about that? What does a successful deployment mean if her service is one that's foundational? I'm sorry, say again? Well, the biggest thing it means is that we don't have cascade failures. If something goes wrong deploying this service, it needs to not take down the entire application. It needs to be better behaved than that. It's worth pointing out that at any given moment, some services can be in any of these stages. You may have people developing brand new services while there are other ones that are running in production while there are other ones that are foundational. It's also worth pointing out that not every service will become a foundational service. Some of them don't need to be. Some of them you get part way through and realize that they weren't really what you wanted to be working on in the first place. So, what's Jane likely to be worrying about? Her first worry is probably just gonna be to get the thing running locally at all, right? She's gonna have some tech stack in mind that she wants to use for this, and that one is a care from stage one all the way on through. She's gonna need to get running in whatever her cluster is, Kubernetes or whatever, deployments, replica sets, that sort of thing. Also in stage one. She has to worry about debugging this thing in a cluster environment, which can be a royal pain when you can't really connect to it very easily. People tend to start off with just attaching to it and running shelves, and she's gonna need to provide access for users. She might use an API gateway for that. She might use the Ingress resource. There's a lot of different things along those lines. I'm gonna, if you'll forgive me, a bit of shameless self-promotion. DataWire has things that are intended to make all this stuff easy. Forge and Kubernetes are trying to solve the problem of it being a hassle to get stuff running trivially in Kubernetes. Telepresidents for debugging, Ambassador, which we'll hear more about later for access to users. This is all lumped together under the project Blackbird, there's more on that later. Other things that Jane has to worry about. In addition to the mechanism of being in a place where she can see what's going on, she has to make sure that the stuff is there to look at to debug her service. Fortunately, there's this shiny technology called Logging, it's been around for a little while. Kind of the core of a lot of what we do for arranging to have the information for debugging available. She has to worry about that, of course, from the very beginning. One of the next things she gets to worry about is being able to safely upgrade her application. A new, she gets a new release ready, she might be doing a Canary deployment on this one. Roll out the new version of your service, route some of your traffic over that way, find out whether or not it works. If it does work, life is good, you can route all the traffic that way and then bring down the old one. If it doesn't work, well, you know, just route the traffic back to the one that's known good and tear down the bad one. Some obvious technologies for that one, she's gonna need metrics and building the service to be able to determine whether it's working correctly. She needs a way to route or request coming in to the new version versus the old version, but not all of them. Any guesses for what stage that she's gonna start caring about that? Very good. She doesn't really care until she has new users, until she has real users, rather. Before she has real users, she can just tear down the running service, leave it down as long as she wants to, and then bring up a new one, it doesn't matter. But at stage two, once there are real users using this out in the world, you care a lot about this stuff because you don't wanna get in their way. I think somebody earlier mentioned that one of the things Jane is gonna care about is whether her service can carry the load. Grace under pressure. Maybe Jane has a friend who's a YouTube star who goes and does a video log about their evil genius cupcake company and suddenly their traffic goes up by an order of magnitude in an hour. It would be nice if the thing didn't just crumble over and die.  One of the technologies that can be helpful for this is rate limiting deployed in certain fashions. Where would she start caring about this? Three, I hear a three. Anybody have any other opinions? I think it's two. Three is the foundational service where there are other services relying on her. But really at any given point, once there are real users, then you could end up with a swarm of them that you're not expecting. So have to worry about this a little bit earlier. And of course, as we've mentioned, you can be in a situation where the evilness tracker going down takes the entire application down. None of the customers can do anything. That's terrible. That's the sort of thing where you might deploy circuit breakers for it. You might deploy rate limiting in a different fashion. And that, as you were mentioning, definitely is the sort of thing that shows up at stage three, unless Jane never has to think about it at all. Maybe this is something where the service mesh can take care of it for her. Felicia's take on this is kind of different. Felicia is gonna start off thinking about things in terms of transparency. Mostly in the sense of at any given moment, it would be nice if Felicia can figure out what exactly the whole application is doing. She's gonna be relying on the logging and the metrics that Jane builds into the thing in order to do that. Felicia will probably start caring about situational awareness. It's a term for the military. People flying fighters with other people shooting at them don't actually have the luxury of stepping outside the cockpit, chilling out for a little while, studying the positioning of everybody on the board, and then getting back into the cockpit to react. They have to do everything very, very quickly.  You don't really wanna be studying things. You wanna be able to just look at something, get an instant sense of what's going on. You want some sort of a good overview of the way the application works in general, of what it's doing. This is the sort of thing that you tend to build a top logging and metrics and such, but it's a pretty different thing. Felicia cares a lot about consistency, both in the sense that she doesn't really wanna have to deal with 15 different logging formats, but also in the sense that she'd like to know that things that need to be authenticated are actually authenticated, and things that have company policies around them, the policies are being followed. Really the only way you can get that is by making the tools that follow the policies and that create those formats are reusable and available to everybody. If you make it easy for the developers to go through and do the right thing, they tend to do the right thing. If you make it really hard, it tends to happen a lot less. Felicia of course also cares about grace under pressure. She would really like a lot of this to be happening down in the infrastructures again so that the developers can't get it wrong, but in some cases the developers are going to have to help her out. If you're talking about a canary deployment, it tends to work a lot more usefully if you can decide that the people who get routed to the new version are people who've signed up for the beta program, or people who have otherwise indicated that they're not gonna care if things go wrong for a moment. Another one is that all of the metrics that go into figuring out what's going on, those are really gonna be driven by the business. The ops people are not gonna be the ones deciding that. And finally, Felicia is gonna care about avoiding pre-medonna developers.  The only way to really get around that one is to make all your stuff platform agnostic, which is a pretty big ask. So Felicia actually cares about this pretty much all the time, sadly. If we go from what they care about into how do we make this happen, it's important to recognize, to remember, that Jane's basically trying to focus on solving a business problem.  She is a developer, but she would like to be focusing her development energy on things that actually help solve her problem, as opposed to things that are a hassle that are unrelated to her problem. Felicia would like to be paying attention to the application as a whole without getting sucked into every little piece of the development world every time. You can use technology to help get both of these people what they want, but it turns out to be a really hard problem. And for why it's a really hard problem, let's look a little bit into a particular technology we talked about earlier, circuit breaking. So, this is Felicia's view of the application. We're gonna take a look at one particular chunk of it. And as we take a look at that one particular chunk, it's really, the monolith might be talking to any of the instances, in this case, three instances of this particular service. Circuit breaking is the idea that if one of those services just randomly dies, you do this radical thing and you don't talk to it. There are two other instances of that service. Let them give you answers that are good. After a while, you can try to talk to it again, see if it's up. If it's not, no big loss. Just don't talk to it for a little while longer. If it is up, of course, you can put it back into the rotation. Pretty simple to describe. If you actually look at what you have to do to make this happen, though, you have to start off by keeping track of which instances are part of the service. Obviously, you have to keep track of which one is dead, but you also have to keep track of when it died so that you can keep track of how long to wait before you try it again. You probably need some sort of exponential back off because it doesn't really make a lot of sense to keep trying this thing every second. Try it a second, then try it two seconds. Who knows? You have to actually implement the retry logic, which includes understanding what it means for this service to be healthy. It could easily be the case that it will answer your connection and give you an answer, but that the answer is utterly wrong. And there may be business logic going along with deciding whether you should trust it that goes beyond whether it's actually up. You have to keep track of whether or not new instances have spontaneously appeared for this service, and you have to keep track of whether any of them have gracefully bowed out as opposed to just dying when you didn't expect it. So this one particular thing, which is fairly simple to describe, is not at all simple to implement. And of course, it's only one of the technologies there. Rate limiting brings it its own set of complications. Even things like logging and metrics, once you start digging into how do you actually do them as opposed to how you actually describe them as so many things go in computing, as soon as you start dealing with the real world, the details get to be really, really nasty. So trusting the individual Janes of the world to get this right 100% of the time in all situations, all the time forever is basically a non-starter. That is no commentary on Janes' skills. That's commentary on what we as human beings can actually do. The easy way to deal with this is to take all this hard stuff and move it down into the infrastructure. Same paradigm we follow with operating systems, device drivers, all this kind of stuff. Get it right once, let everybody use it, and your life will be easier. This is what the service mesh is really all about. This idea that you could take all of this nasty stuff, push it down so that the developers don't have to constantly re-implement the wheel and then run with it. This of course raises the question of how you actually make one. There are several ways to go. There are always several ways to go. Data wires kind of biased. We happened to like Envoy, open source project that originated with Lyft. And we kind of especially like Envoy when you stick ambassador or SDO on top of it. This is Envoy. URL is there, it's just accepted into CNCF which is kind of cool. It's at a very high level, it's a smart reverse proxy that can operate both at layer four and layer seven simultaneously. It was built for the ground up in C++ to make HTTP2 and GRPC first class citizens. It's one of the only things that actually properly handles HTTP2 trailers for GRPC for example. It was built at Lyft, it ran there for a couple of years in production before they open sourced it. So to say that it has been battle tested is something of an understatement. It handles a brutal amount of traffic right now. And this is how you can use it. We'll go back to our Felicia I view of the world here. Arranged to have a little bit more room to play with. Everybody's probably familiar with the reverse proxy pattern, right? You have a load balancer at the edge of your cluster that allows people from outside your cluster access to inside. You stick a reverse proxy behind that. You let people outside talk to your reverse proxy and make queries. Those queries get interpreted by the proxy. The proxy decides which of the services to talk to and for is the request on. Worth noting, not all of the services have to be exposed to the outside world. Also worth noting that in this case we've got the reverse proxy set up so that the monolith is behind the reverse proxy. Some people do it differently. It doesn't really matter.  You can add a bunch of extra smarts into it. You can put in routing that can be dynamically updated or that can be dependent on who, can look at a cookie, figure out who the user is and decide where to route based on that. You can put in authentication and authorization into the proxy. You can put in logging hooks and statistics gathering things. You do all of that. You get something that is not really a reverse proxy anymore. Now it's an API gateway. This is basically the ambassador product. And the way ambassador works is that we use Envoy for the reverse proxy. Stick the ambassador code next to it. Teach the ambassador code how to go and look into a bunch of config maps in Kubernetes to figure out how it's supposed to be doing routing, what other features it's supposed to be enabling. Ambassador can go and configure the Envoy and all of this well. Like I just said, API gateway from data wire. It's important to note that Envoy is actually doing all the heavy lifting there. So you get all the performance wins and all the battle testing stuff. The main thing from ambassador is that if you go and look at how to set up Envoy, it's not the most trivial thing in the world to do by hand. And so ambassador ends up introducing a simpler and much more opinionated idea about configuration to make it easier to do the easy things and still make it possible to do some of the hard things. Most of the hard things really at this point. One example of a way in which Envoy is easy to get wrong is that when you're defining routes in Envoy, they're actually order dependent. And if you end up putting the wrong one first, very, very unexpected things can happen. Ambassador can handle that on its own. When I say simpler and more opinionated, that's a complete example of all the information you need to do a request route in ambassador. So basically, ambassador is kind of a light version of a service mesh. But the interesting thing is that once you have this idea that you've stuffed a bunch of this complex logic down into what used to be a proxy and turned it into something smarter, you don't have to stop with this configuration. You can put Envoy's everywhere. It is light enough weight and high performance enough that you can literally stick an Envoy next to every running instance of your services. And at this point, you can do all sorts of interesting things like you can allow the Envoy's that are running next to some of the services to be following different rules about circuit breaking or rate limiting or routing than the one running out there at the edge of the cluster. If you are careful about keeping them all in sync, then the whole cluster can both respond to routing changes but also not have to do things like hand requests all the way back to the edge to be reproxied, they can handle it locally. They can gather statistics from all the places throughout your entire mesh feed it all into the central aggregator. There's a bunch of really cool stuff you can get with this. This is basically the Istio project. Another open source thing, this one out of IBM and Google partnering with Lyft, they got a ton of developers working on it. One of the things that's cool about this one is that in order to deal with the whole platform agnostic thing, it's all about sidecars rather than libraries. You stick a running Envoy next to each of these services and then actually Istio uses IP tables, magic to force all the traffic through the Envoy's. The end result is that you never get the Haskell problem and you don't have to worry about whether or not your client libraries have been updated to the latest and greatest in any given platform. Huge win there. Also worth pointing out that ambassador plays nicely with this one and that you can stick an ambassador in front of an Istio cluster and kind of get the best of both roles there. So at that point, the way this one kind of works is that Istio ends up putting a little bit of code next to an Envoy, but they actually do that for all of the Envoy's in the cluster. And then they have this thing called Pilot. Istio is the Greek word for sale, so most of their terminology draws on nautical terms. So they've got this thing called Pilot. Pilot ends up able to look at the various configuration resources that go along with Istio and then it can update all of the Envoy's all at once and keep them all in sync. Istio also has a thing called a mixer that is sort of an escape hatch for insanely complex application-specific stuff. So basically the point of the service mesh is that you're trying to give both Jane and Felicia the things that they want. You're trying to do it in a way that's sane, in a way that's consistent from application to application from service to service. And you're trying to do it in a way that's easy for everybody to actually use, for everybody to actually do the things that you'd like them to do. Some of the obvious things that you can put in here are routing, logging. There's a bunch of metrics you can do there. You can gather them into a single aggregated spot. You can do circuit breaking and rate limiting down at the infrastructure layer rather than having to reimplement it for every service. You can centralize authentication and authorization. It's kind of a nice win. Again, the point is get it right once, let everybody use it. Don't make people reinvent the wheel constantly. There are things it cannot do that a service mesh will not help you with. It is not going to make everything an ops problem so the developers in the world don't get to just pretend that it's somebody else's issue and they don't have to think anymore. Some kind of specific things on this that we mentioned, for example, the canary deployments. It's a good idea to be able to pick users that make sense from a business point of view to route over to the new version first. It's also a good idea to be thinking about the business logic behind the metrics and behind the determination of whether a given canary is actually still breathing or whether it's being poisoned. Likewise, the mesh is not going to turn everything into the developer's problem. There is still going to be a need for people in the operations role where if something goes sideways, then that's not the time to be digging into a particular service first. The first thing that you're going to need to do is to get a sense of how the entire application is going wrong and then direct your efforts that way. Another thing that's worth pointing out is that going immediately to a full-blown service mesh may be overkill until you have some of the services at stage three. You can actually get a really long way by putting an API gateway in front of your stuff and starting that way, starting out by using the sort of light version of the mesh to go through and get some benefit from this thing without going through all of the pain and bringing up the whole thing. Some really good examples of this one are routing and authentication. It's actually really nice to be able to centralize authentication into one place and know that if a service gets the request, if it needed to be authenticated, it was, very handy. And you get some of the simpler mesh functions. You can do some centralized logging and centralized stats with just that part too. It's in a lot of ways can be thought of as a place to stand as you're going through and starting off with some of this common stuff. And then as you need it, it's usually easier to go from there onto the full mesh than to just do it all at once. Full disclosure, I work on Ambassador, I am therefore biased. Seems to be working out so far though. In conclusion after all of this, the first thing to take away here is that it's all about the users. It's all about the people trying to use this as opposed to trying to create the thing. Our job as developers is not to make our own lives easier, it's to make the lives of the users easier. Sad but true. Meshes can be a really handy tool for this, although they are not a silver bullet. They never will be. Pretty much nothing will be, sadly. And it's worth remembering again, the full blown mesh might be overkill at first. There may be a way you can start with just the API gateway and then have a way to move forward onto the full mesh. And as I mentioned before, as we've looked around at some of these things, we can like Envoy and Ambassador in this year for doing these things. It's worked out really well for us. But if you're gonna take one thing away from this, this would be the one. Don't forget that. Any questions? There's the Blackbird stuff. Anybody wants to see it? We'll put that back up later, but in the meantime, let's see about showing a couple of things here. All right, so this is a Kubernetes cluster. And that is, this is actually gonna be kind of interesting doing this with the windows on the side.  And what we're looking at over there is just the set of pods that are currently running. You can see the Ambassador pods. Let me see if that's gonna be a little bit more legible. A pod is a thing that is running a container mostly. You can actually have a single pod running multiple containers. It's actually a better analogy might be a virtual machine. And I'm waffling a lot here because Kubernetes actually has nodes, deployments, replicasets, pods, containers, and it's turtles all the way down. But yeah, you can think of a pod as a virtual machine in this case that's executing one or more containers. I'll also plug my colleague Phil who's gonna be talking about Kubernetes tomorrow. Hopefully that's a little easier to read though. Now, that's the configuration that Ambassador is using right now. There is an external authentication service running. So everything that comes into the Ambassador at the beginning of this, at the edge of the cluster, gets passed through an external authentication service that decides whether or not it needs to be authenticated and if it does, does it. If their authentication fails, then the Ambassador kills the connection before handing it off to the service. If it succeeds, then it goes ahead and hands it off to the service. That bit with the loud headers there is letting Ambassador know that if the authentication service supplies that XQOTM session header, then that header needs to be passed on through to the service itself. The other things in here are mappings. So if you get something whose URL starts with QOTM down here, yeah, that is the right one. Then it gets handed off to the QOTM service, which is the quote of the moment service. If it starts with reviews, it gets handed to reviews, but it's reviews version one. Ambassador actually covers ordering all these things properly so that they work. So let me also do this. I'm sorry, Sega? Right. You mean the thing that's going to be right? You mean the thing that determines where QOTM goes as a service name? Yeah, Kubernetes does an internal DNS service. So Kubernetes itself keeps track of which pods and which instances are running the QOTM service and then populates its internal DNS with that. So you can just refer to things, its full name is something like QOTM.default.cluster.kubernetes or something along those lines. Awesome. All right. We actually have quite a few services. Oh, actually, you know, let me be more clear about this since there was a question about pods. A Kubernetes service is actually a mapping that is telling the Kubernetes cluster that some pods are running containers that are prepared to answer for a given service name. The pod is the thing running the code. So you can absolutely create a service that has no pods behind it and you won't really be able to do a lot with it, but one thing you can do with it is, for example, use it to set up your DNS to go through and do TLS termination and things like that. But when I ask Kubernetes to get me the services, then it shows this huge batch of things. Some Istio stuff, some ambassador itself, things like that. If I then turn around and ask it to show me the pods, it actually gives me something of a smaller set, but you can see that there's one ambassador service there but two ambassador pods so that if one of the ambassador pods dies, the other one can take over. Likewise, we have reviews version one and reviews version two running at this particular moment because sometimes it can take a really long time for Kubernetes to pull down a Docker image and spin up a container and I didn't really want to wait for it. There's only one QoTM service running. This is one of the things that's a little bit weird about Kubernetes if you haven't gotten used to it, but if we go back here and we ask for the ambassador service in particular, my laptop is not in this cluster. So from my laptop's perspective, I have to talk to the external IP and if I were going to do this with TLS, for example, I would need to use that external IP, set up a DNS name pointing to it, get a certificate based on the DNS name and all that crap, which is why it can be very important to follow the distinction between a pod and a service for that. But if I do this, oh good, we were actually able to get something through it. That's always nice. I can actually open this in a web browser because that is a web page. It will give me this page. For those of you who try this on your own at home with the Istio stuff, this is actually the Istio Book Info demo running right now. Okay, under the hood, these are coming from the review service and the rest of it is coming from the product page service and there's turtles all the way down again. Another thing we can do though is having actually generated some traffic. We can look at this Grafana thing and we can see once it reloads, come on Wi-Fi, you can do it. Yeah, so you can see that Grafana is showing us that there have been some requests made and they've gotten good results and this is all very nice. If anybody is curious as to whether I'm faking the whole thing, that's not right. Okay, typing while looking sideways is really hard. Quick, where's my mistake? Nicely done, sir. As we say in software, never precede a demo with a comment more predictive than watch this. There we go. So yes, I am actually sending traffic over here and yes, HTTP IE is a Python program so it can't do very many requests per second. This is not the fault of the Kubernetes cluster. Just wanna get that out there. All right. Now, I'd like you to remember this, looking at that graph and looking at the demo and all that stuff. Another interesting thing is that HTTP IE doesn't actually die when you control C in that loop which I found really entertaining the first time. So, another thing we can do is we already brought up reviews version two. If I come back here and I ask for the pod list then you can see that I have reviews v1 and I have reviews v2 running. I can change the configuration mapping my reviews for the record. I don't use VI except in demos. Let's point this instead at reviews v2. The reason I do use VI in demos is that it's a nice way to make sure that you're not totally confusing everything with opening new windows and such. All right. The way Ambassador works in this mode is it's paying attention to this config map called Ambassador config. Very creatively named I know. And this is the hideous Kubernetes idiom for I want to update this config map in place. There are things about Kubernetes I really, really like. Mostly that it does a pretty nice job of making it fairly straightforward and predictable to bring a bunch of services together and let them cooperate. But man oh man, its command line is not one of the things I like. Kind of reminds me of Git. Now we're gonna force a redeployment of Ambassador. We're using the Kubernetes deployment machinery to affect configuration changes in Ambassador. The idea here is that if you're running code in an environment whose job is orchestrating containers and orchestrating pods, then there are some real advantages to treating configuration as immutable and then just triggering a redeploy when you need to change it. One of the earlier versions of Ambassador did not do that. We had a long running process that would go and make a bunch of changes and such. It, this way is better. So now if we go through and we reload our web page here, we should see something different. Look at that. Reviews version two has stars on the page. Now, if we wanted to, actually let's just go ahead and do it. We're gonna delete a deployment in Kubernetes is a description to Kubernetes about basically here's a Docker image or a set of Docker images that I want you to run in containers inside of a pod. I wanna have three of them running at all times. I want you to do a rolling update when I change anything about the description. And you're basically letting the machinery of Kubernetes handle all that junk for you rather than doing it all by hand. So what I just did there was to delete the deployment for reviews v1 entirely. If I now come back and get pods, then you can see that it's marked terminating. I actually have a watch running over here. So yeah, now it's gone. And if I go back to Chrome, nothing happens because I'm talking to reviews v2 taking down reviews v1 is no big deal. All right, almost done with a demo. One other thing I wanna show you though is the quote of the moment. It's a very profound service. The quote of the moment is a service that my colleague Phil wrote that you can request a bizarre and existential quote from it. If we repeat this, then we'll get one of a set of bizarre and existential quotes. If we ask for a specific quotation though, the, you remember I said ambassador was talking to an external authentication service? The external authentication service is designed such that it requires authentication for anything starting with qotm slash quote because you see you can also push quotes using this thing and that's not something you should be able to do on your own or just anonymously. So if I do that, I get a 401.  Psy. It'd help if I remembered which program I was running. There you go, so that works. Now, the other thing I'd like to show you though is I would like to show you the source code for the quote of the moment. And before I do that, let me point out also that if I come over here in Grafana, then we can see, you know, we can see the 401 that I got, we can see the successful 200s. We're basically seeing a fair amount of information about what's going on in here. I haven't customized this graph at all. That's literally just the first thing that opened up. If I bring over the source code for the quote of the moment, yeah, that's pretty much all of the relevant thing. That function up at the top there is the one that responds if I just say, you should hit the URL and have it get me a random quotation. And the one at the bottom is the one for asking for a specific quotation by numeric ID. And what do you not see in this code? Remembering that we just saw it deny us because we were unauthenticated. There's not a single thing about auth in that code. There's also not a single thing about logging. All that's being handled by the infrastructure underneath the application. Likewise, the routing from reviews V1 to V2 that you saw, the reviews service doesn't have anything. I'm not gonna show you the code for that one because it is in Java and it wouldn't fit on the screen. Oh, actually that's a good point. The book info demo and quote of the moment between them cover Ruby, Java, and Python. And they're all using the same infrastructure with no client libraries. Unless you can't flask as a client library. So all of that, the request routing, the authentication, the logging. All of that is stuff that we pushed down into the infrastructure to let anybody use and let anybody use in the same way. And you end up with stuff like this that works and can be changed on the fly as you like. That's the goal. That's it for the demo. There's a ridiculous amount more that you can do with the combination of ambassador on top of Istio, on top of Envoy. But this is it for now.  I will be around for at least a few minutes for questions. Come on up, say hello. Thanks very much, I appreciate it.


Live from New York, extracting the signal from the noise. It's theCUBE, covering Spark Summit East, brought to you by Spark Summit. Now your hosts, Jeff Prick and George Gilbert. Here with theCUBE, along with George Gilbert from Wikibon, we are live in midtown Manhattan at the Spark Summit East event. Excited to be here. We did a quick drive by last year in San Francisco, but got the whole CUBE set here. Been going wall-to-wall for two days, really going out, getting the smartest people we can find, getting you the insight and the information that you can use, and we're really excited with our next guest, Justin Langseth, the founder and CEO of Zoom Data. Welcome, Justin. Thank you, thank you. I think last we did see you in San Francisco under slightly different circumstances with not our fancy sets, and now you get the whole CUBE experience. I think it was like an iPhone, a flashlight, some kind of tin can microphone. Exactly, we can go gorilla when we have to, whatever we take. But first off, congratulations on your funding around you announced the other day, $25 million. Yeah, we announced it yesterday. Yeah, two days ago. Interesting, Goldman led it, even though you still, you have a really awesome group of VCs behind you, but Goldman led it, so that's a little bit different. Yeah, Goldman's a strategic investment group, different from their merchant bank, so the merchant bank is looking for financial return, but their strategic group, which is led by their CIO, Marty Chavez, here in New York City, is really looking for strategic things to invest in that Goldman actually wants to use, or is using in their technical environment. So it's kind of a different kind of take on it. Yeah, absolutely. So what was the zine to that? Are they using some, using you guys now? They obviously see great opportunity and opportunity. Yeah, yeah. Dig into a little bit deeper. How are they using it? How do they want to use it? Where do they really see the business benefit? Yeah, so they have a lot of, obviously, modern stuff going on over there, and they're trying to look for tools that are both developer-friendly, because what's interesting about Goldman is 10,000 of their 30,000 employees have CS degrees, and they're actual coding engineers. 10,000 of their 30,000 people have CS degrees. One third of the company has CS degrees, yep. Wow. So they were in IT. I didn't know they had CS degrees. Yeah, I don't know if they all have CS degrees. They're all engineers, so they can commit code and stuff like that. And they're really looking at data, data visualization, business intelligence, and I think this is a general pattern as well, but it's shifting from kind of like a platform side of tooling with the cognosis and business objects of old, kind of looked like enterprise kind of platforms. And like a lot of IT, it's moving more towards kind of apps and building apps and deploying apps for specific purposes. And that could be internal to internal groups of employees. Maybe some traders need to see an app about something in real time, or it could be to external customers of a company. So both the developer friendliness and a lot of stuff we're trying to do around that is really appealing as well as our ability to really leverage security kind of on an end to end way. So the user's security kind of can be passed all the way through the Zoom data platform down into the underlying data stores. And what the user sees on the screen is reflective of their as of right now security credentials, and if those credentials change, literally the data on the screen could change or disappear. And this is a feature that our government customers also like very much as well. Yeah, and that's really interesting. We talk a lot about perimeter-less security, right? The whole security paradigm has completely changed and really needs to be down at the data level as well as the application layer.  Or your clearances, it gets broad-ended. Or it gets broad-ended even when suddenly you have more piece of pie on your chart. So not making copies of the data is the biggest thing. Having one copy of the data, securing it one way, having it in one place, and bringing all the questions to that data as opposed to taking copies of the data and sending them out to the questions is a huge differentiator. And I think that's why our technology that can be run both on-premises as well as in-clouds and also in a hybrid manner is critically important because you don't have to take all your data and copy it to some other system. Because once you copy it, it could be stolen, it could be disjoint, it could be, you know, different security could apply or all kinds of problems. You could lose some of it, like huge problems occur, right?  Which is to tell us sort of the ethos of each generation starting with the business objects, Cognos, and then the tableaus and the clicks, and now the zoomed datas. What's the core design philosophy and the user problem you're trying to attack? Yeah, so initially in the mid-90s, I don't know how far off you want to go, the mid-90s, it was a desktop query tool contacting a single database and that was business intelligence. I worked at MicroStrategy in the mid-90s and we built this thing that did that, right? And then the client server wave hit so we introduced a server level. So business intelligence like all other enterprise software now had a desktop client and then a server side and so more and more of the work kind of shifted to that BI server layer and that's where the prior generation of tools, the business objects and the Cognos really have built out a late 90s, early 2000s kind of traditional client server layer where in some cases it's a Windows-based kind of architecture but regardless, it's not their fault. They started 10 or 15 years ago and things like big data didn't exist, things like cloud didn't exist, things like microservices certainly didn't exist. So any technology that hasn't been kind of built from scratch in the last three to five years kind of cannot have benefited from the concepts around cloud and mesos deployment and Hadoop and kind of microservices architectures. So we fortunately, its timing is a lot of stuff in life, right? So we started three and a half years ago after Hadoop and NoSQL and microservices and cloud, all this stuff was out there and so we kind of benefited from that and built a very modern kind of architecture. Can you tell us more about this modern architecture? You alluded a few minutes ago to not bringing all the data over the wire sort of like one of the ethos of Hadoop is put the compute on the data. So elaborate more on sort of the constraints, the environmental constraints you're living in and so how that affected your design. Yeah, so we knew Hadoop was going to be big when it started the company, but we had, enough of the Hadoop had happened so far that we realized all data's not going to go into Hadoop. Some data's going to stay in traditional data stores, some data's going to go into other new things like Elasticsearch or MongoDB or Redshift or it's not all going to go into one Hadoop place. And I think some of our more newer generation competition frankly started a year or two before us when it was all about everything's going into Hadoop. So as long as you can access the data in Hadoop and run MapReduce against it, MapReduce, note, not Spark, right, everything's great. And so we came in after it was clear that Hadoop's important, but legacy's important, cloud is important and Elasticsearch and MongoDB and all of a sudden no SQL stuffs because Hander's important. And so we built an engine that can push queries and push jobs all the way down into any of those data stores including Hadoop, including Spark, including Elasticsearch and MongoDB and then can also bring that all back into Spark which is why we're here at this conference because we depend on Spark in our middle tier and that's where we do any work that can't be pushed down into the database or into the Hadoop or into the Mongo and you want to push as much as possible but maybe the system is a legacy Postgres or SQL server and you can't push work there because it's not performing enough to be responsive to users or you want to join data together between a SQL server and a Mongo and Elasticsearch and you can't push it all that work into any one of them because you'd have to copy data and again we decided just a minute ago copying data is bad so that's why we used Spark to kind of glue all that together. So Spark is your sort of data analysis application server. Spark is our middle tier where we do anything that can't be pushed down or shouldn't be pushed down to a lower level. We prefer to push work down because generally there's a big cluster there with thousands of machines running Hortonworks or Cloudera ready to receive it. However, if you're trying to do work that can't be pushed down because or you want to join stuff together across things that's where we use Spark. And we use Spark also and back to your question about BI middle tiers. Traditional BI middle tiers will generally hold sets of data in them and those could be result sets or they could be cubes of data depending on which technology you're using. Some of the technologies really require you to kind of cube all the data into a middle tier but most of those are proprietary middle tiers that have been built over the last 10 years often using kind of less scalable technologies. And so we use Spark as that middle tier and we run a little embedded Spark inside Zoom Data if you just turn on a Zoom Data Docker container or something but we can also repoint that to an external Spark cluster. So if you have 2,000 nodes of Spark instead of having a limited middle tier BI server suddenly that 2,000 node Spark cluster becomes your uber scalable BI middle tier for anything that can't be pushed down to the no end stores. So okay, that was my next question. What's the difference between your middle tier today that's made of Spark and essentially the middle tier of the client server era and it sounds like that was a scale up tier and now you're on a scale out general purpose data processing tier. So our middle tier is effectively it creates Spark jobs and jockey Spark jobs. So it's trying to do, in Zoom Data Server itself we're serving up the GUI and handling that interaction. We're trying to do almost no actual data crunching because we want to do all that in Spark or even better in the underlying data store to the extent it can do it performantly. So we are looking at anything a user wants to do figuring out how we can address that question very quickly and get them to see either the full result or an estimated result within three seconds because that's a human attention span. We stopped talking for three seconds. Your audience is gone, right? Let's try it. No, okay, let's not. All right. So you gotta show the user something within three seconds. And so that's what we're all about is figuring out how to do that and we do that through Spark jobs. And if you have one user, one minute and a thousand users show up the next minute you can scale it very elastically because you're using Spark or if you point it to a small data set and then you point it to a thousand times bigger data set Spark can scale in that a traditional BI middle tier can not. The Bob J or the Cogniz would not. Even if they can cluster, to some of them can't cluster across multiple machines you still, you'd have to add another physical machine or a virtual machine. You'd have to reboot the whole thing and point it. Whereas we can run on Mezos and Mezos automatically scale up and scale down, right? Just automatically based on load at both ends of it. Okay. So, how about, one of the things probably leave this name on said but of some very insightful comment from a old friend of the, of theCUBE, said, you know, we've gone from this era where we have a huge amount of data in storage like in a file system. We sample that, put it in the database to kind of massage it and clean it and then we sample that to do the, you know, modeling and sort of get rid of the outliers. And then we sample that and it gets it into the visualization tool. Now the model is we have that, those are, and each tier is a different tool. Now what we have is an entire stack that gets embedded into an application workflow where it is part of a context. How do you, how do you play in that world or how do you see that world evolving? I mean, around applications.  Yeah, that's, I think that's the application of everything that's going on. Like on your phone, you don't have like one app. You have like different apps that do different things. Right. And in the past, you'd buy SAP and that would do everything, you know? And I think today in our consumer world, we're using apps to do, micro-apps to do lots that you didn't want a car delivered.  But like most cases, you're using different apps for different things. I think the world of IT is going the same way. However, especially when dealing with data, that's again, like we said, stored in one place, secured one way, you do need a common platform at some point so that you're not copying data into all these different apps separately. And that's, I think, why some of the SaaS, BI and SaaS software as a service kind of stuff where you do have to copy data if their cloud doesn't work so well. So for apps, it's important to allow the developers the freedom to build the app, to be as simple and beautiful and expressive and have that workflow embedded and have the right data at the right time, but have it leverage a data platform. And that data platform needs to be sitting on top of the enterprise system. It needs to be blessed by IT. It needs to be properly set up so that the Kerberos tickets or SAML credentials or LDAP flows all the way from the users, mobile phone or browser down all the way through the underlying data store and back up. This is really actually, okay, we need you to unpack that carefully. Because you said something, but if we look at it from a slightly different angle, which is very, very profound. Someone else was talking about this earlier today, which was, well, we might not have big end-to-end apps, but each app could be a SaaS vendor and we'll stitch them together because it's easier to stitch them together because there's essentially only one version whereas when it was on-prem, everyone had a different version. There's no way to stitch them all together. It's sort of like the... But then your corporate data all goes into all these different clouds. You can't control it. Well, yeah, except you might have some control because there's perhaps a standard way for each one. It's not easy. It's not like SAP. But you were saying walk us through the challenges of that approach. So again, one copy of the data stored one place, one way with one security model. One copy of the data stored in one place. Secured one way, right? So that way if you need to update that data or you need to ask a question of that data or you need to back... There's one place to go. As soon as you start having the data in multiple places, there's an opportunity for parts of it to get lost, for somebody to change something. Oh, the old normalization problem. For it to be stolen. Just copies, yeah. It's like how many different copies of this number? How many versions of the truth? You want a single version of the truth. The best way to do that is to have a single copy of the data. And Hadoop's a very reasonable place to have that copy of the data. But there's lots of other engines too. And that's where a SaaS software as a service BI starts to... Where it's only SaaS software as a service means you got to slug your data out to DOMO, say. We don't really compete with them. Just an example. You send their data to them. Now they have a copy in their cloud. You have to worry about keeping it synced and keeping it accurate. And then you have to secure it there as well as secure it on premises. And that becomes for highly valuable data a bit of a challenge. So we're running low on time, Jess. So I want to ask you, so where does visualization fit? And we were joking a little bit before about the beautiful pictures that you often see on demos of these multi-colored, multi-line, kind of crazy things that make a pretty picture. But I don't really see anything out of it. Well, what role does visualization do in exposing insight to people? Because right at the end of the day, that's what we're all trying to get to. And I think there's pretty common agreement that just having Ivory Tower, you know, super smart people running traditional VI tools didn't necessarily get the right data down to the people making decisions. So how does visualization fit in? What do you guys do different to really make that click? Well, the best visualization is the simplest. It's like Occam's razor for visualization, right? You could show all data points in this big thing, but maybe a user just wants to know what the growth has been from last year to this year. And hey, a bar chart or a line chart or just a single number of percentage. The best visualization might just be a number, one number of percentage sign with a color. If that tells you what you need to know, that's the best visualization, right? So, you know, we keep it simple. We say if a bar chart can do it, awesome. If a line chart can do it, awesome. We don't assume you need some like super crazy thing. And I think the crazier visuals that you see examples of where you look at it. I think what the problem is, is they're trying to be generalized, crazy views of lots of data. But I think what you need in that situation is a custom view of data, but one that really shows you the process. So think of a supply chain. Like wouldn't it be great to have visualization that like shows the farms growing the corn here, shows the trucks taking the corn from the farm to the oil factory here. And then another, so as you can see that whole supply chain in real time, you can see the performance of it at a high level. And then the word zoom data originally came from the fact that you could zoom in deeper and deeper. I want to zoom into that oil factory. I want to see what the operational issues are there. I want to see then zoom into the farm. And I want to zoom into the row of plants. With IoT and stuff, you can actually do this. So I think the fancier visualizations are really good when a human can look at it and go, I get it, that's my supply chain. Or I get it, that's my company. Not when you look, I don't know what that is. I think that's less useful. And so I think the fancier the visualization, the more specific it has to be, at least to a vertical or to an industry or even to a specific company. And we at Zoom Data allow custom visuals to be plugged in. However, we find the best visualization is the simplest visualization. As long as it's rendered really fast, because again, three second rule, we don't talk for three seconds, people stop looking. I like it, I like it. We're too fast and simple. We're too personal and a question, right? Yeah. We're too a question. I just need it for the question. I don't need the whole thing. Fast and simple is the way to go. And that's what we're really espousing. Well, Justin, thanks for stopping by. We appreciate you coming by. Glad you got to see all the lights and the cameras. Yeah, now I know you guys are legit. You're not just a guy with an iPhone as much as you are. This is what I'm talking about. It was a little more than that, but not much. And again, congrats on the funding. Yeah, thanks a lot. Thank you for the opportunity to do in today's market, I think. Yeah, we're very excited, very happy. Very happy to have both of you on Comcast as investors. So Jeff Rick with George Gilbert here in Midtown Manhattan at the Hilton, we're at Spark Summit East 2016. We'll be back with our next guest after this short break. Thanks for watching.