Alright, here we go 82 plus 5 or players pack first one of the night We got the Cameroon kit on maybe this can bring us a 99 rated card. I beg a sports All right first one first one we're starting off with a bang. Yeah Just boards. All right, so on this account. We've got an 81 plus and 82 plus a double 92 and a 94 plus tots All right, we're gonna start with the 81 plus. Uh-oh. What's this? 1090 player picks a yo It's not what I expected to see in the unassigned 1090 plus player picks. Let's go man. I got it. We have to get a 99 from one of these 10. Please EA Sports We're starting off with a W 96 rated cruise for Fada. Unless he wants to snag him I'll let him I'll let him answer in the chat Very solid pull from the first one oof I'm guessing you want PK for the rating here unless you want to try it ward, but Barcelona players back-to-back. Can we get messy? Can we get messy as well EA Sports? Can we get a third Barcelona player as well? Oh now they give us a real Madrid. I'm guessing we're taking Cusarola here. Yep Wow, man. That's a lot of La Liga. It's a lot of La Liga. Watts.  All right, unbelievable Unbelievable. Griezmann incoming. Give me a Griezmann Bale. Oh 97 97 Luis Alberto Unless he wants Bale. I'm assuming he wants Bale He wants Bale. All right He wants Bale for the Panini. He wants the five star four star Gareth Bale. All right. Here we go Dude, that's four La Liga picks in a row.  Hey, at least we're getting like 95s from like all of these man All right, so through the first five we have a 95 we actually did pick up a 97, but it wasn't chosen All right, what's in this one? Oh Black oh Dudes you're getting there's a lot of high-rated popping out of these bro Oh Black fodder up to to be honest people say O block is pretty good in this game as well actually People say he's pretty good All right, four to go e-sports. Give us a big boy Santy again. Oh Chesney the wink Does he want Wamba soccer for usable card? He wants Wamba soccer. He does he does he does not bad, dude not bad One besaka to besaka three besaka. All right, here we go three left Cozola again, dude. Cozola has popped three times Three times this guy's pop man. All right, two to go. Come on e-sports. We need another big boy Dude Do you want the new the new United transfer? I don't know man England's can't score the whole time The transfer man GG emu dude, these are these are pretty average, dude These are pretty average at least you're getting some high-rated for Fada But if you're doing ten of these packs man, you're looking for a big boy and he hasn't got him yet But it's not over last one. Here we go Steve So from ten of the 90 plus is we got vanda beak. Oh block We did pick up a Luis Alberto, but we didn't choose him because he wanted bail You did get to her staggin. That's the usable card even Marquinhos is technically kind of usable, but All right, man. Let's see what we can get from the rest of these packs 292 pluses If I get him a 97 or above then we'll get them a sub. All right double win for him if it's a 97 plus How about that? Italian striker. Oh, no 96 Oh no, oh No He's got the foot with kit though. He's got the foot with kit boys. You know what that means Packlet the pack luck dude Golden boot boy, that's right Loverin is goaded dude. Did you guys see Loverin's goal today? Was it today? I think it was from today Loverin had a pretty tasty goal man Pretty tasty goal. All right immobile and Rafa in a 92 double. Yeah, and Now for a 94 plus tats player. Let's go Let's go Come on. EA sports. Give us a big boy French striker left wing Griezmann yo Another 96 That's a W. That is a W right there man Antoine Griezmann dub dub Brazil guy you did mention Griezmann was coming GGs GGs. He's missing the top paninis R9 play Cruyff Maradona Messi VVD Ben Yetter and Livendoski. All right boys. We got to get one of those lads tonight. Okay Let's go. We're starting with six 90 plus player picks Imagine Having a club this stacked that you already have all these cards, dude. Oh Gosh, man, do you want can I take Fernandinho? No, no, I got to take the lick bro. I Don't know I have to discard a delict. I have to discard a delict Come on. Let's go. That's he There's these better not all be discards, bro No, let's go. Let's go. That's one of the guys he wanted. Let's go dude get in bang 99 rated in September. Let's go There it is dudes third player pick third player pick bang All right, so three duplicates three non duplicates But we get 99 LeVendoski first owner for the panini. Let's go Let's go And then close your eyes chat You didn't see that you didn't see that. All right boys three icon moments con We have to get Cruyff R9 or Pele. That's what he needs for the panini. We already got a LeVendoski, but we're not done Rio for consolation. I think he already has Rio though. It's all Campbell. Nice. Nice All right, so over one Yeah decent pole, but not for this account, bro. We're looking for something better for something better.  Don't you? soul Campbell Has just scored a brace Dda boys. This is DDA packweight DDA back-to-back soul Campbell's unacceptable Unacceptable I agree. All right, lads. Here we go. This is the last one Now we get a W. Please EA sports. Please. Okay resilient Striker to be fair. This guy doesn't get pulled a lot from moments packs. So that's not bad That's not terrible. All right.  I'm telling you I'm guessing you want me to take cruise here connex Bro, we have a 94 plus tots and you have a five mil five million coins We got to get them coins out of there, bro. You got to get them coins out, man Gotta get them out. All right. Let's go.  All right, you're not gonna get stuff get good stuff from any of these Do the 90 plus player picks instead. Okay, please Please How about a boom, baby? Yeah, I would love to put that in too. That's a team of the season. Come on. Oh Her decky. All right craftable, bro 91 rated That means you can do another one. That's what it means GGs Bang or Missong. I agree. I agree, bro. Oh Two blues one pack her decky and Brant. Let's go man. Let's go All right, four more 90 plus player picks for disguised. Let's go Luis Alberto Wow All right, that's high rated for fodder. I guess I'm assuming he wants to be subberto Much better than the 94 plus tots all day. I take these men all that He probably wants Manolis for the non-dupe but Kimmich again 96 rated, right? By the way, who wins tonight Houston in Lakeshore Lakeshow, baby Hashtag Lakeshow hashtag young king Bella Bella 96 rated I'm assuming cruise dude You got a 96 96 93 and a 97 from these bro.  I was like kind of a mix 0.1. Okay, since it's a zero negative 90,000 So some of you guys are about it and some of you aren't A team of the year I beg Team of the year center back. I We just packed Virgil van Dyke, baby. Let's go 99 rated team of the year bang bang Let's go Get in 99 VVD and 99 level in the same day with the ratkits. Let's go Let's go.  Oh, baby. That's sick Hey, Irby, what are we saying? Irby, what are we saying? Yeah, it's not my account boys. You guys thought this was my account. I joined that share play so fast you had no idea All right boys Here we go. Oh Yes, get in I will take a little little host Seth for the club on Untradal by the way W Do I do a 92 plus double or 94 plus tots? Do a 92 plus double or do a 90 plus player pick? You know You know All right share play 92 plus to end it all team of the year, please end me with a team of the year tonight.  How many on my account zero? See Let's go man. What a pull Imagine what that was from a 92 plus. Oh my gosh, man. Oh, no, he's already got him. Oh dear oof All right, so maybe if you want to do a 92 plus double I mean you could do those two if you want the 92 plus doubles don't seem too bad All right, man. Well, you know what I kept the fresh one in your club. That's ridiculous, bro Two pack pulled first owner Cristiano Ronaldo's team of the year. Can I have one of them, please? GGs man G freaking G's bro


Alrighty let's play this in full first. There is no sound. All right, cool couple things just quickly I would try to take out your image plane or whatever you have there in terms of that X and I'm curious if you can create holdouts. So I'm not sure if it's kind of digging through those pieces now right and we should not see it. Is it in front of that piece? Is it in front of that spoon or not? Is it flying behind it and evading? Excuse me that spoon which I have a feeling no because otherwise you would never see it but maybe that's what you wanted to do but then but then this have I don't know I'm not sure because you are managing this is still rough so here are just my my thoughts on this like this is cool I'm a big fan of this is cool and it's cool to see a shot with a plate it's rare very rare actually you might even be the first one in my workshop to ever present something like that it's cool so my first impression is you have to look at scale right hand spoon obviously all of this and the size of the dragon oh and the dragon goes up this feels like a really big dragon and I would look at this in terms of like a smaller bird so it would be more like a lot of faster flapping of wings besides because there's a couple things where generally when you flap like your body's not going to go all the way up starting here like the wings will be around here to here and until then the body's still going to drop see my horrible scribbles so by the time like I said the wings are maybe around here between here and here I'll probably run here that's when the body goes up and not starting here there's not enough wind resistance and and push from the wings I probably also flatten those wings a bit more and bring up the tips but before we go into wing shapes because even through here you can potentially even break the wings more as as the bird goes up because you have flat wings like this as you go down all the way down here and then when it reverses you got the the tips reversing there and then you have a lot of breakage in the wings and then as it unfolds again the tips are dragging until you go the other way and the tips are like this but it's usually you know not flat but you have the other side of the wing coming up is an offset and and you know the height of each wing and like I said the tips horrible drawings tips at the end like this that as the base changes direction that's when the tip will reverse but then like I said you're going to start having a breakage in the wings as they go up and now looking at your rhythm that's good that's good so it's usually slow down fast up slow down because you have all that wind resistance of the flatten wing membrane and everything and then as the the flap goes up as you break the wings there's less resistance going up meaning that up is fast down is slow that being said again looking at the scale I mean personally I would just I feel like you could treat this as a smaller bird or something like this makes more sense like that type of speed but then here again it's all very slow this feels more like a landscape in there you know guys you're going that kind of size but then this speed feels better so it's a bit all over the map in terms of scale and everything and same thing here as you go up like this creature would up hang and drop and see how the body starts to ease in and stop and already magically go up before those wings are actually here so I'll be careful and just offset the timing of the wings so that that's when they flap that propels the the creature back up there just but I mean again I'm not very structured in my critique I'm giving you some animation notes but generally it's just the principle of it because I think the animation needs to change mainly because of size like this is all cool and it's a tricky thing because it's definitely very fast like that the pot there right so as you do this that roots to me would be around here like it's being pulled away where whoa and the head might be higher if that makes sense where the the feet used to be here now the pan gets pulled so quickly that I don't think the body like the whole creature wouldn't go with it like it would be dragged on their way from its feet again so that it would drag a lot and lean backwards maybe that where you feel like whoa it's gonna fall off the pan that's why it starts to jump off and flap if that makes sense but again then I would treat this as a as a faster creature and then also once you go up here it's it's just very repetitive you know I mean like there's nothing really there's nothing really happening where I'll be careful because you're talking here about if you haven't touched the tail that's totally fine rough past on the head totally fine too and you want to make the end a bit more entertaining maybe some backflip works at the scene you just want more interaction with the frying pan maybe and you're concerned about oh you're the clarifying performance so to me it feels like I will probably what is he doing here okay so to me it feels like what if again this is not like you don't have to do this I'm just thinking a lot what if the wings are more folded and more tucked in and something where there is a clear difference here between the head and the wings and the tail could just be whatever and the wings are back there so all the focus is on the head and the head has quick darts like a bird quick darts following that spoon following what's going on in here and none of the flapping so that we always have a clear view because right now the silhouette's gone I'm not quite seeing I'm just seeing movement I know it would be kind of neat again subjectively if the wings are still back there and just the head the head just keeps looking like what is this what is going on here and only when this happens and like I said the feet are being pulled away from the center of gravity from the root and everything that's when the wings unfold like whoa what the hell and then maybe it might fall even a bit and then flap flap flap flap flap then you can have this but then that would make the flaps a bit more like this even that is too repetitive right it might be over here something like that where it fell like whoa and this is big arc because of recovery and then oh what's going on here what's going on here it's almost like this right you fall back big recovery and then oh okay I recovered okay let me get closer and what is going on here and then get into that but the question is is that to eat like what's the reason for this and like I said before you want to create some holdouts and we need to understand where this character is in relationship to everything here in front of it not in front of it if it is behind it it will be covered all the time so be careful about that like to me it could almost be something where like again because because it is a smaller creature you have the the license to be fast and kind of dive in and pick up things what if it's here falls back goes up and maybe a moment like this it would dive in and pick a piece and then fly over and then when this happens get back in pick a piece and then you can just create a geometry and pick that same color and then just pretend that it's eating a yellow or red piece there's a lot of time here right so does this falls back recovers swoops in to eat yeah I mean you can you can you can time it in a way where you're here whoa goes down it does the whole recovery thing of whoa and then so we have that and then maybe that's the last big one maybe or maybe this one that's when it swoops in and maybe you know it could be an interesting way of flying through it or kind of around it while the head is still grabbing it and then eats it and this could be you know you can do something where it might swoop in and go back a bit swooping go back a bit and then go in if that makes sense like Sue I'm over the map here let me just explain my my crazy thought process here flies down falls down flies up does a big recovery and then in place to kind of reassess the situation and then might go in once and then maybe go back like oh that was not the right timing let me try now like it's almost trying to get in at the pick a piece and then it goes in and this could be one two or one two three where you go always a bit closer than in and then you see how it comes down here I think that could be the moment where a creature could be here and flapping in place and like you know putting that piece in its mean it's throw and eating it and then when this happens that kind of wakes up the creature you know like now wakes up he's not sleeping but like oh oh it continues and maybe maybe changes a bit where it flies back a bit here like it was eating here but then here's that sound goes back a bit to not get attacked and waits a bit and then when this happens like oh it's starting again and then it could either go in again for another bite and what else could you do I mean it could be I mean that could be an idea I'm just concerned now that are you going in picking a piece oh another one goes in and then that's repetitive by doing this again not sure if you have enough time right if you go this and it's eating here or here this happens flies over here like whoa whoa whoa and then sees this and then dives for another one which would be maybe here like he would even you have potentially caught on action right as it goes through but I mean that could be something interesting to hold out where like the head is here open mouth has a piece wings through there and you can just animate little geometry pieces flying off as it pierces through this I don't know it's thinking out loud this is I think these are some of the things you could do I mean or you just cut the shot short you do all this falls recovers whoa and then like I said then it goes either it's like let me eat oh that was not great or let me try let me try all right let me try and have that type of contrast and timing and then what it's doing it's eating eating eating and then when he or she does this it could be just a little whoa too close and flies over there and then maybe cuts or or again this goes in and is able to grab it and then eats and it ends with a nice swallow and then that's it where you don't have to do the full shots potentially um but that's kind of it's a rambling lots of thinking out loud I don't know let me know what you think you can pick and choose obviously do whatever you want with the shot these are just my thoughts on what I'm seeing here thank you all right there's an email you can sign up you can start whenever you want you can submit whatever you want you get 16 submissions either way a like and subscribe would be awesome all right thank you


Hey guys, this is rich from rich TV live, and I just don't know what else to say I'm the only guy that is talking about Bevo agro BVO, and it just keeps going up every single day Like this chart is a thing of beauty and I've been bringing it to you guys for a month And it since I brought it to you guys it was at $2.30. It's now at six dollars and sixty four cents up another 7% today look at the MACD on this thing, which is the second Like list here with the blue and the yellow and orange and the red see how it's all pointing straight up Yeah, this bad boys got power like unbelievable amount of power so You know it's funny that you know stock goes up every single day and people don't want to buy it and they say to me Well rich it's going up. You know you told me to buy in the red Well, there's some there's sometimes guys we have to buy in the green Okay, just because I say buy in the red selling the green It doesn't mean that you always buy in the red if you have a stock that's a hundred percent buy It's got crazy amount of power. It's got a small float and it just keeps going up and all the indicators are pointing up I mean you've got to learn to trust your eyes So we see this go from two dollars and thirty cents once it hits six dollars and ninety cents It will be up two hundred percent since my first original alert my first original alert Okay, I'm gonna show you guys my first original alert. Here we go. This is why I always tell you guys in the markets Don't panic no reason to pan Okay, never a reason to pan ever ever ever ever So many commercials everywhere all the time So You just search BVO Rich TV live You will see here bevo aggro. Look at all my videos that I've been popping out Look at the first one that I popped out before legalization Before legalization I call this thing It's true Who brought it to you first Zanabas Bring all the winners first it's true One of the largest cannabis cultivation footprints in Canada, that's right Zanabas is part of the largest footprints in Canada They're already making 32 million a year guys Wow 26 million shares Want to see the price Tripped out Man two dollars and thirty cents in the red what I brought it to you guys I don't know what to say man BVO now remember risky life is strictly for education entertainment purposes always do your due diligence Always do your research this could plummet tomorrow, and then you guys will all say oh rich You're a pump and dump or you call this you pumped it and you dumped it man I have not been compensated by this company. I just found this pic. Okay. I found this pic because I find you The winners first and we are free Bevo agro ink up another seven percent. I don't have nothing else to prove you know who's the best I am the best. I will challenge anybody We will find winners even in a week market This stock has gone up while everything has gone down This one stock has gone up and I brought it to you guys first And I will continue to bring it to you guys until you guys understand You don't need 20 picks if you find just one and it goes up 200 percent. You don't need anything else You understand what I'm trying to say to you guys I hope you guys are reading through between the lines here if you like this video smash the like button comment down below This is your boy rich if you ain't winning you ain't watching we found it at 230 I predict it's going to ten dollars. You heard it here first Elmo Peace I retire the best the champ. Have a happy Thanksgiving. Have a great long weekend Love you guys though. You guys none of this is possible man. Hit that like button Hey, subscribe hit that bell for notifications If you want to get all the news as it's happening when I bring you these winners that just keep going up And they making you rich you got to be smashing the like button man Seriously, like I don't know what else to say man. I challenge anybody Anybody who brings you these winners 200% winner. You want to talk about winners, man? You know what I'm gonna show you guys something Seriously, because you're watching Because if you're not winning you're not watching so I'm gonna bring you guys the the the juice you want some juice You want to see why we're the best? Let's show you why we're the best look at this Because we bring you the winners before they explode so show see how I showed you the VO before it exploded Oh, yeah, look at this bad boy. It's at a 52% 52 week high and Look at the strong by 100% by top 1% signal strength It's been sitting like this at top 1% signal strength all week going up 10 20% every single Day, look at this up 331% in the last year 207% in the last three months since I brought it to you guys and 95% in the last month alone Who's the baddest? Your boy rich. It's true. I only bring you guys the truth, man. So hey, man You want to hate I got a song for you, but if you a real G then you respect You respect the game if you have any respect for this game, you know Who's the baddest in this game? Numbers don't lie, but people do a You to put on a show you can have The real one Who's the best Who's the champ? Until somebody takes you out my throne. I'm the champ. I'm the best


Okay, okay, James P. Madonna here for Megalike 21. I just want to say that whoever invented the low-flow water faucet and showerhead should be lynched very slowly, publicly, until dead. This is what many businesses in the United States have. Okay, the cheap bastards, the CEOs of these corporations don't have a large enough salary and a golden parachute. They have to install these trickle down, and I'm throwing a political hint, trickle down low-flow faucets. And it's not just commercial restrooms, but it's also companies like Moen and Delta and other American companies. They should be lynched.


We want to welcome everybody here today for this, this conference. This is a great day in the City of Congress in County because it's an opportunity to put a brand new piece of apparatus into play. Before I move on, I want to just thank our leaders. We want to thank our Mayor, Mayor Benjamin, our Councilman, Will Brennan, our Councilwoman, Teresio, our County Administrator, Mr. Brown, and also our Assistant County Administrator, Dr. Thompson. Truly, this cannot happen without the collaborative efforts of all these people. But not only on these people, but also our community, our firefighters, and all of us that have a part in, want to give a special thanks all to our City Manager who could not be here today. This is a brand new piece of equipment that appears. This truck costs a little over $1.2 million. It's going to serve the five points area, also the Shannon area, and any place that this truck is needed. So it's not unusual for this truck to not only be in this neighborhood, but be in neighborhoods abroad, whether it be in the city or whether it be in the county. So we really appreciate the opportunity to put this into play. We always decided in the county, always try to make sure that we got the right piece of equipment, that we got good equipment that can be used to serve this neighborhood, this community, this city, this county. But also something that our firefighters can be proud of. We're extremely happy to be here today to put this into service. Before we move on, I want to give all our elected officials and managers the opportunity to come say something if they so desire. I explicitly told the Chief I did not want to speak. He never listens to me. This is just a, we live in such a challenging world right now. As we deal with the pandemic, as we work our way through the economic disruption and social unrest, and just taking a moment to celebrate. I think something that couldn't be any more American than the men and women who served the Columbia Ritz and Fire Department, putting a new piece of public equipment in the place that keeps all of our families safe. It's what happens, you get great partnership between the city and the county, and you get true public servants who are dedicated to making sure all of our families remain safe. So we're just glad to be here. We're so glad to be here with Councilor Gretin, Councilor Taratio, and Dr. Thompson, Mr. Brown. I mean, we all work every single day to make sure that Chief Jenkins and his entire team, that they have what they need. And it's just a good day. A good day and so happy these beautiful babies can join us as well. The reason why we all do what we do is so they don't have an opportunity to live up to their full promise and potential in this great country. So glad to be here. I'm going to step aside. Chief, thank you so much for the beautiful weather for us to push in this beautiful truck. It's great to be surrounded by city and county elected officials. And I want everybody in Richland County, in every municipality in Richland County to remember it takes a city county effort to provide for wonderful fire services. And we have to keep that a priority. So we look forward to continuing that strong relationship with our friends of the county to provide the funding that our wonderful city, Richland, Columbia Richland Fire Department needs to do their job and do their job safely. So again, Chief, thank you for having us out today and thank you to our first responders, our fire professionals to keep us safe. Thank you so much. Thank you all for coming here to witness this amazing tradition. This is one of the most fun things that I get to do is to help push in these fire trucks. It's amazing that we get to have this brand new piece of equipment to serve the county and the city together. So, yeah, thank you all for being here to help us celebrate. Just very briefly, I just want to echo on what the mayor said and what Mr. Brennan said. It's important that we have a partnership. And our presence here today, Council Member Teratio, Dr. Thompson and myself, the county administrator, just a testament to our commitment to serve as good community partners with the city of Columbia, as well as our other organizations that we serve with. So again, just want to say thank you for the opportunity to come out and show the men and women of the fire department that those who are sometimes are not seen, that we support you, that we want to make sure that we're here and our representation here communicates that to you on today. So again, thank you, Mr. Mayor. Thank you, Chief Jenkins. Thank you, members that are represented here today. Back in the day when they had the horses pulling the fire apparatus, when they get a new piece of equipment or when they get back to the station, the horses couldn't back the truck in, so they used to have to push it in. So we have adopted that tradition when we put a new piece of equipment in and we're going to actually push it in to signify a new apparatus going in the service. And all before we do the push in, we all offer up a prayer for both our men and women that ride these trucks, the community that it served, and just pray that we get good service out of it. I'm at this time with Dr. Thompson with my leading us in that prayer. Absolutely. Let us pray. Lord God, we're just so thankful for another day. But thank you for the investment that is being made in this community here in Columbia Richland County, Lord God. We thank you, Lord, for the men and women who will be using this apparatus, Lord God, please continue to keep them safe, Lord God, continue to guide them and provide them with all the necessary protection, protect their family while they're away from their families, Lord God. And we'll forever give you the praise, honor, and glory. It is in your name, Jesus Christ, we pray, amen.


Hello everyone, and thank you to engineers.sg for that live stream. My wife and daughter will be watching me say this talk for the fifth time in two days, and I hope you enjoyed it just as much. So, hello. Good afternoon. I am Tim Riley. I'm part of the DryRB and RomRB core teams, and I work at ISLAB over in Australia where, for these last nine months or so, we've adopted a whole new approach to building our web apps. So, what got us here? Well, a good many years of this, of following established community practices for Ruby, building our MVC apps, and still ending up with projects that became more and more difficult to work on over time. We wanted to find a methodology to improve this for us. And that's what I'm here to talk to you about today, about building better web apps with Ruby. We'll cover a lot of ground today, and we'll look at a lot of code, but I hope I can give you a sense, a new sense, for what's possible with Ruby, and inspire you to look at web apps from a fresh perspective. We'll look at three things today. We'll look at DryRB, a new collection of modern single-purpose Ruby gems, at RomRB, a flexible persistence toolkit for Ruby, and at Rota, which offers an expressive approach to routing. And we'll also combine these with a fresh approach to Ruby itself. With a hybrid of functional and object-oriented programming that we call Functional Ruby. So, let's start with that. Thinking about functional programming, what are a few things from it that can help us with our Ruby? Well, firstly, if we want to orient an app around functions, we need to treat those as values, so that we can pass them around and combine them to build larger systems. Our app is also simplest to work with if we favor immutability wherever possible, avoiding internal mutable state in our objects. And we also like to avoid external side effects wherever possible, so most of our code can be run in any sort of context. And together, this helps us better understand the flow of data at any part of our application. And we can satisfy all of these criteria by building functional objects in Ruby. Let's take a look at one now. This is the simplest possible functional object. It's a class, it has a core method, just like Ruby's own procs and lambdas. That core method accepts an input and returns an output. That's it. It doesn't mutate the object's state, and it shouldn't mutate the input data either. Functional objects in this way make a clear separation between data and behavior. In fact, the only data an object like this would hold as internal state are its collaborators or any other sort of static configuration. This is a state that we'll never need to change. And we set this state via the constructor only, just like we do here with this repo variable. This is, in fact, an example of constructed dependency injection, a classical object-oriented design technique that works to great effect here with our functional objects. And in the end, what this means is that we can create a functional object just once but then use it many times over. So it means, regardless of when or how these objects get constructed, they can then go on to become collaborators for other objects in turn. So let's take this technique and build a whole web app around it. We'll build the typical blog post example app. We'll build a get request so we can see reading data and a post request so we can see writing data. And we'll look at these step by step, building them up layer by layer. And in doing so, I hope to give you an idea about the small pieces that combine to make the general shape of a larger working app built in this way. So let's start with the get. Let's get some articles. We'll need to do five things to tackle this. We'll need to handle routing and HTTP. We'll need to look at object dependency management. We'll have to render some views. We'll need to query the database. And finally, we'll need to model the data coming back from that database. So let's start. The first thing we'll need to do, obviously, is handle the incoming HTTP request. And to do this, we'll work with dryweb and rotor. Dryweb is a high-level organizing gem that forms the umbrella around our application. And it works with rotor. This is what we'll use in our examples today and is what exists right now with the gem. But the plan is to actually go on and support other routing libraries as well. So in this way, we're not talking about a one-size-fits-all stack here. What we're talking about is a collection of libraries that work really well together to form a larger, more coherent web app. So we can start by installing the gem and generating a new project with it. And then we can go into a route. This is rotor at work. And rotor works by building up a routing tree. And it works by basically lots of nested blocks like this. And in a larger app, this can make for some really concise routing logic. Because we can set context in the outer blocks and have that context available for any block that's nested within it. Rotor is also a flexible routing toolkit because it starts with a minimal core and then offers plugins for us to layer on any sort of behavior on top of it. And that's what we're using here to provide this view method. This view method is provided by a plugin and we're using it here to render our index of articles. And this index of articles is provided for us by another object. And this is where dry component and dry container come in. Dry component sits at the heart of every dry web app. It is a high level organizing system for our app's objects. And it uses a simple naming convention to register those objects with a container. That container is provided by dry container. It's a lightweight inversion of control container for Ruby. And what it basically does is it allows us to access those registered objects later at any other time. So what does this actually mean? Well, it means when we call view with articles.index, it's translated into a fetch from the container with a particular identifier. And what's returned to us is a pre-built object, so we can go to work on it right away. And that pre-built object is automatically made available to us based on this simple naming convention from a Mattsing source file. And that convention doesn't use any magic. It uses basic language-level Ruby features, good old require and load path, which means it's easy to understand and easy to put to work for us. So after all of this, we have a view object. And this is taken care of for us by DriveView. DriveView is a helper-less, data-oriented view rendering system which combines functional view objects and templates to render our views. And just like the router, we plan to make view objects completely independent, or they are, so you can use whatever works best for you. So here's our index view class. What it does is it specifies a template to use, and then in its local method, it returns a hash of locals to use within the template. Now, right now, this isn't quite enough to render our list of articles. We need to fetch them from the database. And this is the job of a dependency. That dependency will be made available for us by DriveAutoInject. DriveAutoInject provides a nice way for a class to declare its dependencies to have those dependencies resolve for us from our container and made available for us to use anywhere inside the object. So let's go back to that view and set up a dependency. Here we are. We're importing the article's repository as a dependency in that include line there. And then inside the locals method, we're using it to pass our listing of articles to the view. How do we get that listing? We get that using ROM and ROM repository. ROM is a flexible persistence toolkit for Ruby. It offers explicit, customizable abstractions that we can use for every step of the persistence process. It's also built around functional Ruby principles, and it's a really big part of what makes these apps sing. And why is that? Because ROM lets us define a clear boundary between the core of our app and our persistence layer. It lets us hide away the nitty-gritty details about persistence and let our app's core objects stay clean and focused on their own jobs. How does this boundary look? Well, here it is with our repository class. We're saying here that we want to work with an article's repository and we define a listing method which specifies that we want published articles only in this case. Now, ROM is actually a multi-back-end persistence toolkit. And since we want to work with SQL here, we'll use this SQL adapter, and we'll use that adapter to set up a relation. A relation is an example of ROM's explicit abstractions that we can use at various points. What the relation does is define a mapping to a particular database table or database view. And here we're using it to define a published query method, and that specifies that we only want the rows where that published column is set to true. This is a great example of ROM's explicit design because it allows us to define exactly the persistence API that we need for our app and nothing more, which means that every interaction with the database goes through code that we've written. So we have full control over those interactions, and that will be a great comfort as apps grow larger and get worked on by many people. So with this relation in place, we can go back to our repository, which will now return the right data. And this is a very simple repository, of course, but in a real app, they'll grow fast because we'll set up methods for each unique data requirement in our app, and we can also compose data from multiple sources. So this is why it's helpful to keep that low-level query logic encapsulated back in those relations. By default, repositories just return plain struct objects, but we would like them to come back as objects that are more specific to our domain, and this is what we do here. We're specifying that we'd like our objects to come back as article entities. So how will we make these entities? Well, we'll make them with dry types. Dry types is a type system for Ruby, not a language-level type system, but more something that just gives you some stricter typing in the places where you care about it the most. And where we care about it in this case is when we're modeling our article entities. We want them to contain exactly a certain kind of data. And dry types here won't actually allow us to even initialize these objects within valid data. So it means when we have an instance of an article, we can be 100% confident that it will contain the data we expect, which means there will be much less second-guessing when we're working with these instances, and it makes building things like our views much more simple. And unlike active record models which we might be used to working with, we don't mutate these structs by convention, which means they're much simpler to pass around our app because we don't need to worry about accidental side effects or accidental mutation along the way. So back to our view with the repository in place, everything is complete. So we can make our template, call the view object, and get the output we want, our list of articles. And one thing to note here is that this is completely stand-alone view rendering. So we can use the same mechanic anywhere, not just for rendering pages, but for anything we want, even in things outside the normal HTTP cycle, just like rendering emails, for instance. So we can come all the way back to the top, back to our routes, and our job is done. We've got our list of articles. What did we do again? Well, we used Drive, Web and Rota to handle routing and HTTP. Our object dependency management was taken care of for us by Drive Component, Consainer and Auto Inject. Drive U rendered our views for us. We used ROM SQL and ROM repository to set up our database queries. And finally, we used Drive types to model the results from those queries. So the get request is done. We've got a list of articles. Now let's go and create one. We'll have to do a few different things this time. We'll need to validate input data from a form. We'll need to write commands to write data to our database. And we'll need to take care of success with the handling of our command. So we can go back to our routes, adjust them to accept a post request this time. And here we're using a different Rota plugin we provided. It's Resolve. And what it does, it just fetches an object from our apps container using this identifier and makes it available for us to use. In this case, we're working with CreateArticle, which is a functional command object that will create our article for us. And then we're calling it with the form post parameters. And again, just like before, DryComponent takes care of this for us. So we resolve this object. It comes out of our container as a ready to go instance that has been created automatically for us from this matching source file. And here's how this looks right now. A simple functional object. And right now, all it's doing is just validating the input data. And to actually make that validation work, we will work with DryValidation. DryValidation is a powerful, stand-alone validation library. It lets us model our validations in precision and work with any kind of data in any kind of context. So here's our schema. What does it do? Well, it requires that all four of these fields are filled. It sets an expectation that our title should have a minimum of three characters. And it also sets some type expectations for the published and published at keys there. One will be a boolean and one will be a time. And DryValidation uses dry types under the hood to give us some powerful coercion behavior around form posts. So we can start with this stringy form input where everything's a string. A boolean is represented as a one. A time is also a string. And then after we've run it through our schema, we actually get coerced data. We get a proper boolean and a proper time object. So this form schema actually acts as another critical boundary in our system. It handles all the baggage of HTTP and then out the other end, it lets our app's core objects work with properly structured and properly typed data. And we'll also get friendly error messages when something's gone wrong. We also get hints about checks in the schema that weren't able to be run yet. So in the end, the user will have all the information they need to correct any errors. So we can go back into our command, check for the success of this validation, and if this is successful, we can save the article to the database. We've done this by injecting our repository again. And then on the repository, we're calling create with the valid form data to persist the article. How will we do that? We do that by going back to ROM again. A big part of ROM's designs is that it separates queries from commands. And again, this is about giving you explicitness in your setting up your persistence layer. It ensures that we create a persistence API with a surface area that exactly fits our app's requirements. So if you want to create command for articles, we'll need to set it up. So to do this, we go back to our relation. This is the thing that models the table. And we'll define a schema to act as the canonical representation of that data source.  Not everything can have schemas that are inferrable, like from database tables. Now, if you're using ROM in a larger framework, where it's more integrated, for SQL, this might be taken care of for us. But it's still great to have this facility available to us because it means we can project data freely in a way that's optimal for any part of our app. But in the end, this schema is used here to ensure only the right data is written back to our table. So we can go over to our repository now with the schema in place. And all we have to do is declare this create command. We'd like it to offer a create command for us. And everything else is taken care of for us. It goes back to that schema and makes sure the data can get written right through. So now, our create article operation is fully functional. But, and we're doing one more thing here in that we're returning an instance of the article as the final result just so we have consistency in what our app's objects return. But so far, this is only handling success. We want to handle failures as well. So this is where Monads come in. Monads are another concept from functional programming that we can put to good use here in modeling success and failure results of our operation. And we'll do this with dry Monads and dry result matcher. We can integrate them into our operation class like this. And what it offers us is meaningful workable output for both success and failure results. And we do it by this. We do it by wrapping our return values in either right or left objects. Both of these are instances of the either Monad. But in practice, it just means that in the end, the single return value we get is something that has a consistent API regardless of whether it's a success or a failure. And it enables things like this. This nice flexible result matching that fits really well into our rotor routes. By working this way, we actually elevate failure handling to be a first class concern because working with failures becomes just as easy as working with the success case. And in this case, we do two things. Success, we redirect, and on failure, we re-render the form past the validation object back so we can present the user with some errors. Also notice here that this is the only place in this request that we've actually handled anything to do with HTTP. So just like our repository was a boundary between our app and the persistence layer, the rotor routes here are another boundary between the world of HTTP and the world of our app. Our app's objects only need to concern themselves with doing their job and then returning success or failure knowing that back up in the routes here they'll be handled as appropriate for the request to be completed. So we've posted our article, this request is complete, and we used dry validation to validate that input data. We went back to ROM to create some commands and then we handled the success or failure results with dry monads and dry result matcher. But our app is not really done here. We've done these two requests, and as we know as developers, our work is never done. Clients come with more work for us and we have to satisfy that. In fact, dealing with change gracefully and effectively is a key part of our jobs. And everything we've done so far today has been to prepare ourselves just for this very thing. So let's look at some examples of change. Say we've got a request to make a new article form for admins or power users. Well, to do this, we can do objects in a new template. And then we could decide whether we want to reuse that existing create article operation or make a new one that more appropriately satisfies our requirements. And either way, we can be confident about the changes we're going to make here. Because in an app built around loosely coupled objects, focused on single tasks with external dependencies injected, writing tests becomes very easy. In fact, we can do much more in true unit test style in complete isolation because our dependencies can be passed in as test doubles which lets us focus more on the code under test rather than the world around it. And this is particularly useful for things like database backed applications because there's no need to globally stub out database calls or anything like that. And the end result is better faster tests and more confidence in our work. In fact, Dry Component does a little bit extra to help keep our tests fast. When we use it and run any single individual tests, the components in our app boot up completely empty. They don't load any files. And then when the test runs, the smallest possible set of files just for that test to complete will be loaded. This means that individual tests run in the minimum possible time and it makes for a truly responsive TDD cycle. So that change was fairly straightforward. Let's look at a more complex one now. Say we want to email some subscribers a notification when a new article is created. Well, there are various ways we can handle something like this. But we have a truly elegant one at our disposal here. We can use dry transaction to make this change. We've already laid some groundwork in our app. We have a big container of functional objects. Each of those objects returns a left or right monad for success or failure. And with this in place, we have everything we need to start to combine these in interesting ways. Like using dry transaction here to model this as a business transaction. How it works is that it is a sequence of functional objects called one after the other with the output of each object becoming the input of the next. And this sequence continues as long as each one succeeds. This is the perfect arrangement to satisfy this change. All we need to do is create a notify subscribers operation, run it after create article, and it will only run if that initial creation has succeeded. And then, outside of this single transaction, these two operations are not coupled in any way, so we can reuse them as we need in any sort of context throughout our app. And dry transaction also offers an enhanced result matching API so we can hook into specific failure steps and handle them as we want, which is a really powerful new feature and it's been made possible by the design choices we've made And what are those design choices? Well, it's a decision to found our app on simplicity. What is simplicity here? It's an app that's built of many small components, each with a single focus, and we combine the best of object-oriented design principles with the data simplicity inherent in functional programming. This makes for components in our app that are therefore easy to understand, easy to reuse, and easy to test. And I think they're actually really easy and natural to write in Ruby code as well. We've also taken care to design for our app's domain first. This isn't a web app. It's a Ruby app first and foremost that just so happens to have a web interface around it. We've taken note of the critical boundaries in our app, the boundary between HTTP and our app, and the boundary between our app and persistence. And we built around them appropriately, leaving the core of our app free to focus on its main tasks on its own specific domain. And in working towards this goal, we've chosen tools to serve our app first rather than the other way around. We haven't contorted our app to fit the requirements of some external tool. So by focusing on simplicity, by focusing on our domain, on proper boundaries, on small components working in concert, we have an app that is easy to change. We have created a change-positive architecture. And this is the crucial advantage we create for ourselves as programmers, because this is what we have to deal with every day. A change-positive app is maintainable. We can easily understand it by starting at any point and looking back and forth, and we can adjust it as required just as we looked at earlier. A change-positive app is sustainable. Because of our design, we can work on it just as easily at day 1,000 as we did at day one. And this makes a change-positive app joyful to work on. And it's this sense of joy, this happiness as a programmer. It's that visceral thing that brought so many of us to Ruby in the first place. So we owe it to ourselves to carry this joy throughout every part of our jobs to building web apps, whether it's starting something fresh or starting on something with a rich legacy. And by choosing to work in this way, we've not only created an investment in our app, we've made an investment in Ruby because a healthy future for Ruby is a diverse one with a diverse range of competing tools and techniques. This is what drives innovation in our community and will give it life for years to come. Now, everything I've showed you today was put together by a hard-working team of volunteers. So I'd like to thank at this moment Piotz, Sonica, and Andy Holland for their work on DryRB and RomRB. Also, Jeremy Evans for ROTA and to all the contributors to these communities. They're on the rise. People are adopting them, people are contributing, and we'd love to see you there too. And it's not the only community of this type in the Ruby world either. You may be interested in checking out Hanami or the Trailblazer project. Both of these integrate with various DryRB gems and they offer their own unique takes on web app development. And one of them may be just right for you. This is my DryRB. I have a web page full of resources to go along with today's talk, including slides, links, and real working open source example apps. Please buy me at the conference. I'd love to talk to you. And of course, I have stickers to give away. Many of them, so come find me and let's have a chat. Thank you very much. I think we can open one question for Tim. Anyone? I want to build apps this way. I'm going to go to the Bitly, draw a RDRC, I'm going to learn about it. What's going to be the first speed bump that I hit, because you painted a lovely glorious picture, and I know that a lot of it is going to be easy. But when I hit that bump, if I hear from you what I'm going to expect, it might make it easy for me to get over. I think what it might be is that, yes, what we're doing here is doing a little more of the work than the pieces in our own apps that might have otherwise been taken care of for us by larger frameworks. But I think that so you might find there's a little bit more friction in getting those first few features out, because you've got to perhaps discover some of your own design patterns. Some of these, well, these libraries are young, so you may hit up against rough edges, but we do have really active communities, and they're getting better day by day. So I think we're going to go through those early steps. And also, Rom, compared to Active Record, for instance, gives you a lot more power, but you also have a little bit more work to do, and you sit a little bit closer to the SQL. But just like everything else, this isn't building our apps so we can be productive in the first week. It's building our apps so we can be productive for years. So I think it's keeping that end goal in mind, and knowing that the decisions we make today set us up for making our life much easier in years to come with the same code base. Thank you very much. Thank you.


So I'm going to talk about analytical dashboard and how do you build them at scale. So this is one of the effort that we have in before that flip card before for information is our analytical platform which we are building so you can check it out at our booth. Okay, so what's so special about that everybody has seen live dashboard and they know how to build it but they are two different differentiators here. We wanted live of course we will want our dashboard so we actually updating with live information as it's flowing in so you have clicked an order. You have browsing our website. We want to create dashboard that update that reflects the latest information that is happening on our website. Second thing is analytical. So most of the dashboard are not analytical in nature what they are used mostly for its monitoring. Yes, we want them to be used for monitoring but we want or we would allow our users to actually drill down to filtering and do other sort of things that they normally get. They cannot normally do it on a regular dashboard. So these were the two motivating factors. Once we would want to give business users the business actually what is happening inside flip card and second thing is we would want to do we want to make them powerful enough you don't want to we would want to enable them to actually go and drill down the data a little deeper. Okay, so what we have what we have as a system so flip card is largely has adopted so a service oriented architecture. So what we have is a lot of services. So these services actually they are microservices services that take care of usually a very small part of the process. They actually deal with certain a couple of entities in that process and what is happening is that people are just they are orchestrated written that go and update entities to update the process data and these services they what they do is they generate event in the process they generate update event what we have is a lot of live event so by the way those who are those people who have a question that why do you need event? Why don't you go to the database and pull out all the information first? It is going to be very cumbersome. Hundreds and hundreds of services going to each database every some few seconds and pulling the data out of the database is not going to scale at all. So what we have what we have done is that made sure that services they push event they push update event in a well-defined semantic format. So once we have those events there are certain challenges that there are certain challenges in building live dashboard. It's not very simple that you just collect those events right to some processing pipeline and just sort them in a database and then start doing query these are the three major challenges that we start we face when we start building this analysis of them first is metric definition. So these are not just just a random kind of event. Okay, so these are a function of fees. So like like I told you in the previous slide what we have is a set of services and these services are actually dealing with entity and they are generating update event. So metrics are not just the counts of how many events have happened like this is the common case in when you are doing web analytics number of page views number of sessions number of checkout then you have a conversion funnel and you are done. No, that is not the case here. What you have is event that are actually associated with an entity that have changed the state of an entity. So you are dealing with set of entities in a in a certain point in time that even so what you have to do is you have to take these events you have to find the entity make the changes reflect on these entities in our data warehouse and actually extract the field relevant field from there and then on each event or a batch of event we have to statistical analysis. So suppose we want to calculate the average we want to calculate the percentile. We want to calculate a histogram we want to calculate the distinct values. So we have to collect a batch of event there and then associate these values to a set of dimension. So every event will have set of dimensions associated with the dimensions actually set the context of the update. So for example, when you place an order the dimension would be who was the customer? What is the product that he bought? What time he bought it? How many of them he bought it? So these are the dimension that we get with each event. So we have to actually calculate the metric extrapolate or basically report it on certain dimension. So the second is scale. So when you have took as in this slide again, let me put this slide again since they are hundreds of microservices which are catering to 10,000 of request for second from the user in the process. They start generating some lack of event 100,000 of event in the process. So we get into the second second set of challenges with the scale challenges. What you have to do is you have to start getting the dimension. So first step is that you have to actually merge the event to associate that entity that's the first time first thing and then you have to find out the dimension from these entities look up into the dimension table find the actual dimension. So for example, a product a product is usually identified by an ID and even right, but that product actually has multiple levels of dimensions within itself for example, color for example brand for example form factor. So there are n number of attributes associated with these dimensions and we would like to provide filtering and flashing and desing capabilities on those dimensions as well. So what you have to do is whenever an event comes in to your system, you have to go back look up the dimension that is the all the associated dimension and actually ended in this the event at hand. Again thing is that once you have the event your process the event you have to write back and when you're when you're writing back since there are so many events you have to you really need a data store which can support high report right. Then while querying these are dashboards you are updating at a very high frequency say one minute and it does say 5,000 users that are pulling updating these metrics these dashboard what you need is a data store that has low query latency. So in order of some few milliseconds not more than that so what we have what the requirement that came to us was around 10 millisecond 10 millisecond is the cap for 99.9 percentile under that you have to have to serve the metric for the dashboard and typically a dashboard doesn't contain just one measure. It contains a lot of measures and when the entire dashboard refreshes for say 5,000 users then it actually is more than 5,000 calls to you. And finally since all our measures are actually reported against the dimension you need a database that support storing measures across a dimension. That means that if I want to filter on certain dimensions or certain combination of that dimension the query should be fast so there should be an inbuilt support for multidimensional queries into the data store itself. Then reliability since it's an analytical dashboard that means it will be used can be used by analysts who are monitoring the progress of a process or it can be even used by control towers in our supply chain where they are actually monitoring the entire supply chain pipeline and they would want to know what is happening in the supply chain pipeline. Reliability is a very big challenge for us. So most of the dashboards or most of the dashboarding techniques what we have seen or the architectures we have seen they don't care about reliability that much what they talk about is a general trend of what is happening. They don't talk about okay, what this is a general trend. What is the actual values there? What we have the challenge at hand was will dashboard that are actually accurate and to meet that accuracy standard we have some challenges that we have to maintain the consistency of the data and we have to be call tolerant. So to give you a simple use case around consistency again, we have event events that are update events that are changing the state of the process. So you have an event a occurring at time t1. It is going to trigger update in other entities say even e2 in the same time frame when you are joining or when you are aggregating data across these events. You have to take into account that I am aggregating or joining these events again the similar event. I cannot say that I am going to join an entity at time t1 that update at the time t1 with an update at time t2. No, you cannot do that because the results will be wrong. They will not reflect the current state of the process as it is okay and fall fault is something that you have to deal with in a any distributed or cloud environment fall can happen at various places. Your system can go down your source system can go down. Your own analytical system can go down your data store window go down so every place is wherever there is a downtime we have to anticipate that and we have to actually built in fault tolerance into our analysis system and usually again coming back to the traditional dashboard. This is not the case if the fault occurs because simply accept that fault and move on because again that they are used for monitoring purposes and left for analytical purposes. So having said that what is the solution so real time let's scale is equivalent to stream processing is equivalent to Kafka plus Tom that is that is a standard set of technologies that everybody use. How many of you know Kafka and Tom good number of people for those who don't know. Let me give you a simple introduction. So Kafka is a queuing message you that was built by LinkedIn for a similar use case where they want to gather a lot of event from a large array of servers and do analytics on that. It's reliable it has actually with the 0.8.1 isn't 0.8 release they have also they are also supporting their application building in more reliability into it. It distributed distributed at score the positioning of event was built-in you when you create a topic in this message you you have to create partitions for that topic and you can distribute these partitions over multiple brokers second thing is Tom again Tom needs no introduction but still Tom was built by Nathan Mars who is currently at Twitter. I think he has left Twitter. I don't know about that but he built it for stream analysis of Twitter stream it again built on distributed distributed stream processing system what it has it has to work on that one is about and one is a bold what is about about is basically data from some kind of a queue and actually send it to both both are unit of computation these both they run on the incoming event sent by the spout in a and you can distribute actually spouts and both independently on multiple machines. So right now stream processing has talk of the storm is equivalent or it's in on M for stream processing so we went without any much thought process we went with Kafka plus now it has proved to be very stable up till now. So what we do is take all our system take all the event and fund them into Kafka and from Kafka this Tom topology which pull these even from the from Kafka and does the process but a story doesn't end here. There's a lot going on behind storage the story it is we are not so fortunate with the storage choices actually in the storage domain. We have a lot of choices so these are the requirements on which we actually narrowed down on the various storage system multi-dimensional support that I talked about that quarries the store should be optimized for quarries on multiple dimensions optimized for time series query. So these are dashboards they are necessarily going to give you a display of your system with every interval of time. So time so all quarries of the of time series in nature look what is response time high-right support that we have already discussed and it should be scalable. So as and when the adoption of the system grows the amount of data grows exponentially with it because whenever we integrate one system we are not integrating one metric we are actually integrating around hundreds of meters from that system. It has to be scalable and the best way to scale is scale out horizontally for us. So what were the options? There are a lot of options. There's druid there is open TSP there are people at ever fight there they have a good database there. We had Vertica we had plain edge page. There are a lot of choices in that field but we settled with open TSP reason being that we have used open TSP in the past and it proved to be stable and proved to be true to provide most of our requirements except for locally response time the response times were acceptable where they were within our 99% time. So we went with that the only problem with the open TSP was that it does not support support Kerberos. So most of you people are whenever you actually create a data warehouse you are actually have to be aware about security concerns around whenever you are aggregating data at one place so far to address the same concern how do our database Hadoop ecosystem come with authentication protocol and then implemented Kerberos and open TSP doesn't work with Kerberos. So what we ended up doing was so I'm just reporting the path and passing we have to actually implement a very scaled-down version of open TSP for our purpose. So moving on and metric definition what we did for metric definition. So the challenge is metric definition is I have already told that it has to be a function of media fields in that in an event but there's another thing is that with storm coming in if you people know the architecture of from you have to build a storm topology you have to actually write a Java closure or whatever kind of a code you have to compile it and then you have to deploy it on sound cluster that was not acceptable for us plane reason we have both business users and non-business users and we didn't want it go we would didn't want it to invest in engineering bandwidth in just writing storm topology catered to different metric what we ended up doing what we ended up requiring was a DSL through which we can express our queries and simply deploy it on Tom cluster. So during that time summing word was still not out so that you could have written any kind of a DSL or something but Tom from strident I think called Tom strident. So Tom also came up with concept of compute unit that was still not out that was telling Alpha so we needed a DSL that that analytics or analysts actually and business users can use it very easily and most of the analysts and business users are actually comfortable with SQL right SQL SQL like concept. They are very easy. They know how it is used how what it conveys what are the different cement syntax and semantics about it. So we first and we found something called as per how many of you guys know as per I think hardly anybody so as per is a complex event processing library what it provides is that's the most interesting part is a SQL implementation of that so you can push an event to it write a SQL processing or you declare your what type of processing you want using a SQL like construct and as per is effort will proper process that again there's a lot of details hidden around how we are using effort effort isn't even processing system complex even processing system that is usually used for monitoring event stream rather than processing event stream processing is a side that monitoring is the first class first class concern of effort but we needed reverse we needed processing as the first class construct rather than monitoring so we actually went ahead and you we had to modify not not modify the effort for but we have to modify the way we were using effort to actually get the processing so we actually read the source code found out open interfaces and actually what the processing out rather than the monitoring part so this is how our top topology look like so there's an event on the left-hand side that systems are pushing in into kappa kappa then they are kappa kappa these are actually pulling even and giving it to dimension lookup and nature board actually and nature board join these women to the entities they do a dimension lookup and actually create and put it back into another kappa and an enriched event so in this event is now an object it it's a nested object which is the original event plus the associated entity and the associated dimension the second part of topology took this in this event the kappa first provided them to an effort board so we mounted effort on one of the boards of one of the board and then converted these as per both actually converted the incoming events to a metric name dimension in value pairs associated dimension the value and the timestamp and put it toward it in a open DSP open DSP actually understand a metric name the associated dimension and the value so so far so good but this is not the end of it because we haven't really actually taken care of reliability we haven't actually taken care about a lot of aspects so let me talk about them before that reliability so all are agreeing had different points of time there are great that needs to restart the system then their metric definition that have burst they need to be redeployed for all these what you need to do is you have to replay the entire processing pipeline and that one of the learnings we have is that you cannot delete your old data you cannot you delete your old data or a longer period of longest period of time not one day not today for entire seven days 15 days reason being the birds usually get detected over time debugging you find out the cause and then you actually do a redeployment you need to build and replay for replaying the classic thing is that you checkpoint your computation at the same intervals of time once you have check pointed said that up to this point the computation is fine and once you have to replay you to play after that the point of the player previously stored checkpoint so all this is actually already in built into Tom and that's called transactional topology you can go and check it out the transactional topology actually allow checkpointing of a batch of event you can collect so out they need to collect a batch of event to transactional topology and transaction any point at any point if there is any exception or if there is any issue the entire batch is discarded and can replay the batch again. So we use transactional topology and for batching we did something again something different. It's actually back on time rather than creating random batches so and this time was actually even time or if you go back you see that every entity needs to have a timestamp when it was created so actually we passed event on time on the entity create time what is allowed us to do a was that we could now group events occurring together at one place and then we can do such statistical analysis that okay say in an hour or in a minute how many events occurred or how many say how many am I three were bought by a user bought bought by the customer so we would we were able to calculate status it we could do window join so there are two types of joints in streaming world the most favored one is lookup join basically you have a database of dimensions or any other entity you go and lookup by ID and then do a joint or you do a window join basically if you have two streams of event you time align them you align them on time and then you join the event in the same time window that allowed us to window join and it actually ties back in our early discussion that event that have triggered other event in the same time range in the same transaction could not tie them together and another advantage that you can now we are not tolerant of out of order event a lot of time what happens is the source system is down okay and it has queued up all the events that it needs to send and it has happened that sometimes the source system is down because of various reasons for and for hours and hours and you have to actually get those events and you have to actually attribute them to the right time window otherwise your graph or the pattern will not be correct for a diet code it allows us to actually be tolerant of out of order event as well so putting these time batching plus transactional topology together we came up with a second step if you have noticed that the solution was Tom topology three dash so this was our first step this is what we substituted in between so we have an endless event then we have something called a time batch bold which actually created time based queues or I'll talk about that how we are going we are creating those time based news in an edge based table so we use edge based for actually creating these time better reason being at base because it gives us sorted keys and it gives us good time look up so this is the schema so table one was our event queue so event queue the key schema was even namespace basically what was them what is the processing namespace or what is the process ID slot which is just a key I didn't fire that this is a slot or a time slot and then there is a batch ID batch ID was nothing but even timestamp and finding out which interval it actually like so if you're catering a one hour interval so time batch will be 11 o'clock 12 o'clock 1 o'clock and so and so forth and the actual value so the actual value of the event or actual value of that bucket was put as column and we were just even just on so edge based the table structure is key is a batch ID effectively and each column had the value and the column name was the timestamp so that's how we get a batch there was a second table which was a update log of this event queue remember I talked about that we have to be tolerant about out of order even what we want is that whenever a older batch or a batch back in time is updated we want to know about that okay, so this is where it came in handy whenever a batch in batch which is older in time or which is currently not within the given time window is updated we actually go back and create a journal entry so in this the key is batch ID and the version version of that batch that version is simply a non monotonically increasing number which we are increment which we are adding at the end of the batch ID which simply says that this batch ID has a new version so what we got was a set of keys increasing in order the last key of the most isn't he says that what is a batch ID which batch ID was recently updated so what we did was we created a simple time batch out which is reading from this event update you so it just has an iterator over this event update you and it is finding out which batch is recently updated so we just iterate over this we find out we find out from the last checkpoint which is the batch that is updated pick it out read the entire row which has so many columns which indicate one batch put them together as a batch and give it to the transactional topology and the rest of our topology was same nothing changed there so putting this together so this is our first topology which end this event putting a recap this is our second topology which batch those event and this event and this is the third topology which actually read from these time batches and put it in a dimension source so together they this entire system allowed us to achieve reliability and accountability at amazing degrees we could even debug so whenever there's any discrepancy in the matrix we can actually narrow down it on which time range there's a discrepancy we can go back replace those we play those events and find out where the problem lies so and we can actually we have proven time and again that you are accurate with the offline analytic system to the last rupee or to the last event count. So learning replayability I think is the most important tenant when you are actually building analytical dashboard. There is an event and entity schema should be declared before you should not be dealing with entities or unstructured entities plain reason being debuffing and actually building accountability becomes big problem you need to have checkpointing at the regular interval so that you can actually play checkpoint one to checkpoint two that is very essential for debugging or for actually correcting some mistake during that point of time bootstrapping so bootstrapping is something that when a new metric definition has been deployed you want to actually run it on the path data to see if the results are correct or not so you need to bootstrap or basically some people call it creating a sandboxing so sandbox or bootstrapping whatever you want to call it you need to build that so that you can test out new metric new definitions on new processes in isolation sidelining you cannot lose event again being a reliable system cannot lose event if you see that there's an problem in any event sideline them take those errors in the event and play them back again and finally call it all the time so your system has to be called all at all so so we got about 10 minutes for questions and if we have anyone from Nintra also in the audience I was told to ask for you so please come up to the stage thank you very much hello hi listen the question is that when you use transactions both Bolson storm did you face any issues while running in production with respect to throughput and how did you go about optimizing the bats? Okay, so we actually the number of events that we were receiving was very high and the so we ran these topology that slightly two different frequency so one the batching frequency was very high actually it was creating batches every some few millisecond and the actually batched the topology that was running as far as creating the actual metric it was running at a slightly higher slightly lower frequency which was around half a minute to one minute and so that in the meantime we had enough number of batches so that we can parallelize so that was one thing second thing is that we were receiving so many events for so many different system that you could parallelize or we can distribute this computation like crazy because distribute all of them in the time so we didn't actually see any issues running batch topology so we yeah, yeah, did this answer the question? Yeah, another question here actually I have two questions first one is about the infrastructure that we have this amount of data you get into your Kafka every day how many hardware systems you had deployed what are the events per second that you get and you expect and the second question is about do you have any machine learning skills deployed on these clusters and how they were in tandem with storm or Kafka so we haven't deployed any machine learning up till now around the event we are seeing around 2000 to 4000 at the lean period and around 100,000 at peak so this is the event so the deployment is right now on a small cluster we are not processing all the event we are discarding a lot of event so deployment is not more than 10 machines right now they are three five machines for Kafka collect all these events and put five machines for running the storm topology your Kafka and storm co-exist together on the same node or they are on different so we started off co-hosting them but later on we separated them all because of scale issues. Thank you and we also have two different zookeeper instances for one for Kafka and one for storm because Tom actually storm transaction topology for doing check pointing it actually right back into the zookeeper for checkpointing so zookeeper is the database so actually we have to separate that out because it was overwhelming Kafka thank you hi hi so how are you actually sharding the data that you are storing gives it the automatic de-sharding or do you have your own sharding policy no it is automatic so we haven't tuned open to be up till now you guys have but no that's what I said that open to be we are using two instances of open to be one for a separate purpose second thing we couldn't make it work with corporate we actually wrote on a very small scale down version of open to be basically similar construct similar principles but we wrote it ourselves it was to to class files that's all and what's the what's like the data written once for the dashboard and it's okay archive after a while or just kept it for now. Okay, so we actually format we actually start was for experimenting with the Lambda architecture we didn't really completed it so our policy was we will take the data in streaming dashboard or in open to be for as long as some few days until the batch processing system can catch up that was configured for three days or seven days I don't know remember but three days of 70 not more than that hi we had a we had a use case on real-time analytics here here so we were using just Tom I wanted to know why we come up with Kafka plus Tom and why not just Tom because Tom can also do your stream process is not for stream processing Kafka is for storing even now Kafka is a very unique message you which is very different from traditional message you is in the fact that you can have multiple consumers for the same queue so the consumers maintain the state of the state of the messages up till what point they have read not the server. So if you take for instance rabbit MQ or some other kind of a queue the server actually maintains what length a consumer has read or how many messages the consumer has pulled out of the queue but in Kafka it is reversing the consumer who does the state management so can have multiple consumers reading from Kafka point number one second advantage is that since Kafka never delete messages theoretically can actually make Kafka delete messages. There's a retention policy there but since Kafka theoretically cannot does not delete messages you can actually go back and start raising messages on the start that gives you bootstrapping and it gives you replayability so that's why we went for Kafka. Okay for the next question we have one right up on the screen right there. Can you talk about the size of data? Can you would you be able to shed some light on that one? Third one from the top from the bottom. Can you talk about the size of data in GBs? So I don't have statuses around the size of data in GB what we calculate was the size of data in terms of number of events that we are processing so the number of events I have as I've already told you that the number of even that we see with 2000 to 2000 in steady state or in very when there's no peak at because we are getting around 100,000 events per second and what are the benchmarking metrics so we actually we had the benchmarks of the requirements on the customer so something like I should have metrics updated every minute. I should have metrics the query time should be not more than some few milliseconds and say not not more than say 10 milliseconds for 99 percentile. So these are our actually guidelines. These are our SLAs and we try to engineer our product to meet those SLAs. So first stream processing you know, we cannot process a lot of data so we restrict restricted ourselves to three dimensions of not more than that but these dimensions were actually very large. So they were product dimension user dimension which were runs into millions themselves by themselves and the event size the actual event size we observed that in we didn't get more than 10 KB of an event or as a site so that was the highest that we got question here. So after here after TSDB like what do you use for dashboarding? Okay, so we so we were we actually created our own dashboard using nvd3 which was written over d3.js. We didn't explore any existing that voting solution because none of them provided slice and dice or basically filtering capability so we had to create our own so it is it like a dynamic dashboard anybody can comment so not dynamic. So we actually the dynamic is something that we are building right now. So once we ended up doing processing there was a team that went ahead and create those that work for the user because it mentioned like SQL stop over no SQL was just for create a processing right? Okay, so the team which was responsible for creating metrics measures versus the team which was the first which was responsible for consumption but two different teams. So we are actually developing them both of them in parallel. So one finish that job. Second person is second team is doing it. They got another question. So do you think that tried into stable as of now? So can can you like would you want to move away from as per to try? So yes, we may want to we can but again, we have to provide our own DSL to work with Trident or enable business for X or actually convert SQL into a Trident contract. We have to do something like Trident has like a general SQL constraint. Trident has a little construct but you have to still go and put them right? You have to go and write code come to actually deployed on on stop what we want is that a customer just declares the processing pipeline and done with it and our system takes care of deployment making it live taking it everything. So so somebody declares it in as per somebody declares it in SQL like construct which is supported by as per right now. Okay, so it is called as per QL by the way. Okay, okay, we have time for one more question. But before we do as you can see a peach anybody working on please do not do any self promotion or any things like that we'd appreciate if we can just stick to this with only questions about the actual content going on here. Thank you very much. We appreciate that we have time for one more question after this guy we're going to go these two and then that's it. Once again, you can take any other questions on to sb.lk slash has geek. Thank you. Yeah, so we are building the audit system at the start from the start begin from the end. We are building an audit system which collect these checkpoints find out how many events are processed and report if there are any anomaly. So we're building it right now coming back to how many what was the throughput so actually throughput was around updating 100 measures for now. Again, I said we are getting a lot of even but we don't have that many measures right now. So it was 100 metrics for now lot more than that and point tuning we learned that all the system needs to use their own databases. So that means if there's a storm and it's doing check pointing in zookeeper you cannot have a common zookeeper cluster while another system is using so you have to decouple the system and logging another thing is logging. So if you leave if you just install plain manila storm it logs a lot. Sorry, kappa logs a lot. You have to actually do not otherwise it's going to eat up your IOPS in their plashing today. It depends on your processing your processing is bound. Yes, it is going to see people. So if you have seen we have actually separated out if you bound node and IO node. So if you bound note were actually concentrated in at per volt and all I have bound note were actually separate from topology.  How is it different from alarm that they do use? So that's what I said that I can't say if it is different or same we started out with that aim in mind. We haven't reached that point yet. We started off with that if there will be streaming analysis or stream processing that will be done by or more real-time processing that will be done by Tom or our stream processing engine and in parallel we were developing the back processing pipe but we can't really say that since okay, so we haven't seen or we haven't actually nobody has shared an experience of working Lambda architecture. Whatever Nathan Mars have talked about is again very theoretical in nature and people like Jay Krebs who have created Kafka from LinkedIn. They have law force talking about what are the fallacies of that architecture which might or might not work. So again this is his point of view should not take it as that whether Lambda architecture is just point of view. So it's again very theoretical in nature Lambda architecture. People are experimenting with it. There's only one prototype that has come out right now summing bird from Twitter apart from that there has been no other system apart from the last yesterday. Okay, perfect. That's all the time that we have for right now.