Everybody forced here up being 2223 prison basketball hobby four box number 45 pick your team style Got Alan V Celtics, Bob H Thunder Brian R with a Mavericks Charlie with the Bulls Daiichi Pelicans Darbin has the Cavs Eric W Rockets Kings Fairhawn sons George J Bucks Jamal with the Hornets and the Clippers Jeff T Warriors Jeffrey H magic John with the heat Kevin L T Wolves Costa Raptors Leslie with the Sixers Logan with the Hawks Marvin and Nuggets Michael H Jazz Nick a Nets Oriole with the Grizzlies Robert P Blazers Tycee Lakers Timothy F. Nicks way Ming wizards William and Pacers and Spurs and Zach B with the Pistons Good luck everybody. Come on Bruins. Let's go right number 45 here. We got a red a Sons of the Cal bridges number to 299 the Thunder Jalen Williams Ricky Otto Pulsar bam out of bio heat tar eason you got to teach Jake and Donnie how to top load Silver luck of the lottery Jeremy Sohan Spurs Silver rookie Christian Brown for the Nuggets Jalen Williams special kind Dude, we should send him back to first really Billy Madison. Yeah, Rebecca Black is probably what like 27 28 You got a hyper Chris Duarte Pacers or is she like 30 Silver prismatic down of a Mitchell Cavaliers silver rookie Christian galoka Raptors 25 I was trying to think I don't even know how old I was when that song came out What what year did what year did Friday come out? I can't remember if I was in high school or if I was already like past that I'm guessing I was already past that if that came out. I'm five years older than her review a pretty go Barrett T Wolves hyper shooter for the Lakers Johnny Davis 2011 I graduated in 2010 She was you know, dude, that's crazy She was like 14 15 My mat, what's my math? My math bad Orange Lonzo ball to 49 for the Bulls 3 of 49 Silver Dennis shooter Lakers Yeah, I'm on the war on the same page then Purple eyes Pat Ewing for the Knicks number to 149 with a bulls Justin Lewis silver rookie Otto Hyper David Robinson spurs Jeremy Sohan rookie, so I would have been a Freshman in college and that came out Ruby wave on the echo Kangoo Hawks Silver DeMar DeRose and bulls Blue Prism Grizzlies Tyus Jones number to 199 That hyper clay Thompson Warriors All right legal. What was what was your class song in high school got blue ice Franz Wagner Nice magic color match number 95 out of 125 Silver Kawhi line of Clippers What was your class song in high school Sean into the road? That's pretty good. Oh, let's go Bruins. Let's go Let's go Ruby way Brookie Keegan Murray Kings very nice card got pulsar to soon move bulls Didn't have a class song. I didn't know what mine was until I saw what it said in the yearbook That's how I knew I didn't know idea mine was uh Mine was um, what was it? Party to USA by Miley Cyrus Oh, goodness Yeah, very very very poppy Silver Quentin Grimes nicks. I think it's because that was like one of the most popular songs or like Like it wasn't I don't think we necessarily chose it right Yeah, this is what it is It was a very popular song That was like peak like Hannah Montana stuff like not like Miley Cyrus I was like peak Hannah Montana when she was like still a kid basically.  They knew what they were doing. They knew what they were doing They knew red bogdanovitch For the uh pistons dude. What what's up with a doogie von? No homers until the past like three games. I was home for twice to 299 Oh three at ten mavericks gold signatures spencer din witty Nice hit mavericks Good stuff hyper jordan pool warriors The advice song what what is that? Let's go Bruins Ruby wave mcowbridge's sons silver lude dork thunder Shabari walker isaam obli The advice song File your taxes early like I don't understand. I don't know lude kenny martin jr. Even though I'll never ever do that for some reason it's a 199 Never done it never will Mark williams rookie auto for the hornets Hyper rookie jabari smith jr. Rookie Rockets. I told sam. I do his taxable. I'm gonna do it. Yeah He's going to jail, dude All right, it's hyper jabari smith I got you sam. What's the deadline september? Oh It was uh Last week, you know, it was monday or tuesday Why can't they just take the right amount out of the paychecks? Why do I have to why do we have to do it?  You don't want that or how much we're supposed to what we're not keeping track? Yeah, it's over fireworks. We could dodge You see in this country you trust everybody. Rookie silver jeremy sohan spurs You're gonna believe them. Yeah paliban caro Come on bruins Score more goals Silver fearless damey lillard for the blazers Silver rookie wendell more jr. T wolves. Oh, come on. That's a goal Ruby wave donovan mitral cavaliers with a Hyper bogdanovich for the pistons Mcallan's yovitch Sunscreen to be it Yeah, well, I wear sunscreen because Yeah, I live before and I got pale skin Mom and uncle both had basal cell carcinoma and all that Six to go on eminence guys Let's get it filled up Purple ice ad lakers color match to 149 Pulsar zion pelicans Kenneth lofton jr. Walker kessler Come on, you need a big card Blue ice lu dork to 125 for the thunder The silver patrick beverly lakers All right guys, so I think I might go to the fair tomorrow. What's the first ride you would get on if you go to the fair Gravitron Gravitron. Yeah, so I can have a headache for four days Awesome, you're chucking all day So I roll out of bed in the middle of the night ruby wave rookie trevor keels for the nicks very good Hyper and the everons t wolves. I ain't doing that not before the sg Maybe maybe next weekend. I'll do it after the sgc break Chad Holmgren. It's my day jang following saturday. We should all go silver pat connets and bucks Caleb houston shade on sharp do a little pregame before the fair some funnel cakes Them fair funnel cakes hit different too. Yeah, they use the same oil for the last six years. So, you know It also fires up the filter world It's how they keep it lubed. Yep That's why it's good though seasoned. Oh, it's seasoned hyper kaminga warriors Benedict matherin Oh, man, silver luck of the lottery jaly Williams under the funnel cake line is the real ride Silver john stockton jazz I always think of the sandlot though when they all They're like 10 they grab the tobacco and then they ride the Crazy the spider whatever it was And they just start losing. Yeah red jade mc daniels t wolves number two two ninety nine Janet ivy pistons rookie auto. There we go. Congrats detroit. That's going into a mag That's what we need it right there. Good autos this one so far golden witty now an ivy base There we go pulsar tatum celtics That's what we need it right there. Let's see more of that Illegal have I shown you the picture? Oh, I deleted them. It's literally Across the street from my apartment like I swear to god. I might be able to throw a baseball that far Okay, Blake could throw a baseball that far not not me silver prismatic lebron james Silver rookie shade on sharp blazers It's the first ever winter garden florida fair which doesn't sound right But in like 40 years, I'm like I went to the first winter garden fair I was the best. I remember when eggs were only seven dollars for a dozen Yeah, it's I can literally like it's it's crazy, but it's just a standard fair It looks like it has you know the haunted house has the you know all the different rides the slide the thing That's the pirate ship thing that goes up and around until it spins and it's called the throw-over Yeah, yeah, it's got the zipper probably got gravatron. I haven't seen the gravatron. It's probably in the back somewhere, though And then I'm sure they have a bunch of different food and stuff That's the best part though free admission. I'm in dude. I'm just gonna go walk around the first day and get a scope And watch her be way to assume new bulls and you buy like ride wristbands like 20 bucks a weeknight all you can ride I'm like, oh dude easy rookie hyper Ryan Rollins warriors Walker kessler Yeah, I don't I don't think it's that kind of fair. I think it's just like Like there's no there's none of that. It's just like games food gator rasslin Games food and rides. It could be I I didn't see it on the website Gatorland to come up there Yeah, it's it's funny. I look at my window. I just see like a ferris wheel and like I'm like This is like the true rolando experience everywhere you go. There's an attraction. Oh, yeah Red power brand an ingram to 75 pelicans Yeah, yeah 20 dollars all you can ride on weeknights. I think it's 30 bucks on the weekends I'm like, that's good silver. Andrew Wiggins warriors. I'm going down the super slide so many times Oh, yeah Gotta get you gotta get the good burlap sack though. You can't get the small burlap sack My wife should probably wear like leggings and just fly down that Dude, that's that's good Kendrick non-purple ice lakers to 149 Clark Kellogg pacer's auto here Clark Kellogg Makes one some cereal got hyper crystal white tape pacers Back at my day. Yeah, exactly. Chad Holmgren. There probably is gator rasslin. That was a Jang A butter sculpture eating contest. I don't again. I don't think it's that kind of like there It's just like a pretty standard like There ain't no butter nothing contest. It'll be melted You know, yeah, dude, it's we've been in summer in florida since the beginning of february Ruby wave Daniel ties pacers we went from like our mini winter It was cold. It was pretty cold for a couple weeks for florida. We dropped down in below freezing a few nights Uh, so for florida that's that's called all the plants started to die silver kemba walker cell for the nicks Caleb Houston shade on sharp and uh Yeah, it's they call it the fair but It is more like a little mini carnival like get what you would find at like At your high school I'm just I'm just saying the name they use guys. It's just rides games. Whatever baby red prism Jay shuntate rockets Number to 299 they might have other stuff but as far as I saw it's just like, you know typical, you know Corn dogs funnel cakes, you know, lemonade. Yeah rookie hyper ryan rollins warriors Super slide. That's the one dude. Oh shag maji Yeah, exactly. There's not gonna be like People with like cattle and all this other stuff and yeah, it's not gonna be like it's not huge The best is late night when you sit in front of the glass maze house Mojo fearless damien lilor number to 25. That's pretty cool for the blazers Just watch people wreck themselves in there Slover troddy marten hornets. Yeah I'm guessing there's no alcohol to fare that that doesn't mean there's not gonna be alcohol consumed at the fair What I mean Did you guys they don't sell it doesn't mean anything to happen. Oh, they're probably selling it Dude, if I was a cop, I'd go there and just bust all the teenagers. Oh, yeah, it'll be way of de-weight heat Oh, dude Have a busy friday night over at the fair pulsar tmac raptors So here's what I'll do guys. I'll I'll maybe head over tomorrow night Maybe and I'll report back on sunday with my findings. Yeah I'll let you guys know what uh, what the fair what the winner guard the first ever winter garden fair is all about Got silver siacum raptors March on bow champ jaden hardy my narc Not a narc. I'm just saying like Because I remember my high school days I remember my high school days You know you go on movies on a friday Find someone's older brother or sister or something.  Let's get that powerplay gold. Come on True gold. Oh, come on. Come on. It's jazz. I saw jazz Pistol peat true gold pistol peat maravitch number to 10 Yeah, six to 10. There you go jazz Legend gold there No, it's definitely more. It's definitely more like a carnival silver jalen brown for the celtics It's williams jr. Max christy. I think the company that probably does like the it's like, you know, it's just a company that travels around does there was another one You know 10 miles away like last month might take the same company. Oh, would that it It blew up with the sixers to buy us hair. That's got to be good business though to 199 It's got to be tough being on the road so much, but And redemption patrick baldwin jr for the warriors on the rookie signatures prism silver hyper jonathan isaac magic Vaughn, you don't have to answer that question. Don't answer the legal questions Oh, that's that's that's fair then that's super lucid donchich prismatic Silver brandon clark grizzlies through that's that should be okay. You should it should be automatic jail for if you do that in a Like you're literally in a public space But I'm rated g movie. That's got to be a lot harder a harsher sentence. Come on now Like i'm not a narc. I would have called the cops in that situation. I've been like, dude This is not this is not good two ninety nine brandon ingram pelicans world star Jason terry maverick sensational signatures Jason the jet terry Hyper k g for the teawolves It's illegal. You know what he's trying to do ruby wave contavies called. Well pope for the nuggets Polls are on a nobie raptors As far as by the junior tie tie washington. I've been in the movies in too long, dude I just want to go sit in the dark room and watch a movie for two hours and Eat too much popcorn and stuff, you know rookie purple blake westley spurs the 99 Nice one spurs. You've got a silver d book sons jade and ivy Yeah, I was thinking that too and I was like, oh bond didn't say Silver luck of the lottery jaylon doran for the pistons got sub rookie jaylon williams for the thunder Dude, this late in sports cards your purcells keep hopping and chat doing all these emotes. Maybe you guys should go buy some purcells over there Troll it Blue prism de erin fox kings that one's numbered to 199 That hyper dairies basely thunder Let's go kings. Come on kings beat the beat the warriors and power jazz. John Stockton to 75 since silver kenny martin jr. Rockets malachype ran him He can Murray review wave rookie christian colloco for the raptors for any bedtime activities like fresher teeth And hyper bucks drew holiday Where's your teeth put on put on the sleep the sleep mask room dust Okay, I still got eminence I use I use I use a sleep mask It's kind of bright in my room only because I I want to sleep in past like 8 a.m. You know uh red numbered to 299 and ingram Tate mcdaniels both donovitch bridges Luda 199 de erin fox to bias harris kenny martin jr. Tyus jones cucumber slices Purple ice to 149 kendrick non anthony davis and pat ewing Blue ice to 125 blue dork france vagner purple to 99 blake wesley red power to 75 stockton and brandon ingerman orange scotty Pippin junior rookie as well Our red sox onesie Wish I had I'm gonna get one of those now. That's awesome It's too warm and portons Got orange lanzo as well mojo fearless damien louis to 25 a gold pete maravich for the jazz autos of jason the jet terry cart kellogg mark williams Justin lewis Jane the williams golds and sational signatures din witty for the mavericks and redemption warriors patrick baldman jr. Silver redemption their rookie auto Anna jade and ivy rookie auto pistons That'll do it. Thanks again everybody on to the last of the case


Welcome everyone and thank you for having me and for attending this session. My name is Sergio Maurenzi. I am the responsible and the founder and the CEO of Peperina Software, a cloud native software engineer firm located in Argentina. Well today I will present a project, a modernization project about a tax information system in the public sector where we use a reactive architecture approach. This is the agenda for the presentation because perhaps we have no time. Well the Q&A session will be outside. Well a little bit of context about the project. Córdoba is the second most populous province of Argentina with almost 3.7 million inhabitants. The government agency called DGR or Dirección General de Rentas and Collector UT is a private health company. They are responsible for collecting the taxes for the province, for the states. The taxes are basically fees related to properties, real estate, vehicles, boats and business income. In 2015 the agency started to move the statement of this of this taxes to the website and they start to have many problems of basically availability, performance and that's why in 2019 they decide and ask or request the IT department to think a different solution, to think out of the box a different solution for providing this service to the taxpayers. At that moment they decide one key business objective, allow the taxpayer to search and pay their statement always and the last word always is very strong, you know. The tax agency required to think this solution as I mentioned out of the box because we don't believe in a big bank project, migration project of the legacy system. That's the reason that we provide or we think in a key technology objective that is decouple the functionality of search and payment from the legacy system as much as possible. Well these were the non-functional requirements of the project, I think they explain themselves but basically the solution must be responsive, resilient, always on, performing and have the capability of track every transaction because it's money of the citizens. This out of the box thinking was the reason of the name of the project, Copernicus. We propose and design from the ground up a solution that is accountable, reactive and microservices and also even driven architecture and this is the high-level architecture of the system. Basically it is a pipeline that ingests and processes the changes that we call them events because they are produced in the legacy system every time an important change happens in the business domain. Of course are you seeing oh yes it's it's good it seems good the image because we have a problem with the sorry with the interface and we have also a feedback look from the consumer apps for the commands and queries of the of the repository. Enter into more detail the pipeline start with the data source with the cdc or change data capture every time an entity state from the data source is changed in this database that is an update and here's insert or a delete a db trigger copy this record of the db of the table in another table called Outbox. Then the pipeline continues with the ETL in this process there is some transformation enrichment of the data there are many processors ingesting these these records and then part of the this data path through and a new component that is a business rule layer this is a business rule engine that processes certain events not not all events certain events to classify the depth of the taxpayer which is defined by more than 45 business rules yes in in our country we we have many creative with the definition and exception of the the depth this in this case the business area can change the rule represent a rows in a dmn file dmn is decision modeling notation is a standard of business rules and then the next stage is the messaging system and as the name suggests the main characteristic of this messaging system is it is an event state event current state transfer pattern and that's mean that the event are stateful events and that's why we can't lose any of of them this then the consumer of this stateful event is the aka aka platform or aka cluster who who built the private replica by storing the state in the actual model is a this is a really short period of time that the data is lagging behind from the data source and that's why it is eventual consistency no strong consistency the actual model for the business logic is the core application which is based on distributed and concurrent processing then we choose event sources and it is because the nfr nfr that I mentioned before that is related to the older log of every transaction also we have a cqrs we provide the read data model to allow to be exposed through apis by the bff to the consumer apps the bff the back for front end well and this is the technology stack for every component through the pipeline a patch and ifa for the etl drills for the business rule engine kafkan the messaging system aka actor model programming in scala and the two repository right side and read side are running into a no sequel database kassandra database and the consumer facing application that is that's are in the front for accepting the query and commands are no gs applications all of the components of all the architecture are cluster aware and in the case of kafka scala or yes all of this the aka kafka and the no gs application are living in the kubernetes cluster before starting to code we spend several workshops days with the business and it area trying trying to understand the process the business process we use even storming for trying to capture the whole information about this process and then we have different sessions to moth modeling the data and here here there is a simplified view of this relationship between the the main entities that are in this kind of system the person or taxpayer we represent here we represent here with well the name is to head in spanish is represented the taxpayer the assets of the person are called of head in spanish and the liability or obligations are related these three entities are very related and that's why we are putting in in an aggregate for example if you have a message of pain a liability the message entered by the aggregate root and is root to the corresponding liability to update in this case the balance this also update her parent or in this case the object or asset and then update the balance of the person sujeto and the last step is to emit an event this is a simplified workflow of what happened between these entities well during the project we have many challenges this photo is from cordon mountains i love mountains but guess what this guy is not me i prefer i still prefer technical challenge well during the execution the project well as i mentioned many problems but mainly well here we will talk about throughput observability and scheme evolution one of the requirements of the project was to process the full entity states previous to the beginning of the next year because every year the tax agency generates the liability for the full year for every taxpayer this is a batch process we call it initialization it includes all more 100 million of new records to update during the weekend window on the other hand on the ongoing process every day we process two to eight million of records or events per day when we talk about improve the throughput in the into the pipeline we have many places to tune in are improved for this presentation i just talk about this uh one one of these improvement is related to the message and processing stage i i think that you already knows uh about aka but i don't know if are you familiar with aka streams here yeah well aka stream just a little as a model built in top of aka actor to make the ingestion and processing of stream easy and the alpaca project is a kind of library that's bring us a bunch of connectors to integrate various system in aka streams here we will talk about alpaca only the kafka connector so here's the basic aka stream pipeline you need a little a source and a sink obviously a source has an outlet a sink has an inlet and then you have an arbitrary number of flows in between message flow downstream from the source to the sink but you also have the other channel going to the upstream this is for back pressure in kafka the main level of parallelism is the number of partition in a topic here we have almost 90 topics in this project one for each entity and also we have uh topics for retry retry processing certain events error handling and prioritization each partition is shared by the entity id as we will sell later it is possible to expand the number of consumer instance to keep reading data from the same topic in parallel the consumers subscribe to kafka topic and pass the message into the aka stream in aka streams the kafka consumer is also named source well we have two approach for doing this the first one was um the first approach was about the consumer kafka transaction why because kafka transaction allowed exactly one's processing of message message are consumed transformed and produces to another kafka topic in one atomic operation and all message including the transaction will be successfully written or none of them will be this is the the core of kafka transaction and according to our first test it really worked well but the problem arose when we put many events in the pipeline when we increased the the number of message we changed many settings the number of partition a quantity of brokers the number of aka cluster nodes the processing power of this node but it changed widely we realized that we have a bottleneck in another in another area in another component well the first issue was related to the latency when building transactional pipeline that spawned more than an entity update that that is in the specification or in the the the talks that shone glove for example uh talk about this issue the latency between the components in a transaction but the most important we discover that is also in the documentation is that this kind of consumer is not partition aware and that's mean that if it cannot manage each partition in parallel and that's why there is only one consumer for the 90 partition and that's why we tried with the second consumer a committable partition as source configuration this config use at least one semantics different the source use a single kafka consumer internally but the committing works with partition that's mean that it's partition aware the least one processing means that if the broker that's not received confirmation that the event is consumed successfully the broker will resend the event while this ensure that the event is a get process it also means that can happen multiple times and we solve this problem introducing an identity identity with this configuration we can technically technically size our kafka consumer with the number of partition and the same consumer group run in parallel to achieve high throughput and high rate of the dating gesture is like the the slide show like one consumer for each partition well here's the code example of our consumer main processor i will explain briefly the committable partition source method supports tracking automatic partition assigning from kafka when a topic partition is assigned to a consumer this source will emit a tuple with the assigning topic partition and a corresponding source and when a topic partition is revoked the corresponding source completes here we can identify two map async operators the map async operators if you remember allow us to process the flow in different actors separating the boundaries and allowing us to execute the flow in parallel it's also important to remark that this parallelization of event is done in order we have another method map async and order but in this case we use order because it's necessary for the events that are stateful the first map async that is in the code is the source of each partition and has a parallelization parameter the number of partition and the second map async is the operator to process the message inside this partition here you we use consumer parallelism environment viable to increase the parallel tasks to achieve a better throughput but we couldn't achieve these better results with this increasing this parameter the application turned unstable when we increased this parameter more than five continuing with the code we have the mapping of error in processing the message we have two kind of errors one is related to the timeouts into the pipeline because for example the database is one issue for example in this case the queue for trying to retry this event we implement this way and the other one is normally related to serialization issues when we produce in this case message for another topic is for error topics here we can see the case for successful processing which produce message into the correspondence of sex topic at the end of this code you can see also the sync method and the stream materialization or the writing of the graph well the next challenge is talk about observability of course we have no time in this presentation to show you about every tool every dashboard or graph it's just a glimpse but as you know in distributed system data and application observability is essential because the message that flow in the pipeline are events that represent some state state changes that are important from the legacy application the source of truth in this case we need to ensure that we have no data loss every stage in the pipeline must be observable we need metrics log and traces all of them to look for the trace and each event we use one unique identifier that we define for the cdc we call evid which includes the time stamp when the trigger activate and a sequence number from the source database in this presentation again we will focus on the observability just in main components of the the message system and the business logic and this is a slide well we are showing control center from confluent platform and we can see the consumer lag of message from one of the topics we see the total number of message behind and by partition and also you can see the bar in the middle each bubble represents a partition and a position represent the number of message behind because this tool shows the message consumption in live mode we could discover the latency issue with the transaction consumer that i mentioned before because we saw how the consumer process one partition at a time we we can see this movement one partition at a time but when we change the consumer incommittable partition we could see how the processing of the of the all the topics are in parallel here we can see kafka ui an open source tool to manage uh or to uh to show the message content through the kafka and here you can see the evid that is remark it contains a timestamp and we can compare the timestamp that was generated by the database and what it enters into the kafka topic and then we can see if we have any luck there fortunately liven provides several dashboards uh to help you to help us to observe and monitor the behavior of the actor system in terms of throughput latency and consumption this is one of them of actors and and we have another about akka streams numbers to running throughput latency i will start to move forward sorry another useful tool to capture the trace between the components inside the application open tracing and siebkin and of course we define our own dashboard with cinnamon and kyman we use these libraries to include our metrics of our application next and last challenge present here is uh scheme evolution working with event sourcing system we discover some issues and challenge the first one what is related to the number of read events for recovery operation every time we need to recover the state of an entity the persistent library reads every event from the beginning and replay them to compose the state the problem with entities related as an aggregate is the number of events that they produce for instance when you have a tree like this the parents of the events of the all of the the children all of the three in this example is just five but we have a big family we have thousand of events running of grandchild or child or children because this is a common issue in event sourcing akka persistent library bring us a solution called snapshots you can define for instance that after n events you can take a snapshot of the day of the state and with this feature you don't need to read all the events from the beginning you just read the latest the latest snapshot and the following events this solution reduce raster drastically the response time on recovery the state but introduce another issue or challenge the scheme evolution is about changing the structure of the message according to life cycle of this application the snapshot we use in akka stream are not compatible with the library we use for serialization we use cryo library we try to change the library for Jackson but the performance was worse in the next slide we are going to explain how we solve this issue first of all well we have the snapshot of event of one of the entities in this case the person each snapshot contains the state of the entity and other fields that compose what we call a register this register could include for example in this example three fields other a type address an email now you want to add a fourth field for example the phone number if we change the definition of the schema when we recover the state from the snapshot it fails because the snapshot doesn't know anything about this field what we did here is to create a new case class at the application level to manage this like a new version of the schema and when we read the schema we compare with the different versions and well these are the two case classes the version one called sujeto three contains the all register the version two called sujeto three two is the one that contains the new field and take into account that in our case we have optional fields into the register and that is that is why it's possible to evolve this register this is the code where we do the schema transformation aka persistent support event sourcing with persistent actor the behavior of persistent actor is defined by implementing receive recover receive command and here we override the method to change the behavior the persistent actor receive recover define how the state is updating during the recover by handling event and the snaps a snapshot offer message and here's the way we compare the snapshot with the corresponding schema version one or two and in the case it is the old version the snapshot is copied and updated final slides since the start of the project in 2019 the engineer team had many challenge and on learnings the rollout in production starting in 2021 last year mid-night mid-night last year the system won't live with the initialization process and immediately starting to ongoing the processing gesting of all the events from the source system but in terms of use from the consumer point of view a blockchain based digital wallet it was the first consumer just started to use the data of our platform in a light way this was part of our strategy to allow to tune in the platform but then in the second half of this year the system started to be integrated with different other platform and systems IVRs website new revenue system etc this platform is considered the glue between the legacy system and the new customer facing systems we are continuing developing new features business features like a discount model module that is by nature reactive need to be reactive and of course we continue to evolve in and tune in the platform especially increasing the performance of data processing a robustness of the events in the pipeline well key takeaways there is one universal truth all things that can fail will fail you know it's not matter of if but it's matter of when if a single consumer face that is an accountant for in the design state replicas can divert permanently that's why do not underestimate the complexity of maintaining a consistent state across microservice in a high availability environment at the scale the second key lesson is about restriction and non-functional requirements review carefully for non-functional requirements that could change your architecture decisions for example the security audit logs never the business partner never then asked to show this part this part we choose event sourcing for this the third key lesson is about the resilience and responsiveness as you mentioned today in the morning with these asynchronous and decoupled components we could see how the system reacts to failures like application nodes going down or issue with the network or with the database without disrupting the service because well in one case they activate the circuit breaker or a spring bearing resolver other part of the platform this is really amazing in conventional system if you have an issue like this the application becomes unavailable in any case the observability is essential as I mentioned before in distributed system okay are you already here working with distributed system is hard but worried at the end we have more resilient and robot system and our taxpayer and citizen deserve it thank you very much


Every Christian would tell you they don't love the world. Every Christian would know that they should not love the world. But is that true? How many of us actually love the world? As a matter of fact, how many of us know the world doesn't love us? If you die, if you get sick, if you find yourself homeless, if you say the wrong thing, if you sound too God-like, too Christian, too holy, if you speak too much of the Bible, the world will turn its back on you. We already know that being friendly or friendship with the world is hostility or enmity with God. But what does that mean? Well, when you adopt the world's view on fashion, the world's view on politics, the world's view on dating, love, and marriage, all those things that we seem to have allowed creep into our lives as Christians, well, that's what it means by loving the world. We hold their values more than we hold his values. Don't love the world, they will not love you back.


We're dead. No, we're not dead. Let's go. Let's go. Let's go. My confidence is gone. How's it going everybody? Todd's it only here back with some more dragon ball fighters. And today we are playing a team of Trunks, Krillin and Yamcha. Today was just one of those days where I woke up feeling like playing a certain character and that character was Trunks. I don't know why. Maybe I just felt like changing some futures today. Oh, he's got Super Saiyan Goku. He's trying to get them buffs as well. I see you. That's also me. We got that new balance patch coming out soon. It's time to start begging. Never works that way. I love to imagine that Arxes is just like, let's check out the community, what they're saying, what they're asking. Don't 2H me. My goodness, that would have sucked. I didn't have Super Dash. I'm going to switch out for now. Maybe give them a little bit of a lunch break. Oh, I still got the sense of being 2H. It's too late. The Blue House already gone. Yeah. So you notice he wanted the Blue House. He's got to admit it. Nice.  The perfect range. That was sick. Jump at him. Super jump. He's got a spark to get out of pressure. That's fine with me. Well, that was intimidating. I did not expect him to backflip first thing in the morning. Oh, that was a curse cross-up. I can't believe I reacted to that and blocked. That was genius to say the least. You thought it was a lay-ay. I know some trunk stuff. You got to, well, unfortunately, it's not good trunk stuff. But with his level three, you got to make sure you're there, right? So the, ah, safe jumps. Just kidding. Total prank on you guys. I knew I was safe. Absolute safe jump. Definitely didn't also think I was in danger for my life. That scream hurt my throat. Dude, sometimes my safe jumps are so risky even. I don't know they're safe. Yamcha says take out his assist. He's going to Super Dash. Look at him. Look at him. Look at him. Oh, flip. I'm out of there. I was out of there until I came back in here. Oh, no. All right. It's time to bust out. It's time to bust out the most broken form of Trunks. Sparking Trunks. The mix-ups here. If I can get him to block. Oh, you got to respect Sparking Trunks. You got to respect them. Oh, no, no, no. Huge drops. No. Oh, come on. Even with a sword, my two-inch didn't reach. Unfortunately, I don't think you can kill us here. No, you're just going to take the snap. Oh, you want a beam? You want a beam clash? This man's hoping they had beam clashes in the, in the badge. Oh, that was sick. Okay. Good read. Oh, nice assist. Yes, I am giving myself credit. I noticed I was giving him a lot of credit, so it's okay for me to give myself a little credit. I'll take a little bit of credit. This is going to be two level one, so you know Trunks got to come back out. Burning. Attack. All we have to deal with, how much health does Gohan have? All right. He has a bit, like 75%, maybe 73%. I could, I could work with that. Oh, he's playing safe. No. Oh, that was, that was slick. He got, we traded. So I thought I had the advantage and see my assist, but I guess he also kind of backhanded Trunks. Unfortunately, mixups. Oh, invincible. He doesn't care. He's got a level three, though level one wouldn't have hit even though it, well, it might not even kill. So he might have had a level three regardless, but still, is this going on even leveled up at all?  This is bad. You want a super dash? You want to? I don't want to if you don't want to. Oh my goodness. Oh, come on. That sheet. No way. Look, he's zero bars. All right. Yabcha, turn around. The threat was the other way, Yabcha. Oh, as you can tell, I'm getting very, getting very iffy on my defense here, to be honest. Oh, I did click a button. Sure. Oh, overheads. He's not blocking. I hit a button. We're dead. No, we're not dead. Let's go. Let's go. Let's go. My confidence is gone. It's depletion. It's depletiated. It's depleted. I don't know why I went for a draft. Why didn't I think he would DP? He was trying to catch everything. We could have blocked. He would have vanished. I would have 2H. Oh, that parallel universe I'm living in inside my mind is crazy right now, but none of it's true. We just started slipping up. All right. It's fine. We could make up for what we did. All right. Just checking. Oh, that's huge. Rob2M, where are you going, Gohan? It's time to put him in the blender. Oh, he didn't want to DP. He wanted to spark. Get off me. Krillin rocks. We are just eating up that sparking time. Hold on. Get him to the corner. Flip. Get him to the corner. He's going to DP right here. Oh, but I knew it though. Why did I still get hit? Damn it. Even with all the knowledge in the world, there was so much time. Oh, that was not supposed to be a jump. Oh, nice super dash. I was going to do one myself. Man, this guy is thinking on the fly, dude. This guy is thinking on the fly. He's actually kind of close to Angel, right? Oh, God. I want Gohan dead. I want his Gohan gone. I'm thinking about it, dude. Oh, I should have done it. If I was going to do that, I might as well have done it. Click a button.  Oh, he only has two bars. He didn't even try. Switch. Yamcha. Yamcha. Let's calm down. There we go. All right. Now it's party time option. Now is the time for partying. All right. We might have to spend a lot of cash here. All right. Well, it was worth. Oh, it didn't hit.  I don't play drugs often. I didn't know there were interactions. Oh, three bars down the drain. I should have just went level one to three. Oh, God, the humanity. Oh, my bank account. Oh, and Gohan's still alive. And now we have to get Yamcha out of here. What are you charging for? Oh, he knows I want that dragon rush though. He knows he had to know. Come on, you gotta know. Oh, we beat Galeclaws. But can he beat my Galeclaws? Nice. Kick them. All right. One of the threats has been dealt with. Gohan, obviously, I would put a little higher up there just because of what he did to me last round. But still, one of the threats dealt with. Gohan cut a rock to the face right there. I can't speak. Nice. Oh, no. No, no. Mercy. Don't take Yamcha at level one. Can't disagree. Just lock it up. Easy as can be. I can't disagree, but that doesn't mean I'm happy about it, especially because it looks like it might come down to a no. No. No. All right. Finally, we get a beam there. And he's been getting good beams all day. It's about time we get one ourselves. All right, Rockokey. Oh, damn it. Oh, no, no, no, no. That's bad. He's reading. He's got reads. Three bars. How do you spend them? Level one, level one. Yeah, he's getting a second level. He's already leveled up once, though. Oh, my God. That was just threatening. It wasn't even a mix. It was just kind of threatening. That was unnecessary. All right. You want to be needlessly aggressive. I knew I should have dealt with Gohan from the start. Let me finish what I should have started a long time ago. Getting dp'd. Wait. Wait.  Don't get a bar. Nice. Oh, my God. He tried to kill me there. Oh, we're dead. No. Huge mistake. All we get up for is a run back. Great games. My goodness. The fast fall into the heavy. It wasn't even like, oh, it was just intimidating. It was too scary. Oh, great games that we got to run it back. Hopefully we get the run back. Oh, well, we didn't get the run back, but we did run into another great priest. So we could. We do have a chance to get quite a bit of points back. The only thing that's scaring me is he's playing GT Goku, Broly, and Ginyu. Broly is scary, obviously, just because he's built different. GT Goku just scares me because of what he used to be, and Ginyu's always scared me. So this is a team of fears, dude. Oh, I should have expected the Super Ash, but I did. Oh, no, not the immediate switch to Ginyu for the Yoki, dude. No, no, no, that's that's the most frightening part of all. All right, that's fine. We kind of created some space there. Nice Super Ash. I'm always late to the pickup with those. Damn, we got out of the corner once again, Ginyu. I don't know if we'll be able to do it again. He might throw in a bit of a mix here. I just mashed. I'm a spark. I'm a spark. I'm a spark. We need the momentum. We were losing a bit of it anyway. 6M? I'll take a 6M. It seemed like a good place to go for it. Who would go for a 6M that early, but it was us all along? Do I take a level three? You know what? In this case, I'm going to say, yes, get that hard knockdown. Start putting on a little bit of pressure. So we want to feel if he sparks, so be it. If not, let's go. Let's go. I thought that would work. It still worked out though, just because it doesn't work. Just because it doesn't work doesn't mean it won't work out. Oh, change the future. Oh, change the future. Rock. Oh, I still missed. Oh, double reflex. Still not good enough.  I want to like Krillin take it from here. No way to follow up with that from Trunks. Other than I guess, maybe if we were brave, we could have EX flipped, but I don't know. That's not me. Instead, we'll take this. I'll go low. Oh, there it is. There it is. The spark. Got hit by a beam and Goku GT is off to the races. Roll the assist. Dragon rush. What is he doing here? He's just going for damage. I was so, I was so slow on defense. All right. That's fine. We probably lose Krillin for this. He does also have seven bars. Can we get a quick Dragon Ball check? Can we get a quick Dragon Ball check? Because I really don't want it to be too high considering the amount of bar he has. All right, Dragon Ball check. All right, force. We're all right. I stop blocking. Going in. Yeah, I deserve it. I deserve it. Truly I do.  Oh, that was smart. I was going to say, I don't think GT Goku would have really worked there. Level three from Brolywil. It just erases Trunks completely. Who do we have left? Yamcha? I think we still got this. I don't know. Oh, the Insta Switch. I never expected that. Oh, nice. Oh, just a beautiful 2H. The 2S, the jumping away Key Blast was nice though. I gotta admit. Oh, that was kind of nice too. Wait a minute. Hold on. Confidence is coming back. Please don't tell me I have to level one to level three to kill here. God, I could have done it too. I could have done it too. I missed the follow-up. I thought he wouldn't let the sliding one hit, but he did. We are dead. We are dead. We are dead. Yeah, we're goners. We're goners. Yeah. Oh my goodness. Okay, we gotta bring this back in the next game.  We got hit. We did get hit. That's probably where it started. Well, I could trace it all back to when I started losing my health. Oh my goodness. The rough. Oh, look at the points. Actually, we didn't lose a lot. We didn't lose a lot at all. Mine is 1,890. I'll take it, dude. All right. Hold on. I'm gonna cook then. Let Trunks cook. That was bad. Don't 2H. We got 2Hed a lot as well. Nice tech. Reflect.  But the thing is, if we can reflect, we can 2H. So we gotta 2H those. Don't just reflect the Super Dash. Let's 2H the Super Dash. I'm sparking. I gotta stop sparking out of fear there. That is, that was a fear sparking. Usually, there's some more strategy behind it. That one was all fear 24-7. Oh no, I dashed up and put myself in the corner. We're down cataclysmic this video. Overhead, we blocked those. Overhead, we blocked those. Probably looking for it. We went low. I was gonna say, probably looking for a dragon rush at some point. The overhead again. We don't block those. That's it for Trunks. Okay, Trunks is out of the picture. All right. Krillin can still bring this back. It's just, oh man, we are down bad right now. Goku, just stop. Oh, nice 2S. Oh, you expected that again? Dashin, that was sick. I was gonna dash back, catch a beam real quick. Oh no, he's moving too well. He's moving too well for me right now. Keeping the corner. There we go. All right, I'll take it. This is the start of something new. We're gonna Yamcha level three. Get a little bit of that blue life back since we're already down a character. I'm really gonna try to be building that blue health back up. Plus, we got Yamcha mix, so you never know. Yeah, dude, keep it classic. Just a little bit of a 2M is all you need sometimes. Now, do I really want Yamcha to stay out? I don't know, but I'm trying to get as much blue health back as possible. So, Yamcha's sticking out. Block those. You gotta just jump and block. I just gotta be jumping and blocking. Oh, bad. I don't know if he wanted Dragon Rush there. Maybe he did, but just wanted it a bit later. We're gonna level one either way just to get him a bit closer. Not for anything too fancy. Let's get that sliding. Get him close. Keep him close. Lock that. Oh, that was supposed to be reflect. Oh, no, we got Keyblast. We got down Keyblast. Oh, no. He didn't do anything too crazy, though.  Rokk assist. Oh, no. Rokk, you poor Krillin got blown out of the water. Oh, no, that was huge. Level three Super Saiyan 4. This is bad. This is bad. I feel like, oh, if we would have gotten that reflect, that could have been huge, but it's not time to focus on the past. It's not time to focus on the past. It's time to focus on the fact that we are about to go 0-4 in this video. If we don't, I'm missing point. The beam. 2S, though. That odd G. Just good. Just great patience. That 2S into Super Ash was a oh, what is with me? Rokk's? He's giving me the special, dude. The Flash Fist special. I am very in the know about the Flash Fist special. Oh, he was trying to Flash Fist me. It's fine. I've played Goku GT before. When I'm trying to flex, I also go for the Flash Fist special. We just had to stay very calm there. Oh, no. He thinks I won't level three of him. You thought. You thought, right? You thought I wouldn't level three of you. It doesn't protect you from all beams. Just the beams that don't matter. Overhead. Yeah, you better run. Okay. Yeah.  You know that he's doing the exact same thing. If you've watched some of my videos in the past, you know exactly what he's going through right now. He's like, I did it for the bit. I was Flash Fisting for the bit, but you can't do bits against Krillin. He won't allow it. He won't allow it. Bits against Krillin on my Dragon Ball Fighters. I also did not. I thought that would give sliding. I'm just bad at the game. That was sick. Koldo almost mixed me. He's moving well. Oh trunks. That too well whiff is just a lot of time. Don't switch. Oh, he switches are too good. All right. Don't do that. Don't do that. Don't do that. Me. Don't do that. That's a mental note for later. Who's he switching? Yamcha. Yamcha. You want a Yamcha? All right. I'll give you a bit of Yamcha. Why not? If you want a Yamcha, I'm not going to not give you Yamcha. Is that going to miss? Okay. It didn't miss. That was all luck, but I'm fine with that. I have, I have since gone silent, dude. Okay. He's getting used to that as well. We should stop doing that. He's making adjustments. He's very good at adjusting on the fly. I just wasn't blocking. Oh, there we go. Finally caught him off guard with one of those and that is going to begin you. Please just two bars. Okay. Good. I was really thinking about spending a third just to be safe, but I also did not want to even faked it to dash up. Oh, it was smooth. It was smooth until we didn't block. Also, sorry for going quiet. I'm focusing right now. Check that out on the play something instant replay. My God, ref. Come on.  Come on. There's no way I had to give Broly a little shave or something at least. At least show him losing a lock of hair. That was sick. That was sick. That was sick. Come here. You think I don't got cross ups? Oh, you're right. Oh, I thought I would recover in time. Are we still sparks? I'm gonna spark now. Oh, Jesus. Oh, he would. He did the slow one too. He had to. Yep. He did the slow one. God, he's an on the ball player. He's on the ball. Hopefully this is more blue health. Get off me. Oh, don't even try to get out of that one. Come here. Oh, I got to just be prepared for Superdash there. Okay. It's Krillin against these two. They're both pretty low. They're both pretty low. Come on. Clutch up. He got behind me. He got behind me. He got behind me. He's fast. Goku GT is slippery. He might even hit me with a level three. Nope. That was genius. Just genius. That was really nice patience. I went for it. I thought he'd be in the air unready. I didn't want to get beat to the punch. That was great. Nice landing. Oh, that one really did us in there. But great games. Oh, man. I'm glad we were able to get that one. At least we clutched out that one. Hopefully you guys enjoyed this video. If you did, definitely let me know. Download the comment section while you're down there. Make sure to like button and subscribe to the channel if you want to support it. And you could also check out the merch stuff that we have now. That's also cool. I'll wear the other videos on your screen if you want to watch even more right away. Other than that, I have been Dr. Doia and I will see you in the next video. Have a good day.


Sit there, sit there. Secretary, do you have any kind of commitment on a summit? I'm going to discuss with the President. How are you feeling? Great. I had a very good time in Moscow. Good week. Mr. President, is this a no-win proposal for the West?  You may have to talk about that. Okay. I have a statement here. I have just received a full report from Secretary Schulz on his talks in Moscow and his consultations with our allies. And George, as usual, put forward our positions in Moscow with firmness and great skill. It's clear to me that the visit was very useful in advancing the dialogue between our countries in a number of areas, human rights, bilateral relations, regional issues, and arms reductions. The contacts the Secretary had with divided families, church groups, private individuals paid public tribute for the courage of those in the Soviet Union struggling for human rights. He made clear to the Soviet leaders that self-determination for Afghanistan and Soviet troop withdrawal were essential to peace. Important progress was made in arms reductions on intermediate-range nuclear missiles. We've narrowed the gaps a little more. After we consult further with our allies, we may have new ideas to offer. I remain optimistic about an agreement this year. There was movement on nuclear testing talks and on a ban on chemical weapons. On strategic defensive reductions and defense in space, the talks were detailed and useful and will intensify. When I return to Washington, I will meet with the bipartisan congressional leadership to review this week's progress. It's my hope that the process now underway continues to move forward and that Mr. Gorbachev and I can complete an historic agreement on East-West relations at a summit meeting. And in that connection, I will consult personally with our NATO allies on further negotiations and plans. And again, I am deeply grateful to the Secretary. Thank you, Mr. President. Mr. President, has Gorbachev boxed in the alliance with his position on shorter-range missiles at this point? George, would you like to answer that one? I don't feel boxed at all. I think we had a very good meeting in Brussels this morning and reviewed the whole process. And I think the opportunities before us and we have different alternative ways to go are all very positive. Well, did the NATO leaders indicate to you that they're prepared to give up not only the medium-range option but also the shorter-range option at this point? Well, we're discussing that and they're taking counsel, and we will come to a view. Is it fair to say they're disagreement at this point, Mr. Secretary? No, it isn't fair to say that. It's fair to say that people are considering an important offer that the Soviets put on the table when I was in Moscow. But we don't just react with that. Mr. President, there was not much progress on start. Are you reconciled that there will not be time when your President seeks to negotiate a cutback as to teaching missiles? No, I'm not reconciled with that at all. We feel that we take that very seriously and we intend to keep on that path also. Are you eager to see a summit this year, Mr. President, regardless of whether you've got an agreement in hand on INS? I think... I look forward to it and hopeful that we can have a summit but it must be one that is carefully planned and prepared and that there must be something that we feel we can complete. An arms agreement? Does it have to be on the table at that point? There has to be some substantial agreements that could make it worthwhile having a summit. Mr. President, we're just suggesting a moment ago that there will not be another summit unless there is virtual agreement on some kind of arms control accord. No, I'm saying that Mr. Chair, if you're prepared and that we should have a prospect there of being able to arrive with some very substantial agreements. Mr. President, what about embassy security? Is there any progress? Any change? That is a... Well, one thing was clear when I was there that the advanced teams and the communications people did a terrific job. The old can-do American spirit came out and they provided me... I've been to Moscow quite a few times with the best communication setup, the best set of meeting... confidential meeting places that I have had the experience of using. So, our guys came through. Well, I think we've all been here enough and George has to get back down the mountain. George, thank you. Thank you. Gosh, Mr. President. Isn't this something pretty... I am the agenda. We have the... Sorry to break up your... But that's California. George, what's your job? It's all about... Are you going to be able to sell us to the Europeans? We'll be back. Stay down there, George. Stay down. It's the same. Thank you.