Okay, so we are off to chapter two and we're talking about analyzing algorithms and we're going to be doing This is the introduction to it We're going to be analyzing these algorithms as we go along through the rest of the book So we'll need to know how to do it. Oh, that's nice to know that they can see my screen. Okay. Oh Somebody else joined us. I see Okay, well, I'll find out who it is later on by the way, whoever it's just joined in if you have any questions Just turn on your microphone and ask So the question is When we have an algorithm, how do we know whether one algorithm is better than another and It says there's a difference between the program and the underlying algorithm So let's take a look at this program here Where we're finding the sum of the numbers one through n So you give me a number n and I return along by the way 64 bid integer that means I have more More and larger numbers that I can return and then I run a loop From one up to an including n and return that value And so there's nothing particularly new or exciting here So that's so that's one way to find the sum of the numbers one through n Everybody okay with this code? Do you actually need me to run it or do you gonna believe that it works? Oh All right. Oh, all right, so we're gonna do here's copy this I guess I'm gonna have to make a new Save this And this is called find some I think well now we get to see if there's what we put in the book is correct And we run it and the answers 55. Yeah, so it works So everybody okay with this code, right? Yes, any part of it that you might not be familiar with? Yeah Okay, well it's adding up the numbers from one up through and including ten correct so The sum starts off at zero agreed So when I is one we add one to zero, which gives us a one then I becomes two now we take one plus two which gives us three and The sum keeps getting updated every time through Okay, and if we were to go into I guess I could do it in J shell So I have one plus two plus three plus Man, I'm here. Okay. It's today is trouble typing day. I can see that already Okay Nobody believes a word I say okay wonderful So there's one algorithm for doing it a program to get the numbers one three in Now the book then goes on and says okay, here's another one If we look at this this finds something to this is pretty awful. Okay It's exactly the same program except all the variable names are really stupid so that it's harder to read So the question is this a better algorithm or a worse algorithm the answer is it's the same algorithm It's just a really badly written version of the same algorithm But I'd still say it's worse. Okay, because it's less readable. Um, it also has this useless Assignment statement here that doesn't get used for anything worthwhile so the question is Can we improve it? We okay? We can make it worse by giving us if ridiculous names Is there a better way to do the addition of the numbers one through n rather than running a loop? And it turns out. Yes, there is a way to do that and what we can do is Third, I'll see where's my Sorry, I'm trying to figure out where this is formula object there namely the sum from i equals one to n of n is n times n plus one over two So that formula is a lot easier. Yeah, I can do it with just that one thing So what we'd like to do with this let's let's go back back to this first one here Where we're running the loop the question is how well does this perform? How good a performant algorithm is this and one thing that we can do is we can record the amount of time it takes to execute this method So here I have this Sum of n that's the same Method that I had in the other program and what I'm going to do is I'm going to run 25 trials of this I'm going to ask the user find the sum from one to n and Then what I'm going to do is I'm going to use something called the nanotime method and it's in the system class and what it does is it records this Java virtual machines clock time By the way, the Java virtual machine clock time has absolutely nothing to do with this one down here Which is the time of day has nothing to do with the time of clock on the time of day on the wall clock over there It's an internal timer that the Java virtual machine uses So I'm going to record what my start time is Then I'm going to Calculate the result and then my elapsed time. I'll look at the system clock again And subtract the start time and divide it by a billion Are you all familiar with that notation 1.0 e9? Have you ever used that before? Now that's one of the nice things that you can do in java I can say for example 1.735 E4 and that'll be 1.735 times 10 to the fourth So it's just a convenient notation because 10 to the ninth. I don't want to have to write 90 as a loose count I can so say 1.0 e9 All right, that's big enough that it's not gonna it's gonna give me that so I wonder what would happen if I tried to say integer 1.0 e9 There you go, and there's there's a billion. Okay, and then I'll print out how long it took So I'll say which trial it was what was the sum of the numbers and the time So let's compile that And run it And let's say the number up from one to a million And this is approximately how long it took now. You'll notice something really weird here The first one took 0.002 the second time 0.001 and then everything started to sort of calm down and get to Something consistent towards the end so the question is why did the amount of time it took Take longer the first time Then after 25 trials And the answer is because of the way the java virtual machine works and there's some things that we have to understand is that javaset virtual machine has a Setup time So the first time I run a program it has to do all that setup So that adds the amount of time for my first trial The java virtual machine also does something called caching Which means it stores previous results in memory So that means if there are some Things that are being done over and over again It can take advantage of that without having to recalculate it but building up that cache takes some time And also the java commercial machine Does remember what we were talking about earlier the garbage collection? Did I talk about that in this class? Does anybody remember if I talked about that? get when you allocate objects in java And you've changed the references to them sometimes you'll have a reference that you used and now you've got something out in memory But nobody's referring to it anymore And so what happens to that extra memory that's just hanging out with nobody referring to it? That's considered garbage and the process called the garbage collector Comes through and sweeps through it and gets rid of all of that and reallocates the memory so other people can use it Um, so all of these things take time And that's why Our time taken for the first trial of the algorithm Will take longer than the second And eventually everything settles down Which is what we see here So in these last few trials It's pretty much the same amount of time for adding up the numbers one through a million Everybody okay with that so far? Yes? You sure you're okay? Hey guys, I see some doubtful looks out there Now back to the algorithm analysis, which is the point of this whole thing I had to I'm sorry. I had to talk about this because otherwise people look at this They what the hell's going on with java java is really weird Okay, let's say this as Timing 2.java And this time when I want to find the sum of n I'm going to say long result becomes n times n plus 1 divided by 2 And then I'll return the result Okay, so now we've changed the algorithm from the loop Which was this one here to The formula And now let's see if there's a difference. Let's compile this and run it And we're going to go up to a million again And you'll notice that now everything is very consistent again the first time through everything's pretty wild But everything settles down and it's a much more efficient process, isn't it? Why because instead of running a loop a million times I'm doing one calculation So instead of a million steps I have one step and You can see the difference Do you want me to run the other one so you can see the other one again what the times were? Yeah, let's uh grab these last three trials here. So this is the one line formula and here's the loop Which we will go here And this is called timing dot java By the way, I don't have to compile it again once it's compiled and I haven't made any changes to it I don't have to compile every time And then we're going to go to one million So yeah, that's it's a big difference Okay, the next thing we're going to need to talk about is The number of steps that are needed to do something So if we wanted to compare these two algorithms to one another We might want to find out how many assignment statements get performed again So in the first function the one where we had this we would have One assignment statement. Let's go back to Let's go back and take a look at this one again. So here we have one assignment statement Plus a million assignment statements. So the time taken the number of steps, which is called t notation so loop The time taken to do this sum is one plus however many there are Using the formula The number of assignments we have is two Uh, do we have even two or one? I don't remember. I think we may have had only one there Yeah, there's only one. There's only one assignment statement. Okay. Well, there we go And so notice by the way, this is independent of Which number we're using So t sub n is the time it takes to solve a problem of size n which is one plus n steps The exact number of operations is really not as important as what's the most dominant part of the t of n function So as the problem gets larger some part of this function will tend to overpower the rest Let's come back here. So for example when n is three, okay That one is one fourth of the whole time. Isn't it that does first assignment statement But when we get to a million when we're doing this sum from one to a million Is that extra one assignment statement making a gigantic difference in how much time there really is So what's really taking over here is the one the important part or is the end the important part of this The end is the important part. And so we're going to say in what's called Big o notation And the o stands for The order of magnitude of a problem So for the using the loop It's o sub order n. I think it's sometimes written o o of n or o n The time is proportional to the number of To the number you're adding up to using the formula It's order one The time is independent Of what number you're adding up to And although t sub n is interesting, we are most often Comparing things and when I say order n versus order one while order n Is always going to be take longer than order one order one is the best we can ever do Now sometimes there's also the best case worst case and average case performance Okay, so sometimes for example, if you're sorting numbers And let's say the numbers are in the exact reverse order of what they're supposed to be that might take longer than if they're just randomly Randomly arranged So the worst case might be when something's completely sorted, but in the wrong order That'll take the longest amount of time Best case for sorting and there's a list of numbers is if it's already sorted at the very beginning because then there's nothing to do And the average case is if you just have some random numbers that are thrown at you And they're in any old order and you have to figure out to sort them that's going to be the average amount of time So sometimes it will be characterizing an algorithm by how well it performs than the worst case How well it performs in the best case and how well it performs on the average And here are the common functions for big o if it's order one it's called constant time The amount of time it takes is totally independent of the number of items that it's manipulating Log to the base n is what's called a logarithmic or algorithm And we'll see some of those later on in the course Um the loop that we did to add up the number the sum from one to n that's linear time Because it requires um proportional to the number of items log linear is n log n and again, we're going to see some algorithms now that And then quadratic is n squared If the amount of time it takes is proportional to the cube of the number of items that's called a cubic um algorithm And if it's two to the end, that's called an exponential algorithm And this goes from fastest to slowest Now all of this i'm telling you is pretty much to build up the background of what's what we're going to be Talking about the rest of the semester. That's why you want to be familiar with these terms So say, oh, that's a quadratic algorithm. Do you think we can improve it to be linear? That's going the right direction Quadratic is n squared linear is on the order of magnitude of the number of items And here's one that we can analyze So if we were to go and look at the number of assignment operators We have four operators here, correct? And These are the first four The second term is three n squared. We have three of these But they're this they're done n times for the outer loop and each outer loop goes n times for the inner loop, correct So we have four plus three n squared And then this last one is n because we have two n we have two n here. We have two Assignment statements done n times yes And then finally we have this one assignment statement out here at the end So there's our t sub n is four plus three n squared plus two n plus one Which is three n squared plus two n plus five Now if i'm going through only one item Five is going to be The most important part of this But what happens if I have a thousand items which one's going to be the most important part of this now The three n squared the two n or the five The three n squared And in fact the n squared is the dominant part and the three is really not the factory is not important So we call this fragment of code is going to be order n squared The amount of time it's going to take to do this is going to be proportional to the number of items squared Because of this nested loop here This nested loop is the thing that just kills us So when n gets up to a million, I mean if we have three of us a million times a million That's like an enormous amount, right? And the two million and the five are just saying who cares about those that's just the rounding error almost This is a good example. I really like this one What happens if we want to find an anagram? So for example, it's strings taster and treats are anagrams that means you have One string is a rearrangement of the other string And here's the first algorithm that they give you in the book And what they're going to do is they're going to check off each letter in one string against the letters in the other string So what i'm going to do is i'm going to have string s one and s two Well, first of all if the lengths are different than they can't be anagrams. They have to have the same number of letters, right? Yes So what i'm going to do is i'm going to take the second string and make it into an array of characters I'm going to j so i'll show you what that means if I have string s is let's say Then I can say s dot two char array And that'll give me an array of six items. It's very nice to be able to split that apart Then as long as I have an anagram I look Okay, I'm going to have a position one. This is going to be my first string I guess I have to write this here. So this is So I'm going to do is I'm going to start here This is going to be position one Is going to be there and then I'm going to go through all of these to try and find the letter t And the moment I find it I'm going to replace it with a dash. So I never see it again So this t I found it here And I replace it with a dash Now I go to a and then I go here to position two. Is this an a no Is this an a no is this an a no is this an a yes, it is terrific. I'll replace that with a dash Then I go to the s and I go through here looking for the s I find it If you're there Then I have another t No, no, no, no. Yes And I replace it and eventually at the end I'm going to either have all dashes or I'm going to have something left over So if I have something like Cards well I need something that's not an anagram here. Um So the c comes here I have a dash The a I found to find that I'm looking for a t and it's not found anywhere in here and I can stop right there Because if I have something that's in one but not in the other one, then I guarantee it's not an anagram, right? And that's why I have This thing I have a found so I have a boolean that will let me get out of my while loop earlier So as long as I have letters to go in my first word and I still have an anagram I have to go through all the letters that are remaining in the second word And I go through each one of those and I say okay is the string character Equal to the one in the array If it is cool, I found it otherwise I have to keep moving If I finally find it, then I'll set it to dash That's what I was doing by hand if I didn't find it then it's not an anagram And then I return whether it's an anagram or not And of course Let's call anagram one So we have true true and false Because these are anagrams these are also And abcd and dcda are not Now question is What is the performance on this algorithm? Okay, and the answer is the performance on this algorithm is going to be order of n squared Why let's say my first I have because I have a nested while loop For each letter in the reverse string I have to go potentially and search through all of the characters in the other string to find it And in fact each of the characters will cause an iteration through up through n characters of the list from n1 And in fact it happens to be the same formula that we had before which is One half n squared plus one half n And the n squared dominates the n term we can ignore the factor of one half And this is an order n squared Because this is how we analyze an algorithm to figure out and it's not a bad algorithm It's the first one that most people will come up with But if I have something like that's a hundred Let's say letters Then it's going to take a while because it's going to have to potentially in the worst case Grow through something like well about a thousand times in 1001 divided by two Which is 50,000 some odd combinations Now another way to do it is we could Sort both of the strings So if we sorted all the letters and then we'd compare them one at a time Then we've if they're all the same then it's an anagram, right? It's a pretty clever way of doing it And I'm not going to run this program, but again you can run it yourselves here And What we're going to do is we're going to Change each one of them to a array of characters Sort them both And then we'll have a while loop And you're going to say oh well this is order n because there's only one loop. It's not nested right But there's a catch here We had to do the sort so the question is how efficient is the sort? And it turns out that most of the sorts are actually n log n So those are going to be or order n squared or order n log n. So the sorting operations will really dominate the iteration So we haven't gotten any real improvement here Now this one's really awful. Okay Remember how I can say you can make things better and you can make things worse This is worse What I can do is I'll try all the possibilities I'll generate the list of all the possible strings from s1 and then see if s2 is one of them well That's really bad because it's Do you all know about factorial numbers? How many people you can tell me what if somebody can tell me what effect let's say three factorial what that means Yes, it's a number times the number minus one times the number minus one from that all the way down to one So for example, if I have uh, let's say six factorial and it's used it's written as six exclamation point That's essentially six times five times four times three times two times one, which is 720 Now can you imagine what that's going to look like if I have a 15 letter that I'm trying to get an anagram That's 15 times 14. That's if that's a gigantic number This grows faster than quadratic or cubic. It's it's It gets really really bad really really fast So this is right out. In fact, if there's 20 car 20 characters Here's how many possible candidates there are Okay, so this is not a good solution However, this one's an interesting one Namely that somebody could say well We could check off the ones that as we get them right But when we have an anagram the number of a's let's say letter a in the first word has to be the same as the number of letter a's in the second word And the number of letter x's in the first word has to be the same as the number of x's in the second word Because if those don't match and they can't be anagrams Does that make sense Do you need me to just show an example here? Anybody need me to show an example? Okay, good And here is the Code for that So let me copy this here Nope Paste it there And what we're going to do is we're going to create two arrays of length 26 So this is only going to work with letters, but that's okay And then we're going to subtract the letter a to get our index remember characters are represented as integers in java So we can say Yeah, I better show you this in in in j shell So for example, if I have the letter let's um C minus a that gives me two If I have x minus a That's then Gives me is actually a 24th letter because it started zero and a So I can subtract characters from each other and get integers And that tells me how far away one is from the other And I'll use that as my index Into this array of length 26 So I do that for it the first word I do it for the second word And then I go through and say, okay Let's go through all of them and check to see that the counts are equal and the moment they are not equal It's not an anagram anymore And that bounces us out of this loop I'm really big by the way. I'm using compound conditions So as long as I still have letters to look at and I still have an anagram Check to see if the counts for the letter are the same if they are great Otherwise, it's not an anagram and then when this comes out false that ends our loop This is a way of doing a while loop without having to use the break statement Now probably everybody else in the world who writes this will use a break statement, but oh well Now what's the performance of this one is it order n squared There's two loops, right? So should it be is that also n squared or not? How many people think it is order and still order n squared How many people think it isn't Okay, how many aren't sure Okay, so people aren't sure. Let's see. Here's what I do if I weren't wasn't sure I said, all right. Let's let's do the t the t business here uh view Is there a way for me to split this or not? Excuse me here split window. There we go. Okay, so here I have I have Two assignment statements there And then here I'm going to have let's just say this is the length of the word is n. Okay, so I'm going to have Plus two times uh Why because I have two assignment statements in here and they're going to go n times agreed Now this loop is not nested in the other one. It's a completely separate loop. So independently this is also going to take two n And then here Since I might go 26 times I'll have I'll do either one or the other So that means I'll have 26 is the worst case Okay, so that's 28 Plus four times n What's the term that's going to dominate this? um equation Is it going to be the twice always going to be 20? What if I have a thousand? Let's say a thousand letter word that I'm trying to figure out if it's an anagram to another thousand letter word Is the 28 going to be dominant or the n going to be dominant? The n is going to be dominant and therefore this is going to be order n Algorithm So if you just count them up you can say, okay, what's the what's the Number that's going to take the Take the biggest share of the time that has to be be used And this is an order n algorithm. So now this is definitely an improvement remember this first one Which was really quite nice. I mean there's nothing wrong with it was order n squared Whereas this last one here is order n. So it's a preferable algorithm if we have a long string of things that has to be analyzed If we have only like four or five And you know 25 versus five Now you could live with that But once we get to more than let's say 15 or 20 then the difference becomes pretty significant And you might want to do these first. So let's yeah, let's do these here So what about this one? Is this going to be order n n squared log in or n cubed? Remember, these are not separate loops now. They're nested I've got to vote for n squared. How many people vote for n squared? By the way, when I I'm going to have to rewrite this to use a superscript two at some point And yes a singly singly nested loop Like this is an order n squared Whereas this one here where I have the two separate loops that's going to be Order n correct So it's at an order of two n, but again, I can ignore the constant Oh, this is an interesting one This one you have to look at a little bit carefully Let's say n start in fact, I'll I'll give you a hint here. Let's say um I starts off at oh 64 Okay is 64 greater than zero Great then we add two plus two and then I becomes 32 32 is greater than zero. We do the addition 32 divide by 2 is 16 So question is this going to be an order n or not? Yeah, you won't try see It's log n because the log n usually when we say log is logged to the base two so when we have 64 things it'll take that loop will run six times 128 it'll run only seven times 256 it'll run eight times So for every doubling that we have it takes only one more time through the loop which is pretty darn good So log n is much better than n in fact The value I is cut in half each time so it'll only take log n iterations You'll see these patterns and you'll become familiar with them as things go along So now what we'd like to talk about is what's the performance for operations on things like array lists and hash maps And we're going to show you how to do timing so you can time things now I'm gonna Go a little bit ahead where we're coming up on this Go to array lists here Okay, oh I need to go back for one thing here. I forgot to mention this the book does mention it You'll notice here in our original anagrams We really did not need to do uh, we needed to Add a little bit for the this we need to use a little bit of extra storage for this array Here we needed to create these two extra arrays for the count counting of how many there were So what we're doing here is we're trading off time and memory And this is important. I should probably put that in the notes Okay, we'll give you an increase In speed But the cost of four Java program Um improved an order n squared Not great.  And at the expense of six integer arrays That's a great trade-off by the way So ordinarily we say you can often trade time for memory and vice versa So we talked about array lists already. Yes And they need a refresher on those or not Sure, let's go back to j shell and take a quick look at this Important java dot util dot array list And I can have an array list and this since this is a generic type I have to say what kind of items I have in my array list And I give it a name becomes new Array list And again, I could put the word string and repeat it, but I don't have to And then I can say city list dot add Of let's say kupertina City list oops And then if I look at city list there, they all are and I can say city list dot remove The item at element one It returns it to me And there it's taken out of the middle And so these are the operations that we can perform and we'd like to know how well they perform And here's the big O efficiency. So if I want to get something to index, that's constant time No matter how big or how small my array list is Reaching into the array list and grabbing out the value for one of them takes exactly the same amount of time Setting also is constant time If I want to add n items, it's order n To remove something at a given index is also order n Why? Because I might have to move all of the Ones after the one that I got rid of I have to move them all down To get rid of that empty space Does that make sense or do I need to go into that or explain that further? Okay, great. We're good on that. And then if I want to find the index of to say, okay, is something in there For example, I can say city list dot index of I might have to go and search through the whole list to find out whether it's in there or not So the amount of time it will take me to find out if something is in an array list Is proportional to the number of items in the list The amount of time that it will take me to get rid of something Is also proportional to the number of items in the list. The more items there are the longer it'll take me When you remove the first number all the remaining numbers have to be moved left If I remove the last number, that's the best case. So again, remember we have worst case and best case So for removing something from an array list, the best case is removing the one at the end. There's nothing to move The worst case is removing the one at the beginning because everybody has to move down to the left in order to get rid of that extra space And in fact here if I have two million items Here's how long it takes to return to remove the first item and those for a million items It takes less time and for a hundred thousand items. It takes less time Why because there's less to move So it's proportional to the number of items in the list Whereas removing the last item and this should be item not time by the way. Oh, no, this is a time. Yeah Is pretty much the same no matter what how long my list is whether it was two million one million or a hundred thousand items And now the question is well, how do we write this program? again And and I think this would be something that I'm going to just start on tomorrow on wednesday rather, okay So on wednesday, I'm going to go through the Array list and hash maps and we can talk about those and also talk about the assignment So this is a good time for a break and after the break Do you want me to talk about the assignment anyway so that if you want to get a head start on it during lab? Okay, one thing I'm going to tell you is the way I'm doing the timing here This was what's called Maybe I'll give you a term here. This is an ad hoc solution. Okay solution that's written to fit one case specifically And not generalized So I was looking at what they had in the original book when I was translating this from python into java And I wrote an ad really big ad hoc solution, which I don't think was a really good one And so, um, I'm debating with myself on wednesday Whether to show you the better solution that I came up with um yesterday So I'll carry on that debate with myself and maybe ask one of the other professors and see what they have to say with this about it So, oh, let me continue sharing the screen. I guess So is okay. Should I go should I go over the assignment now or after the break? Now, okay, so And uh algorithm analysis So here's what we want to find out. We want to find out that index of really is order in And then you're going to write a document that describes and summarizes your results So this is really open-ended you can decide how to do the verification You can decide how many items to put in the array list how many times you want to call index of So you decide all of that but you have to at least let me know in your document And in fact, let's go and take a look at a summary So it'll have the course name your name and the date Any format that you like And so this person wrote a program that does the following for seven different several different array list sizes 100 200 300 400 500 million items Sets up an array with the integer zero through whatever the size is Starts a timer generates 10,000 random integers and calls index of with that random integer So it just randomly generates a whole bunch of numbers and tries to find them in the list Then it did each of those 25 times and it averaged the last five timings They discarded the first 20 because remember that stuff with the java start-up time and the caching and all that kind of stuff We want to have time for the algorithm to calm down and stabilize And so that's what this person did when they wrote it And then here's the results of running the program So the number of time in seconds You'll notice that a million takes a lot more than a hundred thousand And it looks like it's proportional and the best way to see this proportional is if you draw a graph of it And it's pretty much of a straight line very nearly linear And then they also calculate the correlation coefficient if you're really into statistics, you can do that Again, this is something you do not have to do in your summary But that will also help you to check to see if there's a linear relationship I will not take off for spelling grammar punctuation unless it's so bad that I can't understand what you're saying So if you have problems proofreading if you want to send it to me to do proofreading I'm really I really enjoy doing proofreading So if you need to say hey, can you correct spelling and grammar errors? I love to do that kind of stuff And you'll upload two files The java file and also Your um document So it could be docx. It could be pdf um Please don't please do not send me one of these macintosh only formats because I don't have a macintosh to read it on and um I've told you all about librae office. Yes Okay, if you want to use librae office, that's fine also Because it also by the way gives you the opportunity to Type in formulas right away and just see them nicely typeset, which is really sort of cool So that's the assignment. I'll go over this again on wednesday And I'm going to stop sharing and I'm going to stop the zoom session at this point Unless there are any questions from the folks who are um watching Okay, very good then. I'm going to stop recording And stop sharing see y'all I'll see you on wednesday, I guess


Hmm looks like this will be a cool breeze Rayfield's mind what what are you doing? Keywells my old pal from the hood see you haven't grown an ounce white Hey Detective stance in a while, huh?  I'm waiting left for Atlanta looking for a slice of happiness Guess you didn't find it. I've always maintained.  Give us a break, huh? Lock us up.  Mr. Fujioka will be taking them now. It's a waste of effort I have no time to testify or play it on investigation Suggesting we let him go sir. I suggest you toss them in the sea Catholic broken so this trash doesn't float you hurt him. It's like this will be a cool breeze Fields mine. What what are you doing?   Look at first slice of happiness. Yes. You didn't find it I've always maintained.  Give us a break, huh? Lock us up.  Mr. Fujioka will be taking them now. It's a waste of effort I have no time to testify or play it on investigation Suggesting we let him go sir.   I'm waiting left for Atlanta looking for a slice of happiness Yes, you didn't find it I've always maintained.  Give us a break, huh? Lock us up.  These the thieves follow the three tracks Got them in custody. Mr. Fujioka will be taking them now. It's a waste of effort I have no time to testify or play it on investigation Suggesting we let him go sir.  It's like this will be a cool breeze Feels mine. What what are you doing? See you haven't grown an ounce. Why detective stance in a while, huh?  I'm waiting left for Atlanta Looking for a slice of happiness Yes, you didn't find it. I've always maintained.  Give us a break, huh? Lock us up.  Oh They're proud be out early Are these the thieves ordinary trash? Got them in custody mr. Fujioka will be taking them now. It's a waste of effort I have no time to testify or play it on investigation Suggesting we let him go sir.


Good day. May 40 here by the Sydney Opera House So I'm listening to this New Yorker article on do we change over the course of our life? All right, there's some people who see things more in terms of you know staying the same continuity and other people who see themselves more in terms of distinct phases, but there's one breakdown of personalities from study at University of Dunedin in New Zealand and they divide people into three camps that there are those who move towards life And so I think that would be roughly Compatible With the secure attachment so when people like you and you like them and they're good for you you move towards them And people treat you badly you move away from them so So there's probably the dominant personality type and you move towards life meet towards people and then There are those who move away from people so if you're Joining a new school in eighth grade right those who move towards will make friends join clubs Get socially active and then those who move away Will read a lot of books so I guess I'm a personality that likes to move away And then those who move against will cause trouble so Got a part of me that likes to move towards life large part of me that likes to move away from life and live stream and read books and Then another part of me that likes to move against life And this refers to the criminally inclined and the thrill seekers so I've got some Thrill seeking personality traits, but I think you see these tendencies all around you Are those who move towards other people towards connection of those who move away And there are those who move against and so if you have a relationship with someone who moves away Then that person's not likely to be very social All right, we have to take them to a lot of social events. They're going to need a lot of time on their own So we're probably talking about an avoidant personality Then someone who moves against the world All right, so someone is low in agreeableness It's going to create a lot of turbulence. It's going to rub a lot of people the wrong way All right, so Easiest relationships I would think would be two people who tend towards who towards life Towards each other towards other people towards connection And if you get a marriage between someone who moves towards and someone who moves away I think there'd be a lot of Emptiness lack of connection and If someone who moves towards life marry someone who moves against life Against the world Right again a lot of conflict Have a hard time imagining that would go too well There are socially effective personalities Less socially effective personalities, so socially effective personalities above average in agreeableness above average in conscientiousness more extroverted than introverted Open to new experience low in neuroticism and Jews generally speaking have These types of personalities These are the personalities that tend to make for success in life Be more extroverted than introverted to be less erotic To have openness to new experience To be more conscientious than average and to be more agreeable and disagreeable That people are more agreeable Obviously Kind of get along better with other people. They're going to have a better reputation in the community people are more likely to like them so There are studies that show that Jews have more socially effective personalities now we can shape our personality to some degree we all tend to become more conscientious as we age We tend to become more agreeable as we age we tend to become less neurotic as we age and We can do concrete things to become more conscientious more agreeable more extroverted More open it's new experience and Less neurotic and we're looking out of the beautiful Sydney Harbor As the Sydney Harbor Bridge there are tours where you can walk to the top. I think I see a bunch of people Top the Sydney Harbor Bridge right now


So thank you guys for coming here, it's a pleasant day outside, I'm sure, you know, there's many different places where everybody wants to be, but thanks for choosing to be in a place where it talks about mainframe and open source together, you know, paradoxical statement maybe, but we'll talk about it. And thanks for the Linux Foundation for organizing and for selecting me as the speaker, and the third thing is thanks to the entire team, there are a lot of people who worked on this one, I'm just sort of representative for speaking on their behalf.  So I'm at least going to partially unravel that so that you guys have a better understanding on how that happens. So, I myself am Seishi Gurudanti, I've worked in the bank for around six years, worked on the data platform previously, and this part of the presentation is actually representing that work from while I was in the data platforms. I live in the Bay Area. And then that's my kind of LinkedIn profile. And then I write, read, travel, and then do photography. So that's, that's kind of a little bit of put myself and when it used to work for me, and was also part of the team, and couldn't make it here to do different circumstances, but that's what this is. Okay. So, a little bit about US Bank, before we can get into the actual technical presentation so that there's a context. US Bank, we serve consumers, we serve small business, and we serve large enterprise customers. This talk is mostly about the consumer segment, and that consumers are part of the consumer and the business banking, and these percentages represent the relative revenue of the bank that gets from different business lines. So, consumers are primarily the consumer and the business banking where we have checking and savings deposits, and then they're part of the payment services, we offer credit cards to consumers, and they're part of the wealth management, because we have wealth management solutions and products to the consumers. The last business unit, the corporate and the commercial bank, we don't have any consumers there. So that's the kind of the landscape on where consumers fit in. And then if you look at the consumer transactions, especially the digital transactions, they've been going through the roof relative to kind of four years ago, and then they're across the board. These are just representative of a couple of them. This one represents the digital transactions and the mobile deposits. So I'm going to explain a little bit about, you know, these are not just like new customers signing up and then doing digital transactions. These are, there's a fundamental change in how the products are evolving and how customers behavior is changing, and that's the trend towards the digital transactions. And the talk is about the mainframes and how we kind of migrated, and these increase in the digital transactions is probably the sort of the main driver actually. And so now I'm going to sort of dig deeper into why the digital transactions are actually going up. I'll kind of skip that. So if you look at the digital transactions, why are they going up? One of them is that the physical analog of it is typically less transactions. If you ever went to Blockbuster and then, you know, rented, you probably would have one transaction for renting it out and then returning the sort of the DVD. You had two rows in some database somewhere. But if you are looking through like Netflix, you would log in, you would browse, you would browse for like 20 minutes and then maybe watch for 10 minutes. So the number of transactions that you're hitting the server is actually phenomenally higher. That's the same trend even in the bank. Anything that is physical analog to what the digital equivalent is that there's more transactions that are hitting your kind of backend. That's kind of the core philosophy.  That's the Kodak example. The number of photographs are actually quite high, but Kodak doesn't exist now. So the other one is customers are kind of having a different kind of engagement. It's not just you have a point of transaction. You went to a bank or a branch and then you have one interaction, but customers are expecting like a 24 by 7 engagement. And then you kind of see that, you know, driven by if you look at right share apps, you order a Uber, you are able to see the driver every turn that they take. Imagine the number of transactions or the number of server, client server responses that you're actually getting that same thing with, you know, delivery apps, same thing with airlines. You can track your baggage. You can track the flight. So that kind of a continuous engagement is being requested by the customers. And there are kind of equivalent analogs to that. And in the banking space where you want to track transactions, you want to track application status. You want to track the mortgage application status. So there are a lot of tracking that needs to be done. So that actually has an impact on the how we kind of evolve the architecture. And the last piece is that, you know, the complexity of the products is changing. So not just we have, you know, for business lines, but with the business line, we have a lot of products and these products are being bundled, re-bundled and then sold. So that means that the architecture of a monolith, you know, cannot kind of function very well. Many different different teams cannot work on it. So all of those are maybe standard things that you guys all heard. So I'm not going to get too much deeper into that, but these are some of the main drivers for why we went on this journey. So if you look at how the sort of the architecture was, I mean, this is a very simplified version of it. There's a UI. Most likely it was a mobile app or a web app or a banker app. And we used to have a thin domain layer, which it was not there was no in business logic or anything, but just wasn't like an API translator, some level of security, like a gateway kind of thing. And then that would hit the mainframe and mainframe had the business logic, the rules validation and also the database. So this is how it was. And then, you know, simple example is that you change an address flows through all the way to the mainframe checks for the address validation and then it saves it and then you can retrieve the same information. And then behind the scene, this is what I was talking about when you swipe a credit card transactions, they don't come through the your mobile app or web app, but they're coming, you know, updating the mainframe from from the back inside of it. So whether it's your ACH transactions or whether it's your credit card transactions, they're all coming. So these are all think of them as, you know, credit operations, either being new records are being created new records are being read, or they're being updated or deleted that that's pretty much the entire gamut.  Nothing fancy, quite simple, and this is how it worked. And, and there are kind of few problems or few few things that are kind of concerned one is the, as I said, the mainframe is a single monolith. So many teams can't work on it. That's one.  So all the time I'm not going to even go into a lot of these details, but but you guys know that every company probably wants to get away from the mainframe. So, and the reasons why is kind of very simple. So how are we kind of doing it and what's the scope of this particular conversation. So, of course, all the new things that we're building, we're not building on the mainframe. So that that's the simple one, and I'm not going to go over that. And the second one is the migrating some of the reads out of the mainframe to open source and other software. So that's kind of the main focus of this talk. And the last one is that we are migrating some of the noncore functionality. When I mean noncore, like the mainframes today also have functionality like your customer management has features like KYC checks. And they're not actually processing any money transactions like ACH or credit card. So they are a little bit easier to migrate. So those are the ones that you're targeting probably third. And the last one is, of course, migrating the actual core, which is complex and actually expensive. So these are the ways that we are doing it. And this is how the sort of the cost and the complexity happens. And for this particular talk, I'm going to focus only on how we migrated the reads away from the mainframe to open source. And then, of course, this is the starting point, the previous architecture. And then, of course, nothing changes. The read patterns, the write pattern remains the same. All the writes are actually happening to the mainframe. The writes like address change from your mobile app, the customer is doing it. Or the backend transactions that are coming from your ACH and your credit card processing are all coming from. Exactly the same. So what we did is we first put a module on the mainframe. We modified the mainframe code to publish an event for every credit operation, mostly for the updates and then creates. And then that module basically publishes it to an event bus. That event bus is a combination of there is an API that abstracts the Kafka. There is Kafka and then there is Spark that reads it from the streaming infrastructure and then puts it in Cassandra. And then from Cassandra, we have the domain layer, which is spring boot based and where the business logic now resides. And then, of course, when the read request happens, the read request is actually going to the domain layer now to the right-hand side. And then it hits the Cassandra. So this is a, you know, from an architectural pattern, if you look at it, this is a standard CQRS pattern where the writes are happening onto the sort of the left-hand side and the reads are happening onto the right-hand side. And then, and then, yeah, probably, yeah, so okay, on the domain layer here. So one other benefit that we get out of moving the data away from the mainframes is that the domain layer could be a lot more sophisticated. We enhance the data when I mean enhanced. For example, in the traditional, you would see the transactions to be like very simple, but these days most of the transactions are actually enhanced. You will see the address of where you shopped. You would see the category of where you shopped. So all of that could be kind of enhanced in the domain layer. And then we also built a voice capability that you can actually search the transactions. And then a lot of voice capability was built, enhancement was built, search capability was built. So all of those capabilities which didn't exist in the previous system, we were able to kind of do that. That's the benefit other than reducing the cost. Cost was driver, but not the most important one. So building the new capability, as I said, was the driver. So I'll take some questions in the later so that we can. So what are the main learnings or the main things that we were supposed to take care of? There were many of them. I'm not going to go over all of them, but at least some of them I'm going to sort of dig deeper. And then the first one is of course, mainframes are known for their reliability, data consistency, and then security. So reliability was probably the highest. And if you look at the mainframe reliability, typically they are in the range of almost three nines and getting close to four nines. There are some of the maintenance windows, but they're actually pretty high bar to meet. So when we went through the open source, most of the technologies that we picked are distributed and redundant. Like if we take Kafka, that's a great example of it's redundant. It's distributed, Cassandra is distributed and redundant. And then if you look at even Spark, you can run it in Kubernetes. That's how we ran it in Kubernetes so that you could get the distributed and redundant. Same thing with the spring boot, the domain layer, we ran it in Kubernetes so that you would get it distributed, scaled, and highly available across data centers. Both all of them were built highly available across all data centers. So that way we could actually hit the first bar, which is the reliability should be higher than the mainframes. And then the next one, if you look at the scale, mainframes are good at scale, but they don't scale up when the volumes go up. And of course, as I kind of explained, the main reason why we are migrating is the curve of the digital transactions are actually going up significantly much more. So if you look at even the events that happen in the financial industry, like stimulus checks during COVID, traffic actually went up like nine times. So we need to have capacity that we can ramp up very quickly. Which was harder, and that's one of the problems that we have to kind of solve for. And then the next one, as I said, the pipe that we built between the mainframe and Cassandra, there are many different business lines that are actually using it. And as I said, as consumer retail wealth, many different business lines are kind of using that pipe. So we didn't want it to be built as a custom pipeline for every business line or every particular event. We wanted to build it once and building it once actually increases reliability significantly because every team doesn't have to test that pipe. And I can go over some of the features that we built with the pipe in the presentation. So there's a lot of core capability that we built on the pipe. So all the data pipelines of every business line are basically configuration driven. Not there's no code that these teams are developing actually. And then the next one, which is probably the most significant one, which I'll kind of dig deeper is the data transfer. Which is if somebody is actually doing a transaction on the mainframe, the time it takes to go to the next screen is typically around one second. So we had to kind of match that. Then the data transfer between the mainframe to Cassandra has to happen in less than a second. So that's sort of the reliability. The data transfer is probably sort of the biggest problem that we had. And then as I said, there is other than the scalability, there are issues on the peak traffic. Because once the Kafka topics are there, you can get a certain burst of traffic. Because the batch processing happens on the mainframe, middle of the night. And then suddenly there's a big spike of transactions that you will see for the settled payments. And then for the most of the day, there's no traffic. So it's very bursty, there are peaks, but burst of traffic is another thing that we solved for. And the next, most of the third order ones are the data related ones that I'm going to sort of dig deeper. The detection of missing events. In a file based transfer, it's actually quite easy to figure out end of the file. Because you have 1000 records, you put 1000 records, clean slate, everybody's happy. There is auditability, there is tracking. But in a stream, you don't know when you began, when you end, you can do accounting on it. So we had to solve for that problem. So the ones that are in italics is the one that I'm going to dig deeper, so I'm going to go. And then the errors actually. So there are kind of few kinds of errors. One is the, is any bugs basically, and then any software testing that you do, you cannot kind of test for the millions of records that are there. There is no way it's possible. So we had to figure out a way that even though we can deploy the code and we tested to best of our abilities, can we test this data in real time as we can add more data. To it. And then we also built a failsafe switch that that means that we are running with the Cassandra version and the mainframe version of it, just in case that that something goes wrong. Can we actually switch because that was key may not be for the entire life, but at least for the six months, three months. While we do the deployments that we learn and then fix any bugs. In the worst case, you have a failsafe switch. That was probably the good reason that we got the approvals because there was always a failsafe switch. The next is because we had to react to any real time events, like if the SLA goes significantly higher in the pipe, greater than one second, then we had to kind of make a decisions that is impacting customers and a lot of things like that. So we had to build a like a robust alerting and monitoring system in the entire pipe and even Cassandra so that to make sure that. So these are some of the sort of, you know, lot more problems, but these are ones that could make to the slide. So these are the three things that I'm going to sort of in the interest of the time that we're going to go deeper into. So again, just to kind of summarize, you had the main frame main frames published to MQ series. And there was a listener on the MQ series that would pick it up from the MQ series published to the springboard Kafka spark infrastructure. And then that puts it into the no sequel with an atomic right. And when we mean atomic right, if there were multiple tables, we would write it atomically so that there is no inconsistency in the data. That's the one. But here when we started out with, you know, the latency was almost around five seconds. And as I said, one second latency was the whole project would be kind of relevant if you didn't do one second. So we had to really, really work hard to actually get there. And then we did a lot of performance tuning like partitions, right throughput, memory settings, et cetera, and all through the pipe. And some of the other key things were going to be we had to learn and move away. One is from the micro batching in the streaming to to continuous streaming. And the next one is checkpointing, which is probably sort of a learning that we had that spark would do checkpointing to a file server. And that was actually very expensive. So we checkpointed to to actually the no sequel itself. And when we write the data, we actually did the checkpointing to that same considerably actually. So our latency is less than a second and mostly at 500 milliseconds mostly. But this was this probably took the most effort effort. And the next one is the is the detection of missing events. So again, mainframe, MQ series, Kafka, and then the Cassandra on the side. So what we did is whenever the data gets written to the MQ series, there is an entry in the journal. Journal, think of journal as just a table like with the message ID unique. So the inspiration but for this came from like the package tracking. Like if you look at FedEx tracking or UPS tracking, they track every package packet. So we wanted to track every event, the analogous of that is simple. So every every event that got published got a unique ID. We stored that unique IDs in the journal table. And then when when the spark job inserted those rows of data into Cassandra, again, it wrote the journal entries. What are the event IDs or tracking IDs? And every hour we would go back and then make sure that these packets are actually compared. And then, you know, as I said, multiple business lines, there's a lot of data there, but but still not too bad. We can actually do the comparison. So that and the next one is kind of important because we are a financial services company. The order of the data is very, very important. So if you miss the data, we wouldn't go back to the queue because the latest data could actually change. So to preserve that order, we go back to the mainframe and then get the data and then insert it. Okay, we'll get back to I'll finish that there's only one more slide and then we'll get back. So the the the next one is kind of a little bit of an extension on that one. This one was for think of the sort of the packets or the FedEx packets being delivered. Now, did I deliver all the contents? Because that was also the problem because if there is any bug in the code and instead of updating one column, we updated other column. And of course, there are so many edge cases that that you cannot ever even test in kind of in a production environment or in a in a IT, UAD testing environment. So what we did is we we export out the data from the mainframe. You know, we already have existing processes which would, you know, take the delta data and then send it across to different systems.  And then get the the data out from them and then compare actually row by row basically on all the data. And then of course this has its own problems. It has the big problem with this one was that there are a lot of false positives that you will get because time has changed because the nightly batches and the new data would have actually hit the Cassandra after the midnight. So we have to make sure we write the logic to suppress all the false positives and then get the true positives out of this. So that took us some time to build the true positives. But but we were able to fix kind of a lot of records. And then there are some examples. There were bugs on the sort of the mainframe on what events would not get published. So you would have some missing records and there are kind of orphan records and kind of mismatches. And this one took care of that that the contents of the packet are actually logically delivered to the Cassandra side. And some of the reasons for all this data checks is we're kind of a bank. And then if you kind of see a post missing and social media or anything like that, no big deal. Nobody is worried. But if you miss a transaction, even if it's a dollar, you're going to get caught.  We rolled it out to select group of employees in a larger employee base and then to small percentage of customers. So that itself, the whole thing was all to make sure that the data and the and the reliability, all of them are there. So that's pretty much the end of the talk. And then as kind of three highlights from this one is that the reason why we kind of rearchitected this is not because of cost savings was one of them. But but to do more services for the customers and to be ready for the customer when when the scale up happens. And the next one is that the open source was actually useful and critical. And then we had to build the additional data checks and parity. And that's what so, yeah, open for questions. Let me see how many minutes you've got left. Okay, looks like around 10 minutes for question. Sure. And I'll come back to your question. Yeah, go ahead. They say, oh, how are we going to get support if something breaks? So, yes. I mean, support was one and then we also had, you know, the vulnerability fixes and then making sure that we are running the latest and then we have to fix the vulnerabilities and so, yes. There was a larger drive for more open source within the bank. And we actually have an open source office to now. And and that said, if I kind of go back some pieces of it was kind of open source supported by vendors and some wasn't. Yeah, like for example, springboard. Thank you. Just a quick questions like how long was the migration effort? So the migration effort if you look at all the business lines, they're still kind of moving the workloads. So some workloads and kind of moved and then we are in kind of production. So if we go back to sort of the core engineering effort that needs to be built was that the streaming pipe that we built was one of the core engineering efforts. The Cassandra was already in production for a different workload already because I said all the new domains were already moving to a different platform. So they were already on Cassandra. So this pipe was probably took us like, you know, six, seven months to kind of build first to so that every pipeline that was being built don't have to kind of do all of the data checks, all of them, you know, the journaling and all of that came free for them. So and then the then each pipeline, you know, if you kind of add that plus the release mechanism, which is about releasing it to the employee and, you know, going through that cycle took around 18 months. So that means that when you kind of swipe the card kind of a transactions, all that happens on the mainframe. I mean, that was not the use case, but but potentially we could actually do that. Yeah. Yeah, I don't think the I mean, I don't want to, you know, speak on audit, but but I don't think there's any restriction on the system. It's just the checks and balances that that probably went into the mainframe and then somebody needs to replicate all of that for that function on the open source side of the Cassandra side. Yeah. I mean, all the reads and writes reads are happening and the customers are looking at the data in a way customers look at the data every day. Everything, you know, like from an audit ability, it's there are two different functions. The customer actually checks it in some sense, but but auditor the, you know, the mainframe is still probably the source of it. Given, given everything that you had to do to make a new service layer work to provide a functionality that to your user, they don't notice any difference. Why would you build a new domain to serve this as opposed to enhancing your existing domain with a materialized view of the transaction? So let me what do you mean by materialized? So so so in the left hand side existing transaction, I make a change and then the change is reflected back to the user in a presentation layer. Why would I build a new domain and all of this to store the secondary view rather than saying, hey, domain that deals with. I don't want to say customer identity information, but like the customer as an entity, why wouldn't I say, all right, you're the domain that owns the customer. You build a materialized view and eventual consistency will say, hey, at the end of the day when the mainframe runs its batch, which the bank considers is the actual final state of truth. Why wouldn't I just say, oh, you maintain transient state and serve it back to the user. So if it's I make it to address change and it's not committed to the mainframe, I'll just serve it back from the domain directly. Why build all the rest of this stuff? So the domain was actually adding as for for enhanced transactions, for example, enriched transactions as we called it. You had additional data, which, you know, we didn't want to store in the mainframe to begin with. So the credit card transaction, they see each transactions, all of them that are there in Cassandra are more than what's there in the mainframe. We added, appended more data to it. And there are kind of search indexes because, you know, for example, in the mainframe, you cannot search, give me all transactions that are on McDonald's. So that's there in the new domain. So the CDC process that moves it into Cassandra is not the sum of the data set. There's stuff that's fed directly into Cassandra that the mainframe doesn't know about. Yes. Okay. That makes more sense. Good. Is it fair then that that Cassandra layer is really to optimize the consumer's experience like the read. So it's almost like a BFF or what I would call a BFF pattern in a way that just serves the front end visibility of the data. Yeah. Yes. In this use case, definitely in this use case, but if you go back to the use case that new domains are built outside of the mainframe for that Cassandra was a read write. Yeah. Yeah. And for the second one is the whole talk about. Yeah. You know, it's funny. I do most of my work in healthcare and it's always interesting to me how many overlaps there are between some of our different industries. I've been working on a project to separate out our on-prem SQL database to have our cloud no SQL database going through the change data capture being able to enrich and have a completely separate database just for read.  I'm curious though, in the CDC process, using the address change as an example, knowing that an address change is good, but what you might need to publish to Kafka would include additional information. Maybe like the customer's name or their favorite color, their favorite TV show, for instance. How did you handle that at a very atomic level where the change is happening? I'm guessing on an address table, but it might actually need more information to be able to hydrate the customer record. So there was a, I was talking to Matt before this. We did like phase one of this before and we were successful partially. We took one or two use cases and we used actually CDC. And then when we scaled it up to like 40 different events, then we actually had a lot of problems because if you do CDC, CDC is you're exactly right. It's a granular level. And then all the logic used to sit in the pipe to kind of put it together. And then then we had a lot of things where you can't put the Humpty Dumpty together actually. So then we kind of abandoned that. So in this model, that's why we had to actually modify the mainframe to publish a complete event. So there is no translation or pretty much we avoided all translations and putting the package together in the pipe. The pipe is just dump pipe.  But we don't open the package and then see the contents and then put it together. Good. That's what I had to do too. And if you could go forward to slides, one of the things that it's, it's not that one is the one where you had like the pretty pictures and stuff and like Kafka. So one of the things that some of our teams do before they go live is they will actually hook their pre-production environment up to the production Kafka.  Compare what's in the NoSQL database for those same records in prod against what they pre-processed in pre-prod. And if they match, then they know that there's no regression. It only really works if you have some sort of near real-time asynchronous right of your mainframe so that you can do real apples to apples. Just curious if that's, you know, a strategy that you all have taken or if I've given everyone in the room a really good idea that hopefully you can do because your mom's data and health care matters a lot. Just like my mom, my mom is actually a U.S. bank customer. So thank you very much for making sure that her dollar transactions got through. Thanks for being a customer. I'll let her know. So we didn't try that, but our kind of deployment strategy was rolled out to the employees first and then get to the customers. Good job. No, actually good question. So no, the data sync was happening for all of them. So the almost like the, yeah, the serving of the, because what we could do with all this, we could actually check this. But the checking of the journaling, the data reconciliation, any alerting monitoring, all of that could be tested. But the impacted customers only with the employees because we made that switch in the UI layer. Yeah, the UI layer. That's where we kind of made that, you know, just select a random list of employees and then switch it between these two. Sure. I'll deliver it just a second. I'm curious, like with, you know, the storyline you told at the beginning, a lot of the work you did was to build confidence in this migration. So now that you've got that under your belt, like what's the next big tent pole thing on the sort of cake layer cake that you're showing? What's the big next problem? Actually, to make all of this kind of relevant. So we are probably in the third stage now. We're trying to think of moving all the noncore. And actually, you know, started thinking about the migrating of the core right now. Does migrating that noncore functionality, do you find like you're having to convince more of like the business folks that sort of rely on that? Like, now I have a whole different confidence building job. So like, like if you look at the noncore, they're exactly like the new domains. It's just that, you know, they were grandfathered into the mainframe. So the first thing that actually built the confidence and open source was actually the new domains. That was, of course, from a reliability perspective, we had that running for almost three years even before we kind of attempted this. And as I said, we attempted smaller scale sinks for one event and two events and then that was done on CDC. So there was some confidence that yes, we could do it, but at scale that we failed and we built this. So MIPS is a driver. So that's something that we actually kind of talk about it. The reason why it's not tangible at a bottom line is this is what happens. While we are doing this, you're getting new customers, just the new customers and their rights. And then we acquire new banks. And of course, we get the new rights from that banks traffic. So you will not see a tangible base lining of the MIPS cost per se because the new customers are being added all the time. So if you track it as a MIPS cost per transaction per customer, yes, you will see a decline. So it depends upon how you do the math. Yeah, good. So yeah, that's a good question too. Like if you look at all the four cakes, the mainframe cost would have gone up with the new domains. And then, yeah, the cost would have gone up basically. So you may see a flat line, but you're actually declining because considering all the things that would have gone on the mainframe are not there. That's a bit of hard sell for the finance, but that's true. We prevented an unrealized cost. Yeah, that is true. We say that, but it's a little bit harder. But yeah, good. But did you implement this in the cloud, private cloud? So yeah, we have a version of private cloud. All the Kubernetes things, like here the spring boot, like I was trying to get to that. Yeah, the spring boot ran on Kubernetes. The Kafka is on a VM. The Spark is on Kubernetes. The Cassandra is on VMs. And the domain layer is again in Kubernetes, internal cloud. Yeah. I mean, we wouldn't use probably no SQL for that. But yeah, we would design something different because core migration is a totally different track with totally different set of problems. Yeah. Okay. I think the next talk.


To what extent do you expect the new ways of working that came about in response to the pandemic will become a permanent part of the way your company operates? We certainly saw a lot of change during the pandemic. Of course, we moved very quickly to close our offices, which of course meant individuals had to start working from home and we've obviously helped them establish those environments in their home places across the region and across the globe. In Asia, given we have started to see some rescinding of COVID in some of the markets, we have been able to do phased reopenings of offices. But we do see, even as we're seeing people return to those offices, that we're going to have to do it in a more flexible fashion moving forward. So very much we're going to see different personas and different work profiles. Those individuals may spend two days in the office, three days in the office, four days in the office, but it is going to be far more flexible as we move forward. And how much has your own leadership style and that of your companies and the culture become more inclusive in the past 18 months or so? So diversity and inclusion belonging is like number one on our radar. And I think the pandemic has accelerated it because companies have needed to really understand and embrace all of the changes that have been happening around the world. So we've got a really big focus on creating an environment where all of our employees can say they are safe, they belong and they matter. So that's really been permeated throughout our organisation. So we've set up a whole bunch of capabilities to do that. We had a whole 24-hour period where we've done what we call the days of understanding, where we're trying to create more awareness of different cultures, different genders, different challenges of different subgroups inside the organisation. Really to try to educate everybody, create us a more inclusive environment, a more understanding environment. We've also created employee connection networks to foster those communications. Actually, just this morning I was on a call with our Inter-Asian Connect team looking at some of the activities we're doing in India to support women and how they're progressing through their education into the workforce in India. So these are all great things that I think we're really focused on as an organisation and I think we'll continue to accelerate as we see into the future. And does your company have the strength it needs, do you believe, to thrive over the next year? I think on the business operations side, Equinex is in a very fortuitous position. We're really at that hub of the digital connectivity piece and enabling companies to digitally transform. And as of course with the pandemic it has changed a lot of the dynamics to create digital as the primary focus. So we certainly feel in a good position to help our customers and support our teams in creating that kind of environment as we move forward. And then what trait learning or new habit that you may have developed during the pandemic that you're wanting to take forward into 2021 and beyond? Look, I think for all leadership teams and all leaders, 2020-21 during the pandemic has been a great learning experience. One that I'm personally going to take and I hope my leadership team takes is the empathy that we've learnt during this period. So we've had staff and markets and different teams in all different sets of challenges depending on where they are. And we've all learnt to understand that. We've all learnt to work with that better. And I think even when the pandemic rescins, I would love to see that globally enterprises continue with that empathetic approach and we continue to execute and create a more inclusive environment for all of our teams.


Everybody forced here ripping 2021 Don Russ optic football hobby hybrid five box number eight Random teams you can check their results at leighton sports cards.com at any time. Thanks for joining everybody and good luck Alrighty red Mojo ranid rookie Dolphins Javon Hollands Kyle Pitts inserts Smith-Marcette Shane Buchel Jibril Cox silver circles 41 of 125 Cowboys rated rookie Red Mojo Bucks Kyle Trask rated rookie Zach Wilson rated rookie Jets Penny Sewell hollow rated rookie Lions Jermart Jefferson Devontae Williams Broncos hollow climb over to layer chiefs Patch shirt and the second Eric Stokes Packers red Mojo rated rookie TN Fox red hot rookies Devontae Williams Broncos you draft in Shane Buchelager Number one overall fancy quarterbacks Probably third rated rookie red Mojo Ian book Saints One makes her you have a solid investment Something to root for right Rashawn Slater rated rookie Chargers red Mojo trace sermon blue rated rookie Marquez Stevenson Bills, that's 58 of 179 Jalen waddle base rated rookie Dolphins red Mojo Devontae Smith Eagles Calvin Joseph hollow rated rookies Steelers Pat primary Trumbull Kenneth Gainwell Kaderious Tony Red Mojo rated rookie Washington Jamin Davis and Trevor Lawrence gifted rookies Jags. There was some damage Oh, it's worse than I thought it was up here. You see it on the Ritchie granted down here as well I'll send them at the box and wrapper to the Falcon spot Lane's right red hot rookies Devontae Smith Eagles. I know Lane's got him on his big board Jefferson rated rookie red Mojo Packers Kylan Hill Big retro series Lane's all in sounds like it on changer show using the right the right terminology Devontae Smith Eagles rated rookie Red Mojo bears Justin Fields rated rookie nice fields. Yep Dean with the Bears congrats Dean That one mag for you, buddy Travis eats the end rated rookie or not rated rookie, but the rookies insert. Excuse me the rookies insert Rontael Moore to to at well red Mojo Rams rated rookie They've released in the past and pink velocity Ravens JK Dobbins 48 of 79 Let's just go Quinn minors rated rookie red Mojo Broncos my house Kylan Murray Cardinals Hollow rated rookie Eagles Devontae Smith Trying to think is rich gonna take it seriously enough to where he's like I've got fed got feed Rocco, but I set my lineup legendary logo Zach Wilson jets Is it gonna be like that or like is he just gonna like forget? I don't know He's too competitive, especially against like Lane Cortes red Mojo rated rookie Colts Sam Ellinger He wants to talk trash to everybody. Oh, yeah, hello, Brandon Euk, Niners How many playoff teams Lane? I have no idea the structure of this league I'm just gonna draft based on PPR. I'm guessing Holland red Mojo rated rookie Steelers Pat Fryer move Trey Lantz great iron Kings rookie insert trash got awkward cowboys Emmett Smith 44 of 299 red Mojo chiefs Nick Bolton rated rookie To Neil Hunter look hollow rated rookie Rams Jacob Harris Not she Harris rated rookie red Mojo terrace Marshall Jr. Panthers rated rookie at heart rookies bears just a field sounds like lanes procrastinating all these options Red Mojo bears rated rookie Khalil Herbert hollow Packers Devontae Adams One more box. You got a lot of these impersonals No She's in for a break maybe Got red Mojo rated rookie Rashon Slater Chargers. No, I'm not gonna call out Lane on on his job No way, man Sermon pink velocity CD lamb Cowboys 71 of 79 Not a waddle rated rookie for the Dolphins red Mojo Devontae Smith Eagles rated rookie in my house the Mar Jackson It's just too much work Lane It's too much work, man. Holler rated rookie Rams Jake funk There's Tony red Mojo Jamon Davis Washington and the rookie Zach Wilson Jets downtown Broncos John Elway Heck yeah, Denver.  Oh No, man. Who's got him? Trying to read Like Jimmy Neutron thing thing thing thing Anthony P. Congrats Anthony sweet downtown Dak legendary logos Cowboys Yeah, they had it last year Lane. It's just a lot tougher to hit the odds are a lot lower Gain well rated rookie red Mojo rated rookie Packers Kyle and Hill even though they seem to be more common this year You'll eat series Mac Jones Pats kind of like Kaboom in absolute just more common Haven't seen an auto yet Hollow Hayden Hurst Falcons hadn't seen an auto at all actually two downtowns and 15 boxes And I think every rookie quarterback Fields one back a lance We had a rated rookie red Mojo fields for the Bears and the downtown Broncos John Elway That'll do it. Thanks again, everybody. We've got some more people coming up


I did not drink you under the table by the way We were even Equal My god, but thank you so much because I would have never came here if he did not actually talk me into it or just and he wrote me this three-page email like two days later after Aspen and I'm just a sort of a hermit I like to I love traveling and I love meeting new people But I'm I'm extremely shy, so you're gonna have to be patient with me on this speech also dyslexic So I go all over the place, but I'm gonna try to rain it in Thank you again David, thank you Renee what an opportunity I just can't even believe I'm here I drove from Cape Cod the typical Cape Cod yesterday morning to get to the airport I got here at 7 a.m. And I'm still in shock, but You guys this whole room inspires me And the mad team bad-ass mad team is is wonderful All right, so I've gotten so many questions Asked to me about my sanity and my crazy As I was opening up Actually my first restaurant, which was 15 years ago I Was that crazy I needed to raise two million dollars Well, I was living in a housing project with my mother the last of seven. I Ode the IRS seventy-five thousand dollars. God knows why Who the hell would open a butcher shop when she doesn't even know how to butcher and put a retail a meat retail case in a wine bar in Boston Who would open what would be the most luxurious fine dining restaurant in the city of Boston doing the during the worst? economic downturn in decades Me I Think I'm all right I'll always say to myself is this luck luck luck because everyone's like oh, she's so lucky.  That balls But what I really think it comes down to a couple of things right here. It says three I love that word honesty honesty be honest with yourself Anything that you do don't lie You're cheating yourself Have clarity Your vision it's your vision. So when you want to have your vision played out Don't listen to anybody else Put that tunnel vision on because it's always worked for me.  You got guts and I'm you know a couple of glasses of wine in I'm like, yeah All right. I I got guts. All right. I'll give this another thought My whole mantra in my life has been I'm not scared like I will talk myself out of everything That in the shittiest times the stupidest things I've ever done. I'm not scared and I would go through it and It worked But from the time I was a kid I must have been hardwired to take any dare that would come my way. I would do anything I stole an MBTA bus when I was 13 Oh, my mother bought me a Mary Poppins umbrella and I really thought I could fly when I was seven So I climbed up a roof and jumped and that didn't work So Very little people know me here. So for the most of you, let me just I'm gonna just Share a little bit about myself just to let you know why I am who I am I was born and raised in a South Boston housing project Next door to whitey bulger, by the way, I don't know if anybody knows what he bulger. Yeah, it's interesting I Was the youngest of seven children my father died a month before I was born She worked three jobs to keep us off welfare to feed us I never graduated from high school was forced busing in Boston was awful was like a war zone Um And I really just couldn't learn it I just couldn't retain information in a classroom But I did have one oh, and I was a bookie I was a bookie in high school. I Was so good at it. I was so good at it. I Wouldn't even call the numbers in and I'm putting the cash in my pocket And then I'm going shopping and then I'm like this. This isn't gonna last Especially living next to whitey belger was not a good idea doing that And I'm telling you if a if a psychic told me 20-something years ago that I would have eight different restaurants in the city of Boston run a 24 million dollar company I Would say yeah, you can go shove that shit somewhere, right? But I think running numbers was really helpful in the entrepreneurial part But my home Mac teacher she became a mentor. There's no home Mac anymore. What happened to home Mac none But she you know I think it was the first time because my mother was so busy that she actually showed interest in my passion or The ease of my cooking we had a she had a great little Culinary program in high school. That was the only class I went to And then in my mid-20s I had lots of dead-end jobs and my best friend talked me out of one of them and I went on vacation with her I came back. I didn't have a job. I got a job at a telephone company for two days Thank you for calling John Hancock mutual life insurance.   I gotta leave so I moved to Martha's vineyard and she came with me She quit her job. I went to go and interview for a position on a dinner cruise I think I was 22. Yeah, I was 22 and The owner of the cruise the dinner cruise Was like well, we don't need a host and we don't need a server and I said no no Oh, I want to cook because you want to cook. I was like, yeah, I want to cook He says well, what kind of experience do you have? I Told this guy everything he wanted to hear I'm making co-hog chowder. You should see the doble soul like at I worked. I was a server at the same bit off club and I'm like dover soul Sweet breads under a bell I'm from the project. I haven't even read a cookbook yet, right? So Oh, I'm making a refined Zab I own sauce He's like wow, okay. I Must have convinced him because I got the job and Then two days before this cruise was gonna go out to Falmouth and around the chef quit And so the owner asked me well, do you think you can you can do this job?   I went to the library on Martha's Vineyard, which isn't really that big and not a really great selection of books Like not a lot of good porn food books at all It was just like I needed to know how to filet a tenderloin how to even cook a tenderloin I never did this before how to kill a lobster how to store food in a galley kitchen. I Did it I did it. Oh my god. It was so successful. I had so much fun My friends are all like, where is she I'm like, I'm cooking on a boat. They're like, no, you're not I'm like I am they're like Who do you think you are Julia child? Like no, but wouldn't mind being that So then eventually I left the vineyard. I came back to Boston and I You know, I worked around town. Oh, I worked for yeah some crazy chefs and then Throughout that there was a lot I learned from other restaurants Restaurants that don't take reservations. It's stressful. You never know how many Guests you're gonna be cooking for Etc. Etc. What I did I put my head down.  I've never had discipline Discipline is so key in perfecting anything you want And I learned as much as I could possibly very fast all by cookbooks Even the French cookbooks I had to get a dictionary and translate So that's how I got inspired and that's how I learned Five years ago Montaigne opened. This is a it's a relaying Chateau property in Boston.  What do I want to tell younger chefs?  Let me see your vision. What do you want? I don't have one yet. What designer did you use? Where'd you get your China? Etc. Etc. I don't mind sharing that knowledge at all because I think that is another key Very key thing to do once you have that knowledge. We need to share it.  I'll check in how you're doing and You know the ones with the vision usually make it I Love that word. No a lot of people say you can't do it that way. You can't do it that way But I can't do it any other way like I can't do it However, there if there's a rule and this is the way you do it. I can't follow that. I don't I Just don't I don't know why this is when we started cracking up When we were out and I kind of always think outside the box I love it when they when you hear these programs about children and your parents are saying color in the line color in the line Inside the line get outside that line do whatever you want. It's your creation Get out of the box Then you know, you know breaking through barriers. That's tough When I opened my first restaurant of 15 years ago, I had two partners. Well, I gave a Third and a third to two guys one's a general manager one was my business partner business manager and they Literally treated me like I was a kid from the projects. I knew nothing I'm a plumber basically get back in the kitchen, you know, nothing about numbers, etc. So I took that Then I got I self-taught myself. I write about it. I got it. I understood it I changed the whole P&L system to the way I could read it. I don't want a yearly P&L It's too late. I need monthlies by weeklies if it's not working well And then I got rid of them But they were you know, they were good for when I started because I literally opened a restaurant on You know, basically self-taught chef. So I did spend like two hard years Cooking in that kitchen and I couldn't worry about management.  Yeah, but you're gonna go through those things This is life you're gonna evolve and you better get some balls if you don't have them you have to get you have to trust your instincts It's it's you I'm I might have grown up poor and I might have grown up not educated But I knew what my restaurant should be like I knew I knew what I could cook I knew what I could handle I couldn't go over that that would be too overwhelming So this don't don't take on more than what you don't think you can handle Because life's gonna come down with a lot more.  That's a key one take inventory of your strengths and your weaknesses. I never did this I was so insecure. I was so afraid to ask for help That changed I think When I went and I it was on my fifth restaurant and now and I knew I couldn't do it all I was like suicidal And my weaknesses are management and I'm I'm I don't like a micro management. I'm a group person I love building teams. I love having my Like it's called Barbara Lynch Grupo, you know, I don't have barely have turnover Sometimes we gotta get them out because it's costly and we all know that but You know, I've had my same my wine director and I've been together for 25 years. We started when we were puppies So they don't really leave and that's a really important part because turnover in your kitchen or in your front The house is gonna cost you a shitload of money. There's a lot of training a lot of wasted time So if you're not strong at trying to build a culture and trying to you know, work with management rethink that one Having nothing to lose could be a good thing it worked for me. I had nothing I had an Azuzu trooper And I would like like drive it and pretend that I was like in a Range Rover was awesome Like you like it I'm the one I was living with my mother I owe the IRS money and here I am trying to raise money on Christmas Eve in Boston from my first restaurant But what did I have to lose right? Nothing. I got Two million. I got one phone call 500,000 Christmas Eve And he said all my other friends. Well, they're Jewish, but they're away for the holidays So maybe when they get back, they'll call you and I'm like, okay, great I knew nothing. I had no financials. I had no background. I had a vision a white tablecloth restaurant 70 seats. I Was very Low on the check average so that I knew I could climb up and I trusted and I listened to my investors You know, I didn't have the menus written too small. I listened to their advice I went to lunch maybe once a month and these investors who helped build Boston their old Brahmin Bostonians And I'm like, wow, how'd I get here? This is just crazy But I wanted to succeed because you know what somebody believed in me. They're giving me money. I was a bookie crackers But it made me want to work harder it made me want to figure it out It made me want them to be proud of who they invested in eight restaurants later They make a lot of money off me But I love it Money and ego are the two things you don't want To think that that's what's gonna run your restaurant. You don't passion passion passion You can only fake Shit for so long passion Okay, here we go. Sorry. I told you I get lost Ted talk was awful. I had to do a Ted talk. It was awful Know the value of patience timing and intuition intuition is key I've always trusted my intuition I think women have a stronger intuition so they have to really trust it Patience and timing I've gotten everything right one. Oh, I have one concept that failed I had a little store called plum produce is a cutest little thing 250 square feet right next to the my butcher shop and then I have a cooking demonstration bookstore called stir and I'm like, well, I should be my own neighbor. I really don't need anything but alright I'll put a produce shop there. I was like the prettiest produce shop ever. I think people were afraid to come in But I wanted to show the public that we can have great porcini mushrooms or truffles in the squash The window display My stuff was really mad because that command I'm just spritzing the lettuces and they're like chef I think you have more things to do than this. I'm like, no, but I really like it here It's really nice You know, I didn't do my homework though Where I am in the south end young urban they cook on the weekends So I really wasn't moving the produce and every night I would dehydrate and I was dehydrating everything like then I was making powders and salts Then the health department came in and he's like miss Lynch. You're just you're crazy. You can't dehydrate. I had ten Those the quip dehydrated is on the floor.  I'm like, I hope not but So There is I was making money and all the dehydrated stuff. It was access. It was food They no one had to peel or chop or throw away kids would come in they're eating bananas and pineapples raspberries blackberries So I listened to the health department I didn't want a commissary because I had just signed on to 15,000 square feet of space To build three other restaurants. So I had to put that one on the back burner But I knew I was on to something in the timing. Thank God the timing wasn't right Five six years ago. So I'm starting that company like in a month. It's it's dehydration. It's instant food It'll be in vending machines. It'll be in airports. It'll be in hospitals. It gives you a healthy choice, right? So now I'm focused on that But anyway, I don't want to bore you anymore. So canels of steel. Oh My god, this do cost here because I just sat in a seat for a minute I always always I wanted to be a lane to cost or Joel Rubish on I'm telling you that doesn't like that was my carrot I've finally come to the point. I'm so happy to be Barbara Lynch