So the topic of our presentation today is titled Backdormulatory, Another Security Tales From Gaming. So the overview of the talk will go through some introductions as to who we are so you can see the context. What has happened since 2011 and why that date is important to me is it's the last time I was here speaking at DEF CON about eye gaming security issues. Give a brief historical overview of some attacks and how the focus has switched from the physical to now more on the logical side and then probably what's most interesting and what Evan is going to spend a lot of time talking about is the Eddie Tipton case, how we discovered how he rigged the lottery, what Evan did to reverse engineer of the code, et cetera. If time allows, we'll talk a little bit about the recent Russian slot attacks that you know they were in Wired Magazine recently and the report is coming out with another story in a couple weeks and then we'll try to wrap everything up. So first one page, one slide about who my company is and say pay for us, Senate International, look us up, always looking to talk to people. Who I am, my name is Gus Fritchie, I'm a CTO of Senate International and about five years ago after my presentation on eye gaming, I transitioned a significant portion of our practice into the gaming sector because I was more interested in doing the government work that we had been focusing on and now I'm pretty proud of what we've been able to do and the client base we have going across from the lotteries to travel casinos, corporate casinos, daily fantasy sports, et cetera. Hey, so I'm Evan, I work for Gus. I work with Gus. So I live in DC, I have a Linux distro, it's a small one, I do reverse engineering and stuff and my free time I hike and climb and I live in a van too, so here's my van. So Evan limited me to only two memes, so this is one of them. So this is not going to be a super technical talk but then again, it's Sunday, the last day of DEF CON, I'm sure we saw, there's been a ton of great, highly technical talks. It's more of a story talk but Evan definitely does get into the details with the code which I think those of you who are interested in that subject should find pretty interesting. So what has happened since 2011? First I just buy a show of hands, how many people saw my last one on eye gaming security? I was just curious who attended, so a handful of people. So the good news for all of us online poker players is we got paid back our money from the sites that got shut down for the most part, so that's obviously a good thing. But what we have seen happen is not a lot of movement in eye gaming. The green illustrates the states where eye gaming is legal, so really we only have three, Nevada, Delaware and New Jersey and the yellow is pending legislation. Why I find this slide interesting is even though it's a small number, it shows that the attack footprint is expanding and I think if I don't have a slide but if I show the slide of where it shows where land based casinos are legal, you would also see that expanding. We have tons of states even five years ago that have casinos that didn't use to have casinos. So we see the potential attack footprint expanding. Is there a question? Sorry, eye gaming, online poker, online slots, gambling, basically online. Caesar's and other companies they like to call it gaming because gambling has a bad sound to it. So this sector has not been immune to security incidents, unlike just like any other sector that you read about in the news. I'm not going to talk about any of these in any detail. I just picked a random sampling of some of the breaches that have been disclosed and of course as you know there's plenty of other breaches that occur that never get disclosed that we don't hear about. So this is obviously an area that needs to be secured and I think when it comes to the public's trust and the integrity of gaming is extremely important because without that you don't really have a solid business model. With the Las Vegas Sands I do find this interesting. This happened in December of 2014. If you didn't read about it, I'm not going to spend a lot of time but this occurred to arguably the world's most profitable gaming organization. They make millions and millions of dollars and at this point in time where the breach occurred they had a very small IT security staff. I don't have not done any work with Las Vegas Sands so I can't speak to their size of their staff now but I understand it's increased a lot and they put a lot more money into securing those environments. But the way this breach occurred was just to show you how easy it was. It was just a development server that was stood up on the perimeter that they didn't know about or they knew about but it was used as a foothold and they pivoted inside and they just ran havoc destroying data and it could have been a lot worse if they were more sophisticated and did not look to be destructive. So history. Let's get into some of the early attacks against slot machines. Not going to spend too much time so I want Evan to get to his piece which I think is more interesting but obviously in the earlier days we had physical attacks with fake coins yoyoing with a coin on a string pulling it back out attacks against bank note validators when those started getting installed. And then we have Tommy Carmichael here who came up with a couple of inventions or devices I should say that allowed him to commit fraud with a monkey paw which he was able to use to jam up there and release the coin hopper and then of course with the light wand which manipulated the sensors and allowed the money to come out of the machine without being won. So we saw these early attacks were more physical in nature and another physical attack that we saw that happened in 1980 and my partner in the lottery sector Herb Delhante wrote about this in his book and it was also made into a movie. I don't think a very good one but it was there. And they waited the balls so they waited the sixes and fours and this obviously was able to happen because you had collusion between multiple parties. Obviously people realized pretty quickly that it was a fraud and no one got paid out even the illegal books knew that prior to the lottery even admitting it. So we had these physical attacks against slot machines and other forms of gaming. And then we see a transition to attacks on a logical side and I think when you start seeing these stories you'll start seeing how they tie together to what Eddie Tipton did in the muscle hot lotto RNG fraud case. So some of you may be aware of Ron Harris but he worked for the Nevada Gaming Control Board and his responsibility was basically to perform audits of the gaming software and platforms and one of his responsibilities was to audit the eProms in the slot machines and make sure that they were correct. But what he did was instead of auditing them he reprogrammed them so that when a certain sequence was pushed on the slot machines or a certain number of coins were entered it would pay out a winnings and I spoke to Rex Carlson who was the director of Nevada Gaming Control Board and he said that they really don't know how much money he actually made from this because he didn't get caught initially because of this. He got caught because he turned his attention to Kino and this is where it sort of parallels what Mr. Tipton did in the hot lotto case is since he had access of course to the source code he found a flaw in the pseudo random number generator and then he wrote a program that allowed him to predict what the winning Kino number was. So he went to ballies in Atlantic City and with an accomplice they probably got away with it but they just had such poor planning as far as what they were going to do after they won. So this of course raised suspicion and the authorities performed investigation and they went back and looked at the other work that he was performing. So what we see is we see individuals with trust who were trusted to perform these reviews and had access to these devices and this will parallel what happened with Eddie. And one last example from a technical perspective and I talked about this in my last presentation but this was from the absolute poker ultimate bet super user scandal where the owner of the site I guess I should say alleged because it was never actually 100% proven but he had a convinced one of the programmers that he needed a backdoor into the program so he could see players whole cards because he thought his whole idea was that there was cheating and only he could figure out if they were really cheating but he proceeded to use his backdoor in the poker software to illegally win about 20 million dollars from players and this graph shows the individual one of the accounts and you can see that he's way off there as far as the norm from a winning percentage. So once again we have examples from Ron Harris with the Kino being able to access the source code and find weaknesses. Here we have an example placing another backdoor in a system allowing the, allowing cheat and fraud to occur in the poker. So now I'm going to turn it over to Evan where he's going to get into the current events and what I think is the most interesting topic of this talk. Alright cool so this is my first talk by the way. So in case you guys aren't familiar with like how the lottery works, there's individual state lotteries and each lottery manages itself but there's something called muscle which is like an organization that oversees all the state lotteries, well most of them. So back in like 2005, what was it? 2004, it doesn't matter. So like a while back this guy named Eddie Tipton got a job at muscle to write an RNG. So some of the state lotteries use like computer RNGs to draw the numbers. Others use balls out of hats and stuff like that. Well not actually hats but yeah. So he got a job to write an RNG and pretty much immediately he rigged it because obviously he wanted to make money and they weren't paying it very much. So here's a couple of other faces involved in this. Tommy Tipton is like his brother, he was involved in it, he helped cash out tickets. Robert Rhodes also helped cash out tickets as like a friend of his. And then Rob Sand is the attorney general in charge of like the case. Alright so in 2010 Eddie went into a gas station in Iowa and purchased a ticket for a lottery game. He waited a whole year to claim the ticket which was pretty suspicious. And then also he used a mysterious company incorporated in like Belize and he went through an attorney to try to cash out the ticket. So that was obviously extremely suspicious and lottery security caught wind of that and decided to do an investigation into it. They also refused to give them the money until like they completed the investigation and because they refused to give their identities which is like possibly illegal. So they withdrew their claim to the prize. I'm assuming they did not want to get caught. Let's see here. And apparently they also received a tip from somebody that Tipton was the person in the video. I actually think I know who that was. But yeah. So they started a full investigation which involved the FBI and stuff too. And they determined that it was him that purchased the ticket. And because he worked for the lottery he was banned from playing the lottery. They kind of used circumstantial evidence to convict him but they convicted him before they knew how he actually break the lottery. Or that he did for sure. He was sentenced to ten years of prison. He was out on bond or bail pending a pill for a good like year. He actually just played guilty pretty recently. This should be 2017. But yeah. And here's a quick timeline of events. So he was hired in 2003 at Muscle. And the first known case of fraud was in 2005. That could have been earlier cases though. Because he was breaking the dozen four, right? And what's interesting about the timeline here if you look at where it says that the Colorado lottery fraud was constant, Kansas lottery fraud. Do you see a pattern there? They're all on the same date. And that's what Evan will talk about when it gets into how his code actually worked and how it allowed him to predict what the winning numbers were going to be. Yeah. And also this is just known cases of fraud. There were probably more. Especially since we found a third date in the code itself which there were no known cases of fraud for. So let's see here. No, I just wanted to also, a couple of gaps just in the story. As Evan mentioned he was really, he was convicted initially of a charge on really circumstantial evidence. I mean they had the video footage from the store which when you think about it he really made a big mistake because one in the other cases of fraud he actually never bought the tickets himself. He had his, accomplishes his brother or Robert Rhodes buying the ticket but in this case he actually went and bought the ticket himself and perhaps that's because the prize value was so large that he just didn't feel comfortable having someone else buy the winning tickets. And he actually went to a convenience store that had audio and also video surveillance. So there was other stores that he could have gone to that video or the audio and it's really the audio was what convicted him because his voice is very distinctive and they were able to prove it that way. Of course he felt and his lawyers felt it was circumstantial evidence and they did appeal it and during that appeal process that's when we were able to get a hold of the Wisconsin Lottery R&G which Evan will talk about and what he did but you know how do you go about rigging a lottery and this is my second and last meme so I was only allowed to but obviously you become a lottery developer, write code and have your friends buy the winning numbers. It's that easy and luckily for the first time we have video surveillance of Eddie Tipton actually performing the programming. Now Evan will tell you how he really did it. Yeah so I do kind of went over this part but so basically what he did he worked for muscle so not all the lotteries, all the state lotteries used the R&G some of them used third party R&Gs and others used like machines that just like throw balls up in the air and they come back down and stuff like that which is what lotteries should do by the way should not use computers to generate numbers but it's a business for us. So while he worked there so they have like a supervised build process so he got co-past that process which made the numbers predictable in three days with other conditions as well and the binaries and source code were certified by a major testing lab but the way he did it wouldn't have mattered unless he really went in depth and checked the binary against the source code and stuff like that. He also had access to the computer images too which he could have used like a root kit or something simpler to change the numbers as well so even if he wasn't the one writing the code he could have break the lottery as well. In 2016 we were contracted to perform imaging of one of the R&Gs and we were actually given permission to review the images at some point and I was asked to try to figure out how he did it because at this point nobody had any idea how he actually did it. He didn't seem smart enough to use a root kit so when he was convicted they assumed that he used a root kit to change the numbers because they didn't find anything in the code. Apparently they didn't find the binary to be malicious in any way and I find it questionable how they analyzed that but yeah. So I decided that since he was the one writing the R&G itself I would just look at the binaries and compare them to the code so I didn't actually started like just going through the binaries and just reverse engineering them so there were a few binaries like the main executables and libraries and stuff like that so the most interesting binary was the one that actually contained the R&G itself so it was actually the first binary that it looks at so I started looking at and this is pretty much what I just said so it was coming through all the functions in the binary and one of them caught my eye pretty much immediately and at this point I knew that all the winnings all the known cases of fraud run certain dates so this one I saw it had a bunch of date functions right at the top so I was like that's probably it like I saw another reason for them to use dates so I started obviously reverse engineering it and I saw pretty much immediately that it was referencing those specific dates the two dates that we knew in a third date as well so I reverse engineering it and I figured out exactly what it was doing and how it was seeding the R&G and everything like that so also it was at the end of the binary which is pretty suspicious because it looks like somebody tacked on that function at the very end of a file and imagine we got the source code for the R&G and we saw that there was no that function wasn't in there there were 25 functions in the source code and 26 in the binary as you can see but yep so yep that's about it that's how he did it he just I can go into details on that real quick so basically each time a number was drawn this function was called and it would reseal the R&G with predictable values on certain days of the year it also had to be like a Wednesday I think it's supposed to be Sunday Wednesday or Sunday and here's just like the code of his function so he actually seeded the R&G up a bunch of values from the computer I'm not really sure why he did that he could have used anything really he's like a computer name and he added it up threw that into the seed along with the values from the game and everything he kind of made it more complicated than necessary which made him have to buy more tickets than necessary in some cases he bought multiple tickets because he wasn't sure what the values would be yeah and so here's why certification did not work so it was certified by one of the major testing labs and their testing process was to run the output of the R&G through a bunch of statistical tests which is great to ensure the results are unbiased but it doesn't really catch anybody reading it they performed an audit of the source code but the source code he compiled was obviously different and he was able to slip that super bus bill process pretty easily so that's questionable and here's how he could have done it better too he worked it on only three days a year which made it pretty easy to identify the winnings if you work it on every single day of the year it would be extremely difficult to identify the winnings because what they did like what the investigators did was they knew that he was using like certain dates so they just went through all those dates and looked at all the winnings and they looked at the ones that were most suspicious and they followed those leads so obviously if you work the lottery definitely do every single day of the year and also he could have made the method of working it more discreet he could have used rootkits and changed the numbers in memory and that would have been much more discreet than having it in the binary because what we do now is we check updates to the R&Gs and spread them to the source code like Bindiff or if it's in Java I've got a custom tool for that so we can catch updates like if a vendor tries to like pass a like a backdoor into like the update we can catch that pretty easily so how can this be prevented source code should undergo end-up third-party reviews I think supervised builds are important too as an additional layer but for updates if you have a binary and you're not concerned about the system image it's pretty easy to check those like Bindiff and make sure they're not malicious after reviewing the source code because what we do is we compile the source code and we compare it to the actual binary we get but there's another issue with the system image too where these guys are building the system image and that's not supervised at all so they could obviously slip something in and it's pretty difficult to supervise that entire process and you can't check the image you can't be certain of finding anything in there yep, there you are that's about it thanks Evan, I think you covered that very well when you think about it he actually had a pretty good idea because once he got this qvrng.dll file certified by the testing labs and once again, the testing labs supposedly and I know they do now because we do review the source code line by line and they were supposed to watch them compile the codes because they end up hashing those binaries so in some way we're not completely sure he was able to get the testing lab to certify that this code was valid and then he had it on all these boxes and that never changed he would make modifications perhaps to the executable but he never ever had to modify the dll so actually what was interesting when we reviewed some of the more recent muscle rngs and you have to remember, he started this process in 2003 this fraud case didn't happen until 2011, he wasn't convicted until 2015 that's a lot of time going on to make modifications and as Evan mentioned there are other commercial rngs in the lottery sector but some of the state lotteries their whole mission is to give back to education or whatever whatever is written into the law so they're very frugal with their money and they didn't want to pay 200,000 whatever for rng so they would pay muscle at a much lower cost so you had all these rngs being used in other states and when we reviewed some of the other ones in other states at least the ones we could get the images to it was interesting because the executable was not calling the qvrng.dll correct Evan? yeah I'm a newer one so somewhere along the line I think he got scared or nervous perhaps after his jackpot win and switched to code so people wouldn't even look at that qvrng.dll and maybe his plan was once things die down he could just call it again in the future so I think he left the binary on there when he updated I think he just left it on there to be honest there's been some speculation that he left it on there to switch back in the future but I think he just left it on there like when he updated he never deleted it yeah there's different theories for why he did that but in the end I think it's a case of where we see breakdowns in a couple different areas and probably the most obvious and basic is that separation of duties here you had somebody who was the director of security of muscle who was a lead programmer he had physical access to the boxes he had everything and to give muscle credit now they have completely revamped their entire process and their operational management and technical controls are a hundred times better than what was committed so we can probably definitely go into more detail and talk later about the tipped-in case outside if you guys have specific questions for Evan like get into the weeds of it let's just check on time here we got plenty of time I felt we rushed because I was nervous about the video issues in the beginning so the Russian slot machine hacking and this sort of ties everything back to the present John Harris back in the mid-1990s we have the eye gaming the ultimate bet absolute poker we have this tipped-in case and here we have this Russian slot machine hacking which is a very good article which I'm sure most of you read because it was widely tweeted at least in my Twitter feed and I give a link to it there and also at the protection conference he was also quoted in that article and World Game Protection Conference is a conference here in Vegas in the December timeframe that pretty much focuses on physical security but he's broadened it and he has some inside information as far as how this fraud occurred as well but basically just to rehash the story and tie it back in in 2009 when Putin made gambling illegal in Russia there was this flood of slot machines on the black market so some of them or a lot of them were sold to other casinos but some found their ways into the hands of the Russian mafia and they ended up reverse engine I'll just get to that piece of what they did but in 2011 some casinos in Europe were noticing some suspicious payouts and then in 2014 in Missouri they noticed really unusual and high payouts on some particular slot machines so they started investigating this and this is where you have compensated controls and other pieces actually worked to end up detecting the fraud they had to go back to the physical surveillance cameras and tie this all together but these individuals they came back and they were later arrested in Missouri and were arrested in Singapore last year but what did they do? they reverse engineered similar to what Evan did with the code but they reverse engineered the software the binary on the slot machines and they found a weakness in the PRNG so then what the guys would do is they would take a video of a certain number of spins and that data was transmitted back to their comrades back in Russia where they were using very high powerful computers to process this data send it back to them they had an app on their phone which would then vibrate a couple seconds before they were supposed to hit the button and it wasn't always successful but it did result in a much higher payout so this is an example where there was a weakness in the computer programming in the RNG this wasn't built on purpose it was just a mistake that the developers made and this vulnerability impacted a particular vendor Aristocrat I believe on their older versions but the claim now is and I think not to steal the reporter's thunder but they claim to have working code for more modern slot machines and they're threatening to release this code to the general public they were trying to blackmail Aristocrat I believe but so far that hasn't worked but it will be interesting to read the story when that comes out so we talked about some issues here in a lot of different sectors what does an operator do to better protect themselves I think it's a lot no different than any other industry from healthcare to banking we get caught in this trap of compliance we gotta be compliant we gotta be compliant we waste a lot of money on paper doing that but we have to understand and I know I'm preaching to the choir that compliance is not equal security and when I first got into working with casinos and operators I went in there thinking wow these guys are going to be super secure you see all the movies Oceans 11 they got all the surveillance cameras and they are very good from a physical perspective for the most part but quite honestly they really are lacking in the technical security controls but that's really no different than any other sector now to improve themselves they're being proactive I know other firms are working with them so they really are taking it to heart to improve their security and protect their players data more money needs to be spent on information security and also the operators need to start asking the game manufacturers what how is your system secure what controls do you have in place and I'll give you an example we were working with a casino organization and we were doing a security assessment just a basic vulnerability assessment and realized that from the corporate network we could actually get to the slot machines and touch the interface card in the slot machines and when I brought that up to the director of IT's attention because I was curious I was like it didn't seem right he was like well let me fire off a note to I'm not going to say the game manufacturer's name and they came back oh no the only thing these need to talk to is a player tracking database it should be firewalled off so you have operators who are trusting their vendors to do their installations and sometimes not doing it correctly and then also questioning what other controls do you have in place in this process with this code obviously these huge organizations they're not like muscle they don't have one guy writing the code they have hundreds but how do we trust that code is secure especially when we're dealing with some a lot of the gaming operators do a large amount of offshore development what are the controls in place luckily we do have regulations with security components when New Jersey the second state to legalize eye gaming they came out with very very comprehensive security controls at least more comprehensive than what was in the unregulated industries if you recall at that time there was really no security controls in place and to date New Jersey is very proud of the fact that they haven't had a license or a breach or their controls are working effectively Maryland for example their gaming commission requires their land based operators to undergo an annual security assessment once again I'm not saying it's perfect but to step in the right direction to have these security compliance regulations in place and then of course we have a regular regulatory compliance standards PCI etc but as last bullet says there is no operator to determine the level of security that is implemented there's not strict guidelines even when it comes to New Jersey the testing is sort of left up to us working with the operator to determine how deep we dive and as you can guess probably a lot of that comes to funding and budget for those operators so conclusion while regulated eye gaming has added additional controls there's still room for improvement and this goes to brick and mortar casinos as well and also to the lottery sector you know in my opinion as I mentioned I think one of the key risks is in the code and that's what concerns me most you know and I think this applies to all forms of online gambling and also brick and mortar I mean it would be in my opinion pretty easy for something to be added to one of these eye gaming sites that could allow fraud to be committed it's happened in the past I think it's going to happen again and it's very important for the regulators and operators to work together and you know the last point is as you saw on the slide with the map you know while it's still small with eye gaming it's growing daily fantasy sports is here as you can't watch a football game without seeing draft kings or fan dual advertising you know it's become more widely accepted sports betting looks like it could become legal in other states besides Nevada no longer is gaming focused on Nevada and Atlantic City it's across the entire United States and as it becomes you know as it expands that tack footprint expands and the opportunity expands for crime and fraud to be committed so that's what we have I hope you enjoyed the presentation I'm sorry for some of the technical delays and we rushed a little bit in the beginning when we probably didn't have to but we would love to speak with any of you outside if you have any specific questions especially for Evan I know he went through those code slides pretty quickly in a little bit more detail as far as how that code worked the various functions etc etc but thank you for attending and enjoy the last day of DEF CON


complexity about modeling complex system to go and react. So technically speaking, I don't really need to give this talk or do anything of this with Go, no react. But it's something that I think is pretty interesting. Then Go is something that you can actually do this with. Most of the time when we actually use technology, and I assume everyone here is an engineer, right? I only know, I don't know three guys, the rest are actually. And most of the time when we use these technologies, we write business systems, and we actually help the business to deliver systems that generate money. But I wanted to give talks like this to showcase that we can use technology for other purposes as well, for many other different things. And one of those things that I want to talk about today is using technology to do some investigation, some experimentation on complexity, complexity being sometimes called complexity science, being a science that talks about the behavior that emerges from a group of interacting parts. And this is not directly resulting in the interactions between individual parts. It's a bit worthy of one of the examples, analogies that I want to give as with this. I know this looks strange to you. What do you see in this picture? A dead dog. A dead dog, OK. A lot of people in the picture, dead dog. If you look at the whole picture yourself, it's a portrait. But if you look at individual components, then you will see that two people are dead. Yeah, so it's an optical illusion. And again, most people would not use an optical illusion to depict complexity science. Complexity science is not an optical illusion. But what I wanted to show here is that if you look at it from a certain way, it looks like a portrait of an old man. But if you look at it from a different angle, it's actually consists of two people, one old man and a lady carrying a baby.  Slipping. Probably sleeping. So there are characters. If you look at them individually, they are different. But if you look at it from a whole, it's actually a different kind of character. Hey, more people. Welcome. So complexity science is somewhat like that. It is not exactly the absolute technology. But it's just that if you look at it from an individual point of view, you will see something. But if you look at it from a bigger picture, it actually becomes something else. So that is the idea behind complexity science. So I'm going to talk about three cases of how complexity science can show us something interesting. And the first thing is about something like what would you do under these circumstances? In 1964, a 19-year-old girl called Kitty Chanovisi was murdered by a vagrant. She just recently graduated from high school. She was working as a bar manager in a bar. And she was going home after work one day. She parked the car, came out of the car, was approached by a man. She tried to run away. The man chased her and stabbed her a couple of times. And when she was stabbed, she screamed out, that help, help, I'm being murdered. Apparently, the place where she was stabbed was actually in front of an apartment complex. And the windows were open in most of the apartments. And obviously, there was a time where most people were at home. So it is most likely that a number of people saw her as well. It was reported that 37 actually saw the murder, but none of them caught the police.  So that's like 51 years ago, kind of co-blooded. Is that what's happening in society and everything? But more recently in 2011, in China, a two-year-old girl called Wang Ye was hit by a van in a city in China, then ran over by a large truck. And there were a large number of bystanders who also watched that happen and into anything. She was left lying there for about seven minutes before finally somebody came in and helped her. But unfortunately, she also died a few days later. So you look at these tragedies, you see that, hey, what's happening in this society? How can people be so co-blooded? Is that what civilization is coming to? Is it like the end of the world? Why are these things happening? If you were one of them, would you have helped? So some of the questions you could come to mind. This is actually a known phenomenon in sociology, social sciences, and so on. It's called the bystander effect, where it happens to have a lot of people seeing the same thing at the same time. You would expect maybe somebody else to maybe call the police or help, because if the more people happens to be around, it is less likely that the person will be helped. So this is actually a well-known phenomenon. And what attempt to explain it is using game theory. Anyone here knows what game theory is? It's a different case with INB. If I don't do something in INB, we have some strategy to fit the system. It's something like that. So obviously, it's not really about games. It is about playing games. Do you use sport popularity in this respect? Yes, you do. So it's actually a mathematical study that studies models between people, conflicts, and cooperation. And it's often used in economics and political science. It's mathematics, essentially. There's actually quite a lot of mathematics. So it sort of overwhelms you of what will come in a while. Specifically, the model that we're looking at bystander effect is a model called the volunteer dilemma. And it's not just found within humans. Actually, it's found in nature as well. This animal is called a meerkat. Most popularly, probably, you first seen of this in Disney's Lion King. The character is actually a meerkat. So the meerkat has one very interesting behavior that sort of illustrates what the volunteer dilemma is. So during time when the meerkat forages for food, one or more meerkats will actually act as a kind of sentry. And they will stand up like this to make themselves look taller and then just look around to see whether there's any predators around. If there's a predator, what will happen is that the sentry will let out a cry or a warning to warn the other meerkats to run away for safety. But you can realize that the dilemma here, the volunteer dilemma, once the meerkat actually cries out a warning, obviously the predator would target the meerkat. So if there are a few sentries and they saw the predator, would you run away first? Will you sound out a warning?  But of course, then the other meerkats will get hunted, might get hunted down. Or if you sound out a warning, then you yourself might get hurt. So this volunteer dilemma, your altruism is actually not rewarded because your altruism might be sort of very fatal for yourself as well. So that's in nature. But let's go back to the model just now on the murder of Kitei Jonevisi. And let's look at the mathematical model in modeling or modeling what happened. If you look at V as being the value gain if one person volunteers, C as the cost for volunteering, and A as overall cost if no one volunteers. And then we draw a matrix, what is called a payoff matrix in game theory. So you have a matrix for two people. Let's say there are two other people and both you and I saw the murder. What happens? If I volunteer and you volunteer, obviously, if I volunteer here, this is the value gain for volunteering. And of course, if you volunteer, this is the cost of volunteering. So it's V minus C. And here is V minus C as well. If I volunteer, it's V minus C, but you don't volunteer, you get the full value. You get the benefit, but you don't actually suffer the cost and so on and so forth. So this is a payoff matrix for two players. So here, if I volunteer, it's V minus C. And if I don't volunteer, this is V. If I don't volunteer, but you volunteer, so I get the full value of V. If I don't volunteer and you don't volunteer, so we get V minus A instead. A being the cost of the bad thing happening overall. So from this, we want to achieve a equilibrium called the Nash equilibrium in game theory, where these two sides equate each other. So you have V minus C equates with the P, where the P is probability of volunteering, times V, the value of cost, plus 1 minus P, V minus A. So this is the equation for a mixed strategy for Nash equilibrium. OK, so far? That's for two players. Now, if we extend it a bit for n players, the probability of volunteering gets P to the power of n minus 1. And the probability of not volunteering is 1 minus P to the power of n minus 1. The whole total equation, assuming there are n people in that particular game or in the particular scenario, this is the equation. It looks complex, but actually it's not really very complicated. It's just a simple algebra. You break it down. And what you get is the probability equals to 1 minus cost over A to the power of 1 over n minus 1. We have a formula now for this particular game. So what we're going to do next is to do some Monte Carlo simulations. And of course, obligatory code. And this is really the code I'm going to show you later.  Sorry. Why does it not work? OK, I forgot. OK, so volunteers dilemma. Number of agents are number of players. See, we see this overall cost. We move this around. And here there are no agents as well. The more people it goes. Now let's look at this. The first one is the volunteer cost. The cost of volunteering. Now the cost of volunteering is low. What do you see? The probability of actually somebody volunteering is pretty high, right? It's close to 100%. As the cost increases, you can see the probability drops until it hits zero. And this is obvious. If it's actually so costly for an individual to volunteer, the more costly it is to just likely somebody will volunteer. Now if it's overall cost as well. What is overall cost? It means it's so costly overall that the bad thing that happens if nobody volunteers. If the bad thing is so bad, then you can see obviously the bad thing is so bad, the worse it becomes, then the more likely somebody will volunteer. That makes sense too, right? And of course, if the overall bad thing is bad, it is basically meaningless whether I volunteer or don't volunteer. If the bad thing happens, it doesn't matter. Obviously nobody will volunteer. So that makes sense too. So what I'm trying to get is, yeah, the model makes sense. And of course, you look at the here, the agents, or the players, the more players it goes, you look at it, any difference? There's no difference. What it tells you is that really doesn't really matter how many people are there, right? So let me come back to the presentation. If I could find my cursor. OK, let's look at some of the things. This is where you saw this now. Conclusions. Now this is conclusions from what I derived from the chart. How do we improve the volunteering? How do we make sure that more people actually volunteer? So obviously, it decreases the cost of volunteering, right? Make it easier for people to volunteer. And why do we want to do that? Because we want to actually make it less likely for such tragedies to happen. Increase the world's cost of not volunteering, which means that the cost of nobody volunteering is so bad, so terrible, that people will want to volunteer. Of course, you don't want that, right? It's like, if nobody volunteers, there's going to be a nuclear war kind of thing. Of course, somebody will rush through to volunteer. Increase the difference between individual cost and overall cost. So obviously, that's it. The difference is between individual cost and overall cost. Let me just quickly go back to this thing here. If you look here, the difference here is about 75%. But let's say you put here. But if you increase here, you notice that's the difference as well, right? So the actual thing is that what you really want to do is to, what you really want to do, oops, it goes back again, is to really increase the difference between the individual cost and the overall cost. It's not to really decrease the cost, individual cost, so low, because sometimes it's not easy to. Or increase the overall cost because, of course, you don't want to judge it. But it's really to increase the difference between the individual cost and the overall cost. Reduce the number of players. Again, you look at the distinct again, right? Agents, if I really look at this and I reduce it down to a very low number, you notice the probability here, 65, with three people. And probably it actually drops afterwards. What does it tell you? It tells you that actually with fewer people, it is actually more likely that somebody will volunteer. But after a while, if you go up to here, the number actually doesn't change much, like 40, 40 something. But the initial part of it, you look, with just four people, it's 55%, 51%. But after a while, it just drops to 40% and it's still the same to the end. So from the charts, you can say that if you want to, if you want to encourage volunteering, you should consider reducing the number of players. And that could come in different ways. For example, we use number of players. It doesn't mean sometimes you cannot control the number of people on the scene, but maybe you can control how people, me as an individual, can see how many people are around. So that's really what it is. Not really just absolutely how many people are there. And finally, the other conclusion we can see is that the increasing number of players have a negative or no impact. OK, so far? All right. So the next thing that I want to talk about, anyone know what this pattern is? Except Vincent will probably know for sure. Tintin? Tintin, this is Tintin in Congo, I believe. This is obviously very in an age where it's OK to depict Africans this way, but probably not. And there were people who were talking about Tintin being very racist. But obviously, if you're growing up, you probably never think of it that way either. And this is something to do with what I want to talk about today. A cartographer called Bill Rankin, he actually, what he did was he created a map of Chicago where each dot here represents 25 people. And then what he did was he did a taxonomy of race. And what you can see from this particular diagram, pink represents the white and blue represents the black people. You can see very obviously that this whole city is very segregated by race. And you look Hispanic and blacks and the whites don't actually mix. So Chicago people are very racist. But if you look at other cities in America, Detroit is a huge straight line here. You see this straight line here. You look, this seems a bit more faded. It's more faded because it's, let's say, Detroit is actually less people than New York. And this is why this is darker. One dot represents workers, 25 people. In New York, it's pretty much segregated the same way. Washington DC less, but it's very obvious that there's a left and right. East is more black, and west is more white. LA is actually less, it's a bit more diffuse. But still, you can see that there are the sea of Hispanics here, and then some blacks here, and then white people along the beachfront property. So conclusion, Americans are racist. But not really as well, like so you look at London. This is from another chart, a census of ethnicity, a lot of blues. And the rates are actually segregated from the greens and segregated from the blues as well. And London is one of the more racially diverse cities. But still, you notice people are actually segregated from each other. What does it tell us? If we actually go to other cities around the world, which I don't have the data, I believe that's also very much true. People are segregated, and we look at people in our region. You will also notice that there are more people who are. If you do not actually enforce certain rules, then people would tend to clumb with their own race, with their own kind of people. Just like we are gathered here because we are all go-lang people. So people are actually segregated here as well. Is there a real study about this? Yes, and is there a reason why this is this way? There is this guy called Thomas Schelling. Anyone know who Thomas Schelling is? He's a Nobel Prize winner in economics. He's an economist. One of the most popular papers he wrote was this dynamic models of segregation, which talks about how people are segregated racially. His model is different from the one that I'm showing you today. Basically, what I did was I read his paper and reinterpreted it in a slightly different way. The model I use is one of 36x36 green, with each with an agent. And if you look at it, each cell has an agent, and each agent has eight neighbors, except for the corners and the sides, which they would have less. Let's take a look at the rules of how we model this. At every tick, at every interval, check every cell. If at least n number of its neighbors are the same race, do nothing. And here, say, if I put the n to be 2, if I have two neighbors which are the same race, I don't do anything. If not, I will randomly pick an empty cell and I'll move there. Basically, what I'm modeling is this. If I say in this house, and I notice that my neighbors, there are just two of them who are my race. If the n is 2, and I'm happy with that, it's fine. If now, it is only one of my neighbor is the same race, I feel unhappy at all moves. So that's basically the rules. And I'm basically abstracting it down to that model and displaying on a grid. So we're looking at the parameters. We have n, which is the acceptable number of neighbors, r, which is the number of races on the grid itself, v, the percentage of vacant cells. And policy limitation, should I limit the number of neighbors to be on the same race? Because as you know, if we live in HDB flats, sometimes, well, you cannot have more than certain people of particular race and so on, right? What I'm trying to do here is to see, does it help to have policy limitations? And what does it mean by being racist? If I put a certain number of people within this particular block, rather, put certain people there, would they still tend to clump and how racist am I? Obviously, the larger the n is, the more racist you are. Because the smaller n means that I'm okay for diversity. The larger n means I'm not okay for diversity. And again, more obligatory code, and I can show you this, you know? Let me show you the second model, certification. So, set up, random, this is totally random. Let's say n is two, and there are two races on this grid. The percentage of vacant cells, 20% of the, of course, it has to be a certain percentage, because if all of them are occupied, then nobody can move, right? And of course, the neighbors quota, I put to eight because the maximum number of eight neighbors are eight, so I put eight means there's actually no quota. Let's see how it goes. See, from the original, if my acceptable number of neighbors is just two, they would actually be segregated, which means here that even if my tolerance level is really high for racially diverse set of neighbors, it will still end up being segregated. Let's set up to one, see what happens. You notice what happens? Well, very little movement, which means that most people are pretty happy, right? So you will notice that the acceptable number of neighbors, maximum number is actually eight, so only when one, then there is no segregation, which means really you need to be really, really racially tolerant in order not to be segregated. And if you look, you remember the charts I showed earlier on, that's part of the reason why you see that, you see that the chart seems to be very segregated is because it doesn't mean that, it doesn't mean that you are actually racist to be segregated in the city, right? Without policy limitation. Let's put this, let's say my, I'm highly racist, we said that, okay. I think so, whoa, whoa, whoa, whoa. Everybody starts moving around, right? Huge jump, and they keep moving because why it will never stabilize? Because nobody's ever happy. Everybody keeps moving. So obviously this is good. So let's go back to two where they actually become segregated. So what about number of racists? Is it, maybe because there are two racists, therefore they're like this, let's increase the number to four racists, but if they're highly racist and there's no two racists, shouldn't they eventually settle left or right? So that's the rule is actually. Especially if you run like one million in the nation, it's sooner or later they might find it. No, so let's do that again, you're right. So here the movement sort of mostly stopped. So it's still segregated, doesn't really matter in the number of racists. Okay, so let's go back to here. Let's say two racists, just two racists. Initially a lot of movement, and after a while you notice that the movement actually dies down to nothing. So this is really what it is. It will not seriously get like top down, it doesn't. But just like it's switching positions a lot of time. It was switching positions because people were trying to find a spot where it is acceptable for them. But after a while they just stabilised to this and they're not moving again. See nothing actually moves, nobody is actually moving. Everybody found a spot in the world and they are happy there. I mean they are actually segregated now. I'm not sure what this sound is. Let's say increase the number of amount of vacancy. Again, there's lots of space for people to move. But you notice after a while again that it actually clumps, right? It actually gets segregated again. So the amount of vacancy doesn't help either. Right, increasing doesn't worry at all. Now let's look at this quota. Just now there was no quota unlimited, right? Let's limit it. You cannot have five which are more the same as you. Because everybody is unhappy but you still cannot say with the same people it keeps moving. So does the policy actually help? Well, it does help. But the policy cannot be too extreme. You notice here it's a little bit more diverse. I'm sure I cannot remember what was the best number. And there's a certain point in time where there would be stability but it would not be segregated since it's not acceptable. But you notice the movement is actually less than previously. Probably the best number is 57. All these are all integers. If you look at the bigger grade and a different set of parameters you'll probably settle at a certain number. So the conclusion is let's get back to the slides. Screech conclusions. So segregation do happen even there's a big preference for all the papers of the same type. The weaker the preference the less segregated. So if there's more preference I didn't really show just now somewhere in between. It's not so it's two and I show five and it's two and it's five. But if it's somewhere in the middle actually what happens is the segregation becomes clearer. I can actually show you. You see it takes a while but it does stabilize, right? It stabilizes and you see this is actually less segregated than with the quota is eight. That's okay. Now let's show three. Just now this is about this three. You notice the segregation is more distinct, right? The clumps are actually bigger than n equals to two. So you look at this after all it's stabilized. Let's say n equals to two. The clumps are smaller. The segregation is smaller. And this is what I meant here. The weaker the preference the less segregated. The stronger the preference the more segregated. At a particular threshold the stronger preference results in an unstable but non-segregated state because everyone keeps trying to move around. Number raises have no impact. The both of you can tell us have no impact. And policy has a limited impact. But if you try to impose a very strong policy what happens is just unstable state. So the last piece that I want to talk about is clash of cultures. I got this actually from a game. There's apparently a game called clash of cultures. I got this one there. You played it before? No, I played it past a class game. Oh, okay. Yeah, funny picture I got somewhere. People turn up at a football game and talk about different things. And really what I talk about clash of cultures here, right? Mickey Mouse in Japan. This is Avengers in Hindi. Starbucks in Beijing. This is, I think, in the forbidden city. Of course, we talk about American culture invading other parts of the world. But also talk about other parts of the world invading the US, right? So yeah, Chinese food in the US, which is actually not Chinese food. I eaten some of this before it tastes horrible. And of course, yoga in US in popular culture is not exactly the yoga that's practiced in India anymore. So what is one of my trying to say here is that cultures do clash and do diffuse with each other. And yeah, pizzas as well, the original pizzas and pizzas that you ate. This, I don't know what kind of pizza this is. It just looks horrible. It's an American pizza. There is a paper written by this guy called Robert Axelrod. He's a political scientist and also a complexity theory researcher. He won prizes before. So he talks about how culture disseminate with each other within cultures and how cultures interact with each other. Do they clash? Do they absorb each other? And how does one culture transform into another culture? Again, the same thing as the previous experiment I ran. I took his ideas and I sort of re-implemented it in a slightly different way. Of course, Robert Axelrod never actually implemented this in this way at all. So if I look at culture as being a set of features and a trait being a possible value set of features. So what I'm trying to do here is I'm trying to create some agent-based modeling where each agent represents a different culture. So if you see on the grid that again I will show you, each cell has one agent and that agent represents a particular culture, whichever culture you're talking about. And every culture has six features. So zero, one, two, three, four, five. And each feature has 16 possible traits. So if you look at this, what do you see? How do you think I can represent this? Two, like two digit code then. It's actually a cheetah, right? It's a color code, it's a hex colors. Basically I have six features, one, two, three, four, five, six. Each feature is a hex number. It's basically zero to F, 16 numbers, zero to F. And of course as the numbers change, then the color actually changes ever so slightly. As before, eight neighbors per cell. And the group, what happens is that agents who are similar to each other will interact and become even more similar to each other. This is how cultures work as well, right? You have two cultures who are geographically near to each other. They will influence each other and then after a while become more and more similar. So the rules for the simulation, again, every tick I'll randomly pick a N number of cells and I compare the features of one culture with its neighboring cultures. If the trait difference is less than T, the trait difference here is T, but I randomly select either culture and copy one trait of the feature into the other culture. Basically it means that both cultures become more similar to each other over time. And this is how it works. So you have A and B who are neighboring cultures. And I compare every one of these traits. So five and two, difference is three. You add them up together. So the difference between the two cultures is 34. If the cultures are 100% different from each other, it's like black and white, then you have 100% is 96. If the difference is zero, they exactly each other means both of them are the same color. That will be zero. It's more similar, the cultures are the more likely to be a cultural exchange. So let's say you look at the real life, for example, if the culture in one place and the culture in the place, they speak the same language, it's more likely that then certain traits will actually be absorbed into each other. So what we get is the probability of the culture exchange is one minus the difference between the two cultures by 96. Simple formula, that's the probability. And when there is culture exchanges, what happens? Five here, we copy it over to five here and then the color changes here. So let's look at the final simulation and I'll need to copy this thing here as well. Why do I need that? You will see in a while, okay? So set up randomly, just set it up to different cultures on the 36 by 36 grid. You see the blue here is the distance between two cultures. The red here is the changes. And the orange here is the number of unique cultures. So what do you see here? Obviously the change, it keeps changing, right? Because culture keeps clashing with each other, mismatching and exchanging ideas and so on. Number of units drop, which means as the cultures mix, they are actually less and less unique cultures. Originally the whole grid is full of all unique cultures, every one of the cultures is different. But as time grows, the number of unique cultures actually drops. The distance between the culture actually drops as well. So initially every culture is very different from each other, the distance. It drops, but you will notice that the number of units actually drop more over time, while the distance actually do not drop over time. This actually takes some time to run, so I won't actually go to this, I'll just go back to the slides and show you some of the conclusions. So this is actually what happens. At one point in time, distance will probably, over a long time, will probably drop down to zero. Changes will drop down to zero, the units will drop to very low number as well. What does that mean? It means actually it's the dominance of one culture over other cultures, right? From a grid of 36 by 36, a lot of cultures, you will drop down to very few cultures. So do they, among the last ones, are there actually any that is part of the first set? So I'll talk a bit about that. So average distance tells us how far apart the cultures are. Units, how number of cultures changes, how vibrant the cultural exchanges are. So after a while, there will be no cultural exchanges because they all say so in conclusion. So eventually the equilibrium will be only a few dominant cultures. It will just be a few. It will not be like so many different cultures. Dominant cultures can be very different from each other, which means that even though they are, even though a few of them, it doesn't mean that they're all the same each other. Some of them can be very, very different from each other. And that's probably, again, one of the reasons why they are very dominant because they're very different from each other. If I increase the number of grids, I don't have time to show you the number of grids because it involves some code changes. Smaller areas results in faster equilibrium, or smaller than the number of dominant cultures. If I have very small grids, sometimes you can go up to just one dominant culture. A culture that's more dominant at one point in time doesn't mean it will be dominant in the end. So what happens if you look over time? The number of cultures, the number of dominant cultures. So let's say at this point in time, you see this, let me just show you. You see here a dominant culture here is somewhat like a yellowish here or this light blue. And it could be very dominant over time, but after a while the light blue could actually go away entirely. So dominance, initial dominance, doesn't mean eventual dominance, because the cultures actually change over time. That probably answers the questions, right? So that was my last slide. I know it's kind of abrupt, but, you know. And you can see this slowly when it's actually quite pretty. You wanna see the code, I can show you the code here. So it's a web app. No third-party libraries, send and go libraries. This is segregation, it's actually quite small. This is the one, just again, it's really small. And this is the culture. It's big, that's why I didn't actually kind of paste the code in the slides. The final is basically, say, react. Sorry? Not yet, is it? I think it is. You saw it. So to be utterly a good friend, right? Yeah, actually I can't remember what to put it up with. Okay. You will be, you will be. Any questions so far? Thank you very much. Sir, is there any, who's the dominant culture here? Is it random, or can you start off with some topic in your opinion? No, it's random. So initial, initial cultures are all random. Eventually, even the cultures, yeah. So the initial set of cultures would not be the same set of cultures towards the end either. So it's not seated, and the way the experiment is run, I also did not keep track of whether the culture actually, the initial cultures actually become dominant in the end. But probably it doesn't because all the cultures actually morph into something else. Which again, is another one of the poise here, right? Because whatever culture you start off with, do not expect it to be the one at the end. Like, you see the American culture today, is probably not the same American culture in the, say the 1900s, or the 1920s, 1950s, or the 1980s. It would be helpful if you can draw some code logic in time to your demo, because it was mostly demo and there wasn't any code logic of how you put that in the code. I actually originally had that, but I was afraid that it would be a bit too long. Because the focus was more on the demo and on the story, but rather the code logic. So we did a little bit, as you demo, you can show a little more interesting. Good point. I think that's cultural, for sure, but I think the race stuff can be a cultural show. In some areas, I mean, for example, in France it's not really good to see racial statistics and stuff like that. That's why probably it's a bit shocking, but I'm sure in India, they don't really guarantee probably it's normal. In Singapore, it's very normal. In the US, it's very normal. But it could be triggering some issues. Okay, good point. If I actually get to speak in France, I'll... I'll be there. On JavaScript, we use... Sorry? On JavaScript, we use, well, JavaScript also in the HTML section. Ah, okay. So this is one of them, this is a volunteer's jazz. This is React. This is the React bit. This is basically nothing much more than the container for the code. It's just the container for the code. For the grid, I actually use a good strap. So each grid is basically... I'm basically using a CSS grid. And that's why you see that I have a 12 by 12, 24 by 24, 26 by 24. Probably should have done this a little bit better, but I just hacked it out. Oh, like React components? These are our React components. The React components. See, I'm using... Basically, this is... Probably don't need to put all this, but this is basically bootstrap and covering the background color. React and the code? Obviously, it hasn't got the code. Just like this, it's... There's a click start. It calls culture slash start. And culture slash start is basically... This guy. Culture start. Basically, this produces JSON. And it emits to that guy, that guy takes the JSON and then you would just repaint this. In the JavaScript, I have a... Set interval to every half a second. Refresh the screen. That's how I got the grid. So the go part, does the JavaScript... At each point, there are different states, right? So as it runs, it will finally... So does the JavaScript tell the state to the go part and the go part process and use it back? Or the go part just... JavaScript asks for a second, make state, make state, and the go part stores the previous state and processes. So all the state is at the server. It will just emit and every half a second, it will just grab the current state. Okay. The JavaScript, the React part is all reader only. It actually doesn't send any data back to the go. It just emits. So the experiment is actually entirely in code. That's why I show you go code only, right? The display is in React. I could actually do this some other way as well, right? I could use like a mobile app and sort of get the data from here. This is all JSON data. The JSON data is actually quite simple. It's just a whole lot of numbers. Was there anything unique about using Go to do what you just did instead of using another language line? One thing is probably speed, to be honest. Yeah. I think I would have problems running. So I've not really tried this in Ruby. I only did this in Go, but I did do agent modeling in Ruby in the past. I did my previous presentation about this. And it generally slows out a lot as the numbers become large, whereas this guy is still chugging away pretty fast. Did you make use of Go routines? No, I didn't. I could have done Go routines in more complicated agent modeling because then each agent itself could be a separate Go routine. But that's not something I did for this guy. This one, there are more just different types of simulations. I could have done the whole thing in JavaScript as well. How are you using Go as well? There's a terminal you can pin off on the terminal. On the terminal? It just leaves this blade on the matricle or something. It probably would be possible, but I have no idea how to do it. I would have to repaint it. This was easier for me to do this. It just emits JSON, and then I have a front-end that will just keep on reading it at certain intervals. In fact, I don't need to set it to half a second. I can set it every second, and it's still the same because I'm just reading the state at that point in time, whatever state it is. It means that the data would be... There's two tickers, basically, and the Go ticks on each side. You will go tick on itself. And JavaScript will tick on itself. You will just refresh at certain intervals. Of course, it will look more jagged because the data points that you take are actually far apart from each other. So if you take it every five seconds, of course, suddenly you see this is very random. Then after that, the next tick, it's like, oh, it's done. Because in between, basically, it's actually done through a lot of iterations. Whereas your refresh rate for the JavaScript won't be a lot slower. On both the server and the UI, you're using half a second. No. On the server on Go, there is no... It just loops itself. It just continually loops. What makes it a more continuous state? It's just the state and the final millisecond. Yes, correct. So of course, if I go down to every, say, 100 milliseconds, then I will see things to change a lot more rapidly. And this more contiguous kind of change. If I set it to one second, then it seems more jagged. The change seems more abrupt. But it's still the same, because whatever Go runs, it's at the back end. Did you get the chance to add benchmark?  I did benchmark. Because what I used to do in Ruby in the past was, I would run this, and I would generate data files, and then I would run another program to generate the graph out of it. So it's not real time. Because Ruby, it can be a little bit slower. But Go, I'm actually doing this real time. And React also allows me to do some refresh. But it's not really React. In JavaScript, it allows me to do a lot of refresh quite rapidly. And the charting is done with Google Charts. Charting is done with Google Charts. So it's actually quite fast. Everything actually runs pretty quickly. Any other questions? Any other feedback? Thank you for the feedback. But any other feedback on the presentation? I'm trying to use more variables in the models that we took. To understand, it's very simple. We can get it from like a bundle. To think about more, for example, even in this, apart from distance and the difference in the races, et cetera, there are many, lot more factors, which we can include one by one. Yeah, that's true. That's true. Probably the next time that we have a project, we could try more factors inside and see how complex it could be. My idea was that if there are more factors like that, maybe the equilibrium that we see would be quite different. It might not happen. And it would always be some turbulence or something. Yeah, you're right. And I think one of the reasons why it's done this way as well. I think it's also done liberally. I basically took an existing paper and I interpreted it in a particular way. And the parameters are actually from the paper anyway. But the reason why it's done that way is really to limit the number of parameters so we can analyze certain parameters. If I change this parameter, what happens? If I change that parameter alone, what happens? It's a science experiment. You maintain all the other variables the same. You just change one variable. How does it change? So that's the purpose of that. The more variables that I have, it's true. You can actually try different variables. But it doesn't go away from the initial principle, which is to keep everything the same, just move one. Keep everything the same, just move one. All right, thank you.


The topic of where can you find the best pizza in Jerusalem comes up very very frequently in social media groups for English speakers in Jerusalem. My opinion there's only one correct answer. It's a restaurant called Rendezvous in Bacca in size Jerusalem. This is it here right behind me. It's a little restaurant that you'd never guess does such incredible pizza. This place is literally like a French enclave. You'd actually hear more French spoken here than English or Hebrew. They also do fondue. I'm meeting up tonight with a couple of friends. We're going to go out check out their food. So let's come inside and check out Rendezvous in Bacca. So my pizza has arrived now. It's the tuna pizza packed house here on a Thursday. It's just exceptional pizza. I'm not saying this based on Jerusalem pizza standards. Israeli pizza standards. I'm saying this upon world pizza standards. I really think this is incredible pizza. As you can see among the French community this place is very well known outside of it not too much. Anyone who tells you there's better pizza in Jerusalem are wrong. It is coming with a cheese melted on top of a potato here on our tasting channel is throwing it at you. Would you like a potato? I think this is the best pizza I've ever had in my life. So we finished up our meal here in Rendezvous in Bacca. Everything was absolutely incredible. The pizza was amazing. The creme brulee was also amazing. It's finally closed for the nice here and people have emptied out. They have a great garden as well for people who prefer to sit and this place is very popular with a great meal. If you're looking for the best pizza in Jerusalem you should definitely come to Rendezvous in Bacca.


When I started in the cleaning business about 30 years ago, I thought that I could go up and down the grocery aisle shelves. I know what you're thinking, you've done the same thing, right?  And you're like, oh yeah, I saw this on TV. I should buy this. Wasn't too long into my cleaning business before I noticed that I had chemical sensitivities where my eyes would turn red just by the cleaning chemicals that I was using. And then by the end of the day, my nose was woofy and I couldn't breathe properly. And so going to work every single day and not factoring that if I was gonna be in the cleaning industry for 10 or 15 or 20 years, that I was gonna be responsible for my own physical well-being and the cleaning chemicals that I used. And so somewhere along the way, I started looking for a safer choice, a cleaner choice of cleaning chemicals.