Thank you so much for being here for this presentation, which will be about the Collabora Office Android app and what is inside. So, but before we can see it and I can show it the details of that, let me first get into the history because as you can see, like it's a quite a lot of history in this, like the work on this started more or less in 2011, and the first thing with LibreOffice because you can imagine like it's a huge codebase and getting it to Android meant like first to get something on the screen which was pioneered by Michael and Thor. So, thank you for starting this at all and it was like incredible amount of effort that was needed for this because for example, the linker just allowed like 96 libraries to be loaded at the same time. And of course like LibreOffice itself had like several hundred and it used some component stuff so that like it was all loaded on demand. And like all this had to be overcome to do the state that actually like something got on the screen. The debugging of this was a nightmare like the stuff was not ready for the native debugging at that time at all like you had to upload something to the device, put some weights into the startup procedure and you know all these nightmares but they succeeded. So, they've got something on the screen which like you can see tablet inside there was the screen of LibreOffice. But of course like this is not what you expect from the mobile app. So, like it was unusable in this state but it was possible. So, the next step was like trying to get some host screens, host pages on the screen which seemed like a good approach at the time. So, it got it to the application that like had some list of screen, list of document pages and you were able to switch in this. But scrolling the document was like impossible because like you know you were able only to switch it like slides. So, the next step here was to use actually the LibreOffice kit. So, LibreOffice kit that came from the need of the mobile and web applications is actually some kind of API for LibreOffice that allows you to simply access the functionality of LibreOffice from C or C++. And it allows you to draw parts of the document into tiles. So, like you split the entire documents into like 256 to 256 pixels tiles. And then like you can request them. You can request them like by absolute coordinate. So, like the theory is that like when you just want to fetch something from the document you say, okay well I need it from over here, over there. And then you have some like composition that like composes that into something that looks like the document. Which was great, it functioned nicely. And like later it was extended to actually like that the editing worked too. It meant that like all the stuff had to be updated so that like when the user actually types something into the document. Like that the part that where the user actually typed is invalidated and new tiles are actually fetched for this area. Which was done by Miklos. So this is what we had in 2015. The problem of this was that, didn't I skip something? I did, yeah. The important part of this was that actually like the compositing of these tiles into form something like the document was done in Java. We have reused code or Tomas has reused code for this from Mozilla. Which had this Fenec prototype at that time. And so it kind of worked. But the problem was that like it divert from what the code was doing for the web. So of course the LibreOfficeKit API was extended over time. So that like more and more feature were added there. So that like you were able to type concurrently in the web browser. And also like have their more functionality exposed like through the dialogues and everything. But like maintaining these two code bases was like very inconvenient. Like you would have to be on par with the features like that were developed for the web, you would have to like do it just once again in Java for the mobile part as well. And so the idea that Tor was implementing for the iOS was actually like do everything like in a web browser even in the app. So basically in the background have the LibreOffice and LibreOfficeKit. And then like have full screen web view where you are running the JavaScript that is like normally used in the web version. But use that on the actual device as well. With all the cross-compiled code in the background and all these things. And it worked for the iOS so at some stage like I tried to do it for Android as well. And yeah, it had some problems as well of course. So first of all the old WSD has to be ported to Android. Like luckily it was not that hard because like it was pioneered for iOS as well. It was using the same native code that is compiled for the old like Fenec-based app that is written like entirely in Java. And but there were things that had to be done a bit differently. So first of all like I wanted this to be built in Android Studio so that like the debugging is easier. And like it is more standard for like anybody who comes from the Android world and sees this for the first time. So in the online get where the like web version lives. I've created the Android sub there and like introduced the appropriate configuration options and everything. Then edit the stuff that like had to be built natively. But for that I've created it's in CMake so that like Android Studio actually understands this and it is possible to build it directly from there. Luckily the debugging part of this worked pretty well so now like you can debug this completely like directly from the Android Studio. You can set the breakpoints even for the native part and all these things work nicely. If you want to know the details like how to set it up for this just read the Android slash read me it is described there. And yes so the minimal app was created so like the web view over the entire screen. And some kind of communication between the web view that is there and between the native native part that is like behind the scenes. So we had to we had to like introduce like two ways of communication. So one is from from JavaScript to native and that is like reasonably easy on Android. Like when you have the web view you just add JavaScript interface and then you can you can run it easily. Like like this low WS low message handler dot post mobile message and like you get it from the from the JavaScript to native. But for the native to JavaScript like it's a bit heckish because like you have to use this JavaScript prefix that actually runs like one JavaScript function.  Like it can be the entire the entire tile and like encapsulated that way and and you you will it will be called in JavaScript and then like processed by the by the online part actually itself. I was concerned about like how this is going to be performing but it is not that bad like it is not showing that much in the profiles. It's not terrible then like when this minimal app was was created like lots of functionality has to be had to be ported from the from the old app that we had because like it had the it had the shell that showed like what are the what are the files on the on the device. It had to be done so that like there are some assets that that come with the with the native part that like have to be present on the device so that like things were some of these assets have to be some kind of unpacked so that like they behave like real normal files on the device because like we depend on many many like a third part. Projects like font config and these things and lots of these have some configurations that just have to be like read directly from the disk so so like it had to be they cannot be as assets because assets are very special on Android so so like they have to be copied first at the first start and then can be used later. Then some like important things like displaying the license and the notice file and of course settings so that like you can you can tweak the configuration of the of the thing. Then we had the summer of code student during the summer or the last summer. Who added lots of things that are expected on the on the Android device. So some print support slideshow support which exports the the slideshow into SVG and shows it as full screen device. Inserting images again like you have to like start a separate activity that that allows you to choose the file and and provided back to the document so that it is inserted lots of useful stuff. And of course lots of bug fixing so one of the thing that that was a huge challenge was actually life cycle of the thing because because like the Android life cycle goes through like many steps and it is a bit unclear like in what state. Like how the how the native part is supposed to behave like when you are actually supposed to to tear it down when you are supposed to to leave it like in the background because like you know that that you will return to some state very very soon and stuff like that so. So like we had lots of crashes there luckily it seems to be in some reasonable reasonable shape just now then. Then it was necessary to to debug some pieces like the startup time. It turned out that like we were starting about a minute just because the font config was parsing the huge note of fonts that are on the device and and so like we had to overtake it some way and fix other crashes at some other stuff. And then we tried to publish in Google Play but just at the time like when we were ready or was nicely done Google Play like came up with the new policy and that was that when you have a native app in the app store it has to have 64 bit version as well. And so far like everything was 32 bit so we had to make the low WSD compilable on 64 bit which needed a Poco that that we that we actually use as like the main dependency for the online self. We had to have to port it to 64 bit luckily like it needed just few patches but still it was a bit annoying then deliver office core itself again like it is supported to arm 64 for a long time but on Android it behaved a bit differently. So some things there like necessary to be fixed in the bridges which which is the like very low level part of this so that first it composite all on Android and then and that it run because like the exceptions didn't work there so we had to do some hacks that were like already done for iOS but like not used in generally so so it had to be changed there. And of course the online build now had to count on actually having two trees of the of the compound native stuff so one was the 32 bit the other 64 bit and create some Android app bundle that is then uploaded to the Google Play and Google Play like when it serves to the to the devices like repacks that into the actual AP case and and then like the the users are actually served either with the 64 bit version or with the 32 bit version according to their device. So finally we've got it there and we've got through several iterations since then so again like more pieces from the old app were ported so for example support for SD cards or for cloud storage currently next cloud is supported hope to have more in the future then many improvements interact interaction and the biggest thing that was there was the was the mobile this art we call it this way because we don't have a better name but it's basically the the stuff that you see see down there. So some details and in there so thanks to Kyle on to for some design device as advice here. We were able to actually actually add some dumping of the of the content of the widgets that are that are in the dialogues or in the sidebars. So we are able to dump it as as Jason and transfer it to the to the JavaScript part and based on that like we are able to to display the information that is either in the dialogues or in the sidebar some more convenient way for the users. So like what what you see in the picture is the sidebar that you would see normally in LibreOffice but like present it in in JavaScript rendered in JavaScript. So it works nicely you have all the things like a smooth scrolling and and the interactions and all these things. So if it works I can hopefully show you some demo. Yeah. So this is like what you get when you start it on the on the on the device. Editing. Yes. Was started. Yeah. Okay. So if you have something if you are more lucky than me with my fingers then you will do it on the first attempt. Change some properties. You can insert some chart here. Sorry. No chart but table. So then you can see the the the mobile wizard in action here like change the columns and rows. Okay. Type something into the into the table. And if it doesn't fit you can resize the table. So that is more convenient like there are the special handles done for this for all this editing more scrolling. And that's it. I think we are at the end. Okay. So some future steps will be more to do of course more profile and optimizing. But at the moment like the most most terrible problems I think are are sorted out but still of course like more to do as usual. At the moment from time to time you could see that some document just doesn't doesn't load but I fixed that yesterday. So this is all basically what was going on is that some people with some like a bit strange. The locale setting for their lip a lip a length tag was not finding the data. So and it's it just like then didn't load and presented some like file not phones found the box update the documentation mode. So currently we just like copy the document from some temporary state into the new document and create from that. Like we would like to to use some real templates here for the for the documentation which is possible but not finished. Of course lots of paper cuts here and there. For example the most visible at the moment is smooth scrolling. So like the inertia so that like when you move with the finger quickly that like it goes on and before it reaches somewhere something. And yeah that's it. I think so if you want to get involved you are most welcome. It is not that hard to set up. Unfortunately like you first have to cross compile the the liberal history but then the rest is like reasonably easy reasonable familiar to people who have who has done have done some some Android development. And many thanks to people who who contributed directly to the Android port. But of course like lots of other people have contributed to the to the online. Collaborate that most of the work in the online there. So so thank you so much. So that's it. Any questions please. Yes. Yeah. Yeah. So at the moment like the the Google player releases are a bit more frequent than the the releases on the normal of online because like we have some additional fixes that are like directly in the Android part that are not not general. But of course like because all is called a code is shared. So if we improved something for the Android and it is in the JavaScript part it will improve the the the experience even for people who are using the web version the web version normally like from the but on the mobile device. So like they will see lots of these things there improved as well. But like currently I think it is more often than than the normal normal online. Releases. Or the code. Maybe it is like similar as the code. And from time to time there's like one release in between. So something like that. Like in the recent past it was like every two weeks. So I always so I always is a bit different in the releases because like in the test flight. We updated frequently but the normal app is now based on the photo zero still and we still have to update some stuff so that like we can we can like base it on for that too. So like the official release of the iOS app will update a bit later. But currently like if you are the test flight user you will see like similar pace of the updates. Thank you for the question. Any other question. Yes please. No video support. No video support. No video support. I don't think there is. Okay any other question. Yes. Okay. Any other question. Then thank you so much.


Hello. Good afternoon all. How's everybody doing? Joy in DefCon. I know I saw a lot of people at Black Hat. Probably most of the people that saw it there aren't gonna see it again. I don't blame them. It was worth seeing once, I think. So, okay. Well, we got more people coming in. One of the things I like to do at the beginning of the talk is just sort of have everybody take their right hand, stretch it out like this, and shake the person's hand next to you because you're gonna see them next year if you come. I guarantee it. Every year you see the same. Well, more, right? Some of the same people, but more. So, let's just get into this. The goal of the talk is to try and build a better fuzzer. That's the goal of my research. That's kind of what I've been doing for my academic work for the last year. And I'll briefly go through some background material. Kind of just real quickly what's fuzzing software testing, evolutionary testing. Then I'll talk about the actual system I built EFS and then talk about some initial benchmarking and then talk about some initial results on real-world applications. So, there's no definitive proof that EFS is actually better than, say, like Spike or Sully or any of the other fuzzers that are out there. Mainly because it depends a lot on the application and it also depends a lot on sort of like what you were trying to do. And I mentioned this at the very end of the talk, but like, for now, file fuzzing as one of the ISEC guys said at Black Hat, it's still depressingly easy just to take a file, flip a bunch of bits, feed it to the application, and the application crashes, right? So, there's no need to try and set up an evolutionary fuzzing system when you can just flip some bits. So, it kind of depends on your target, you know, what you're going after. Whereas, like, network applications are generally nowadays quite a bit harder to fuzz. They've been kind of hardened throughout the years, right? You can't just flip a few bits in an HTTP session and crash a patch. That's not going to work. So, it kind of depends, you know, on a lot of what's going on. To my knowledge, it's the first evolutionary fuzzing system that's been released publicly. You can go to my website, www.vda-labs.com, and download everything I'm talking about, the slides, and the actual code, and GPF is there as well, general-purpose fuzzer, something else I created. So, section one. Software testing, this slide is just to say, and I mentioned this last year at DEF CON, that software testing is just hard. That's basically all this slide says, and that it's not an easy problem. It's not been solved. It's kind of an open problem. And there's a lot of companies looking at a lot of different ways to do it. But you got to test, right? It's the only way to give you any sort of confidence that your software is good. And fuzzing, as I said last year as well, is not the only way to test. There's a lot of other things, just functional testing, and GUI testing, and quality assurance, and formal methods of development, all that kind of good stuff. But fuzzing is just one element. So, the basic premise of fuzzing is you get a test case delivered to the application, and then if the application has a problem, then you obviously record that, and if it doesn't, then you keep going on to either the next randomly generated test case, or the next test case that comes from a list of cases, or however you're getting them. You sort of move on. And of course, the most interesting places to fuzze are where there's a change in privilege, right? Like, code that comes across a network interface is unprivileged data. I mean, that comes across a network interface. And if the service is running at some privilege, and you can get a buffer or fall right, you can elevate your privileges from nothing to something. So, that's kind of part of the key, right? And any type of fuzzing is you're looking for some sort of elevation of privileges generally, or maybe you're just looking for robustness issues, whatever you're looking for. So, this is just to say that what you can fuzz is generally called the attack surface, and I think people understand this pretty well. I asked people at Black Hat as well, like, how many of the speakers have you heard mention that were either fuzzing or attack surface? And quite by and large, probably half the crowd raised their hand, which means that I think people finally sort of understand what these terms mean. The attack surface could be, you know, a local parsing of a file, the network interface, inter-process communication, any bit that could be influenced by dirty data, user data. So, evolutionary testing uses GA's genetic algorithms, and I'll talk about kind of how they work and where they failed in the past, just real quick, and kind of where they're going. So, basically the idea, if you haven't heard about this from last year or whatever, it's just sort of like survival of fittest mentality. Take, you know, mom and dad are so-so fit, and they mate and have kids, right? And then the children, ideally, are a little more fit, so it's kind of, each generation gets better. That's just, from a high level, that's the basic idea of genetic algorithms. They're a searching algorithm. They're searching towards something. So, if you have the source code and you do evolutionary testing, you're searching toward slash-slash-target. See that up there? So, here's a function. It's called example. It has four parameters, a, b, c, and d. And those four parameters, a is 10, b is 20, c is 30, d is 40. So, the first check is, and this is like either the randomly generated, or it's evolved this far so far or whatever data, if a is greater than or equal to b, then, oh, but it's not, right? So, a is less than b. So, you stop right there. So, there were two more branches to get to the target. So, the fitness is two branches, two plus normalization of 10, because 20 minus 10 is 10. So, that's sort of how evolutionary testing works in general against software. And this slide is just to say that in the past or even now, people are actually using evolutionary testing. In particular, it gets used in safety critical situations, like somebody might tell you, oh, denial of service isn't too big of a deal, right? I want code execution. Well, I want code execution too, but if I'm on the iron lung and somebody douses my iron lung, that's bad, right? And so, denial of service can be really bad in safety critical, automobiles, airplanes, those kind of things. So, they need to be really robust code. And so, this is one of the places where that kind of testing gets done. Now, this is just to show you, okay, so evolutionary testing, how does it do in some, in all cases? Well, I don't know about all cases, but here's a few cases where it doesn't do so great generally. Can anybody look at the code example on the left? Look where it says slash slash target. That's where you're trying to get. It's called flag underscore example. It takes two parameters, A and B. Does anybody see a problem with that? It'll never get there, as we said. Yeah, well, it could get there, but what happens is there's a notion in evolutionary testing called the fitness landscape, and that's sort of like, you know, you're down here and you want to get up here. How easily can you climb the hill? You know, is it a, it is a contiguous function, continuous function to get there? And in this case, the check is if flag, then target, but the problem is you don't control flag, right? You control A and B. So it's, as a human looking at it, it's not too hard to see, well, if you set A and B both to zero, then you'll get to the target. But, you know, in the past evolutionary testing didn't really take that into account. They didn't have any way of saying, I know what flag, they didn't know what flag is. So there's a way to deal with that, but that's called a flat spot in the fitness landscape. There's also something else called deceptive problems, which is if you look at the code snippet on the right, you can see it's function under test, takes a parameter, and you want to get to where target is again. It says if the inverse of that equals zero, then we'll get there. Does anybody see the problem here? So, it looks like as you give it, say, a hundred, it'll go into the inverse function and do, like, one over a hundred, and then you give it a thousand, and it's like one over a thousand, so it's taking the inverse, so it's getting smaller. So it thinks it's getting closer to zero, right? Your evolutionary testing algorithm believes that it's getting closer and closer and closer to zero, well, actually all along, as it gives it larger numbers, it's getting farther away. So, if you just give it zero, though, it would actually hit the target, but it doesn't really know it. So that's what's called a deceptive landscape. So, all that's just sort of background. This is section one, that's, you know, there's people working on that in academia. McMahon and Holcomb are coming up with what's called the extended chaining approach to do better with evolutionary testing, but the system that I design is better, or I don't want to say better, different because it works on binaries, right? Oftentimes, if you're auditing, say, IIS, FTP, or something, you're not going to have Microsoft's source code, right? So there's no way to really do the sort of traditional evolutionary testing. So you have to use, you know, a debugger in the binary and all that kind of good stuff. And I'll talk about how that works right now. Section two, so what is EFS? Well, it uses evolutionary algorithms, and it also uses fuzzing. I leveraged a tool that I'd written in the past called GPF, General Purpose Fuzzer, and it also leverages Peastocker, a heavily modified version of Peastocker in the PyME framework, which you may have heard of. And this is an overview, a system view of how EFS works. So you see GPF here is written in C down in the left-hand corner, and it's doing the evolutionary algorithms, all the fuzzing heuristics and stuff come from there. And the target monitoring is right there on the right-hand side. You've got the debugger monitoring the target, and they both talk to a SQL database so that each session from each generation can be stored. Okay, well, what is the session and what's the generation? We'll take a look at that, see what the data structure looks like. And I should say right up front that this tool was originally designed with sort of network protocols in mind, but I am porting it to file fuzzing as well. But it is sort of network-centric. You'll see it feels that way. And it has some reporting you can do from PHP. So the basic protocol is just sort of like, you know, the fuzzer sort of asks the process stalker, are you ready? And he says, yeah, I'm ready. And then he talks to the target, and then the process stalker gives the coverage information how many functions or basic blocks were hit, and those get stored in the SQL database. And, okay, so if you don't, it should be fairly clear what functions are. And I think I'd talk about this a little later, too.  So basically you get a list of all the function starts and all the basic blocks start so that you can stalk and either functions or basic blocks. And the basic block is just like in assembly, you have like push, push, move, move, branch. That's a basic block. Whenever there's a branch, that's like starting another basic block. So basically you have the choice when you stalk for coverage. You can either stalk on functions or basic blocks. And we'll talk about which one's better. There's a lot more basic blocks than there are functions. So it makes the stalking more granular, but it also makes it a bit slower. And the fitness function. So genetic algorithms, basically the key to a good genetic algorithm is having a good fitness function and a way to evaluate the fitness function. And in this case, the fitness function is two pieces. It's code coverage, how many functions are basic blocks were hit during the session that just ran, and also a measure of diversity, because what happens over time in a genetic algorithm is there's what's called convergence. And that means that you started out with all these good random pieces and you were kind of diverse up here, and your fitness started out real low if it's all like kind of random data mixed with basic blocks. And as the sessions get more and more fit, they all essentially become a like, right? They all start to look the same. The best guy is sort of winning the strongest guy. And that means also that the diversity is sort of going down, right? Because all those other guys that weren't as strong kind of die out over time.  Because that's important to hit that guy too, the best session that goes all the way through. But there's all these other little sessions that might, you know, trigger corner case bugs and stuff like that. So diversity is key because diversity will give you more total coverage of the tax surface. And ideally, you'll cover the entire tax surface. And it's a bit hard to actually measure, right? Because you may not know, like, if you, again, if you go back to the IIS FTP example, say you, you know, you put it in IDA Pro and you get a list of all the functions and say there's like 1,000 functions, well, how many of those 1,000 functions can you touch remotely through the network interface? You don't really know. I mean, there's not a real clear and easy way to tell. But if you could tell, if you could tell that it was 200 of those 1,000, then ideally, you'd touch all 200 functions throughout the course of fuzzing. That's sort of the goal. And that's sort of, I use the academic keywords corollary, one and two. The first corollary is that code coverage doesn't equal security. But if you don't get code coverage, then you're even less secure, right? I mean, you've covered, if you haven't covered it, you haven't covered it. So you can't buy bugs. The second one is true, which basically just says if you cover the entire tax surface and you do it with diverse sessions that have fuzzing heuristics, like long strings and dot dot slash and percent N characters and all the kind of good fuzzing heuristics that should be in fuzzy data, then that's sort of the best I know how to do. That's kind of the goal. Okay, so how does the actual evolution work? We'll talk about that first before we talk about some results. So basically it works by kind of recombining data and kind of slamming around, and any bit can be reorganized. And well, I'll show you how that actually works with a session crossover, session mutation, pool crossover and pool mutation because data gets organized in pools. Sessions of pool. I'll show you that in a second. So, but one thing to mention that is this elitism, which means that the session that performs the best in each pool gets carried over to the next generation automatically. He doesn't have a chance of sort of breeding and dying out. He just sort of gets to go to the next generation. That's called elitism of one. So how is the data organized then? Well, at the lowest level you have a token and a token could be of different types, like binary or ASCII or space or delimiter or carriage return or line feed or whatever. So each token, maybe you have a string that has the word user or something like that. And each leg is like a read or a write in the case of a network protocol. And so a session would be a series of reads and writes with tokens, obviously. And then those are all organized in the pool and the idea of the pool is to help maintain that diversity because if you have different pools of sessions that and they don't necessarily breed though they can, they certainly can. Pools do breed, but you can set how often that happens. You can help maintain diversity. We'll see that as well. So how does the actual session breeding take place? You have two fine-looking, beautiful sessions, A and B, and they wanna bada-bing, bada-boom. How does that work exactly? Well, you draw, it's called single-point crossover.  And the same thing for B prime. You go up to the red line and B and then after the red line and A and you have two new sessions. And it's sort of unintuitive that this would work, right? Because you don't really know how that data, that data's organized. Like if in order to log in, you need like user space, password, carriage return line feed or something like that, it seems like this wouldn't work, right? Because you're kind of recombining bits and pieces of just either what are called basic blocks, which we'll see or random data. And it doesn't seem like this would actually work, but amazingly it really does. So I know, I was kind of amazed the first time I saw it work. I'll show you that. So here's an example of a session mutation.  That's all a mutation. So it also could be like inserting, again, any sort of fuzzing heuristic. It doesn't necessarily always like just insert random data or flip bits, although it could. That's one of the fuzzing heuristics is to just like, if it's of tight binary, just like flip the bits in that token or something. But it might insert long strings or whatever. There's quite a few different things. And you can look through the source code that I have to see what are all the fuzzing heuristics and how do they actually work, you know? So pool crossover is very much like session crossover. And like I said, you'd want this to happen less frequently to maintain diversity. Maybe every nine or 10, 13 generations, something like that. So pools kind of have a chance to evolve a little bit separately to help maintain diversity. It's very similar. Draw a line, copy from one to the other. Pool mutation is basically just randomly adding or deleting a session. You wouldn't want to do that very often either, because if you have a lot of good sessions in there, you don't just want to lose that information. But every once in a while, it's good to insert fresh blood because you might be missing a basic building block. So I'm very much like, just show me a little bit of an example because it's kind of abstract, right? So this is just kind of a simple example. You see generation one starts out with a lot of randomness in it. And you see some basic blocks in there, right? User and pass. And I'll talk about that in a second, where those come from. They come from what's called the seed file, when you originally started.  You know somewhere along the lines, you need to see user and pass show up. Those are called the building blocks, right? And what happens is if you don't give them building blocks, it could, ideally, theoretically, eventually find the string user. It's just going to take a long, long, long time, right? Because it would have to just randomly combine bits to come up with that string. And you see by generation 15, in the first generation it didn't really do much, right? Because it just had the best one maybe with session one. He had user space and then some junk. So maybe he got like halfway through logging in. But not real fit. But by generation 15, it's got like user space, which is a delimiter. Some junk line feed, which is good. And then pass, and then delimiter, and then some more junk. So that would actually be more fit. When you're stalking that, you'll see that he actually progresses along. And that's kind of how they work. Just over time you'll see him getting better and better. And if you look, if you dump the data out of the SQL database and actually look at the data that's in the sessions, sometimes you're amazed, like after 100 or 200 generations of evolution, you're like, you're looking at the most fit one.  That is ugly. And then you look at it, and you're like, no way. There's a carriage return. There's a line feed.  But text-based protocols are pretty forgiving. So as long as the strings and stuff that it needs to see are in there, you can still log in and like actually work. As long as it's, you know, it's got the basic things it's looking for. Binary protocols aren't quite as forgiving. We'll talk about that, too. So what are the actual parameters that you give to GPF to start this off and make this happen? Well, obviously you got to give it like the IP address of the stalker and the IP address of the database and all that kind of good stuff, username and password and the number of sessions you want in each pool and how often you want the mutation to take place in each pool. The session mutation happens, session crossover happens every generation and all the other ones are configurable. Pool mutation rate, session pool crossover rate and all that good stuff. And there's some basic parameters to get you started. I'm not going to go through every parameter and what the optimal setting for each parameter is because it varies a lot based on the application you're testing. So, and there's also the seed file. You see that and I talked about that. And here's an example again of the seed file that I'm talking about. So if you're going to fuzz SMTP with an evolutionary fuzzer, you might as well give it a seed file that looks something like that. It's got like hello and mail from and receipt to and stuff like that because that'll just help the evolution happen much quicker. It'll sort of give it a boost in the direction. But the key with the seed files is to give it really small basic blocks.  But it'll have a hard time like finding weird stuff that probably won't find as many bugs and stuff. It'd be better if you give it get and a space and a slash and a space and an HTTP and a 1.1 like all a separate seed. Smaller building blocks are better in terms of evolutionary fuzzing but just evolution in general genetic algorithms. The smaller the building blocks, the better generally. So on the other side, not on the fuzzer like evolutionary side, on the other side it's sort of the GUI side that's written. It's sort of like the modified version of PyMEI. What do you need to do? Well, again, you have to put, okay, there's two routes you can go here. When you need to track code coverage, you could like sort of sample EIP or you could use what's called an MSR technique. Or you can create a PIDA file.  Give that to PyMEI and give it the normal start-up and shut-down parameters that you have to give any application. That's kind of one of the key things when you're fuzzing is sort of like, you got to figure out if it's a Windows service, it starts and stops different than if it's a user application. All those kind of things are very important to figure out ahead of time. How's this going to work if it crashes? Because you need to tell it, when it goes to restart it, it needs to know how to do that well or it's not going to take place. So you got to have the SQL database set up, all that kind of good stuff.  So it's best to filter those out so you just manually start it up, connect, shut it down, save those hits to the PyME database and that's really, really kind of crucial in terms of speed. Okay, so that was section two. Section one was the background, section two was how, what is it and section three is kind of like, how do we evaluate it and the way to do that in my mind is sort of create a benchmarking toolkit and this bit isn't, this research isn't complete but we'll talk about the tax service coverage and two servers, a tax server and a binary server to see how well it does against both and then we'll figure out is it better to stock on functions or basic blocks and did pools help maintain diversity and if they didn't help as much is there something else, some other measure, something else we can do to make diversity better? In fact, there is. I just recently added a bit to it that another diversity, and we'll see that. So there should be another paper coming on that as well sometimes. Okay, so like I said, this toolkit could help you see how well it do against clear tax server, how well it do against binary server and also one thing that would be a really good thing to do if you want to benchmark and say, hey, is this as good as spike? Is this as good as sully? Is this as good as different thing or is it better, is it worse, whatever? Well, one other idea too is you could insert bugs that are easy to find and bugs that are hard to find and you can fuzz and just sort of evaluate them, right? Okay, so this is how the tax server works. It's basically just a standard text, you know, a very texty type feeling. The server says welcome and then you send it command x, command space, a number, character turn line feed and it says okay, that one's good and then you send it another number, character turn line feed and then it says the sub command's okay and then you send it, calculate character turn line feed and then it gives you basically the answer. So it's just adding two numbers, very simple thing, but it has, the cool thing about it is that I wrote it so it has three paths. You can set it on low, medium or high and that way it gives you the ability to say, okay, if the attack surface is really small, how does this guy work? Or if the attack surface is larger, if there's more paths, if there's more opportunities for diversity, how well does it work? And that's actually really helpful in terms of kind of evaluating something like this. Here's just an aside that I kind of found interesting just kind of for my own research. So this was the text server set on medium, just starting it and shutting it down was 137 basic blocks of 597 basic blocks. So 23% of the code was just literally like double clicking it and clicking the X, right? So that's kind of interesting. You wouldn't think that there's a lot of code wrapped up in that, but there is quite a bit of code just in starting a service and shutting it down. So the actual network code, which is just like the TCP connect, right? That was 15 basic blocks, 3% of the code and then the parsing, which is the only bit of code that actually has any opportunity to have bugs. So this is like the attack surface, right? This is the part that you want to fuzz. There was 94 basic blocks out of 597. So 16% of the code. That's just kind of interesting. The other really interesting thing to me is that a large portion of the code is just unaccounted for. You know, just like libraries that got linked in that never get used, things like that. You can actually look through IDA Pro and figure out what all that code is that just never gets used. It's really interesting to find out that a lot of it is essentially unaccounted for. The C file for this fake text server would basically just be the nine numbers that are possible. And again, on low, only the number one is acceptable and on medium, like one through four is acceptable and on high, one through nine is acceptable. And then a couple other strings, like character turn line feed calculator, come in. And you should be able to shuffle those all together and, you know, insert fuzzy heuristics and fine bugs and do all that kind of stuff. So it had no problem learning the language, figuring out it found the best session. And by best, I mean the correct session, right, where it completed all the legs of the protocol. That's what I call, that's like the base session or the best session. It does not necessarily all of them, and we'll find that. But it didn't actually cover the entire attack surface. Does anybody know why it wouldn't cover the entire attack surface? Well, I think we see that. No, we don't see that on the next slide. We'll see that in a few. But it's basically because there's a lot of corner cases. You know why. Go ahead. Go ahead. Yeah, right. Yeah. So it didn't cover all the parsing code, which is the attack surface was the question. And the reason why, if we go back and look at the protocol, is I'm not really showing you all the protocol here, am I?  What's it going to do? Well, probably it has some sort of error code, right? It's going to say, like, you know, invalid command try again or something like that. So I didn't really show you all the protocol. And that's kind of an important notion because this is, I think, the place where this kind of testing, sort of dynamic learning testing, where you're getting actual feedback from the subject under test as you test. This is where it excels in that if you create a fuzzer, like a spike file or whatever, and it's RFC compliant, it's going to be a good fuzzer, right? It's going to do well because you design the fuzzer based on the RFC, right? So it's going to get good code coverage because you're designing it to get good code coverage. But what happens if there's some bit of the protocol that's not, like, they didn't follow the RFC when they implemented it, right? And that's the whole idea. You're looking for implementation errors.  That kind of a thing, like, you know, you can do pass first instead of user when really you shouldn't be able to do pass. It should, you know, first should be the user, right? So those kind of things, this is going to be particularly good at doing that. And it's also particularly good at finding sort of like places where you need, you know, weird or fuzzy stuff throughout the various legs of the protocol, right? Because typically a spec file is going to, it's going to, like, fuzz the username first. And it's very sequential. And then it's going to be a valid username. And then it's going to fuzz the password next. And then it's going to fuzz whatever else is after that. So it's going to be a very organized, sequential kind of way of fuzzing, where this is kind of craziness, right? It's going to be slamming all over the place, going up with weird sessions and all kinds of stuff like that. So that was the tech server. The binary server is very binary looking, right? It has the first four bytes are the total length of the packet. The second four bytes are the session ID. The second or the next two bytes are the length of the command. And then the final bytes are the actual command that you want to send. And the server response is very much like that. So the question is, will a system like this be able to learn something like this? Because now you have to have a link that has to be correct and cover the whole packet. Else the packet gets rejected sort of as the first check, right? That's kind of how binary protocols work. And sadly, I don't have much testing done on this bit yet, but it shouldn't be too hard. Talked to Charlie Miller, and he did a little bit of testing on one binary protocol, and it worked. So more to come on that. So next question. So which should I stock on? Functions or basic blocks? Does it matter? Well, in my experience, it didn't seem to matter a whole lot as long as the application was sufficiently large. But here, this is an example where Tech Server was being run on low, and it did make a difference because there was only six functions. The best session is six functions long, and it only found four of the six, because the fitness landscape is very flat, right? It makes a breakthrough, but then it's flat, and then it makes a breakthrough. It's not a nice smooth. Where you see the basic blocks did a little better, right? It was a little on the right-hand side, and the graph was a little smoother as the diversity kind of went down, and the fitness kind of went up. And it actually found, in the case of the basic blocks, it found 40 of 37, meaning that 37 is kind of that base case where it did everything right. And it found 40, which means that it must have found one of the error cases as well, right? Because it found more than just the base case where the other one didn't. But if you run Tech Server on medium, you see that actually the functions did just as well and did it quicker, maybe slightly better in this case. So the graph isn't quite as pretty because with basic blocks, things tend to look a little smoother.  But either way, you found all six functions. So there to say, again, just to prove the point, all I'm trying to say with this slide is that if the application is sufficiently large, it's okay to stock on functions, which is kind of important because it speeds up the process enough that you'd rather, in my case, I think I'd rather stock on functions if I can. It's just that much quicker because the difference could be if you have a particularly large application, what does it really mean I'm stocking on functions or basic blocks? But what it really means is each of those are like a breakpoint, right? So if you've got a list of all the functions in the application, when you start the application, you have to set a breakpoint on every one of those spots. In order to get code coverage, you have to hit it, uncheck the breakpoint and keep going, uncheck the breakpoint. So if there's like a hundred functions, if you're hitting, say, 50 to 100 functions versus, you know, 600 to 800 basic blocks, it's going to run slower with basic blocks, right? Because there's a lot more breakpoints that's going to have to be unchecking, essentially. So in the next slides, I just want to show what were the effects of the pools. Did it actually help? And it turns out it did help having pools that help keep diversity up, but it still didn't cover the entire attack surface, right? And I wanted to add another bit to that to help it do even better because the pools help, but this next thing actually helped even more. This is called niching or speciation or some way of rewarding those guys that look different, right? So all the sessions as they're growing, they all want to be like the doctor or something like that, but this guy, he's like different. He wants to be the fireman or something. So we want to kind of reward them. Even if they're not as fit, they're not hitting as many functions, we want to give them some sort of reward. And I'll show you, that's the algorithm. I'll show you just real practically because it's a little mathy looking, not terribly, but a little bit there. So real clear example, session one got 10 hits, session two got seven, session three got five hits. Well, so, and I actually show you the hits, right? A, B, C, D, E, F, G, H, I, J. And what are those letters represent? The letters represent basically an address and memory. That's all they represent. So it might actually have been 08041234 or something like that. And so your initial thought might be that, well, session three doesn't really deserve to do, you know, he doesn't deserve to breed as frequently as session two because he didn't do as good. He only had five hits versus seven. But the thing you see there is that session seven looks like he's going to grow up to be just like his dad, right? He's kind of an immature version of the 10 hits. He's got all the exact same functions. He's headed along that same line where the other guy, he's hit five functions that the other two have never hit before. So you give them a reward. You take the unique hits divided by the total times the best minus one. So he ends up actually at the end because of the fitness boost that he receives. His total fitness is 9.5. It's actually better than the second one, which I think makes sense. He should actually be better in terms of overall fitness because of the fact that he's following a diverse path. So graphically, let's see how this works. So in the first case, we can see that we had only, this is the first one on the upper left. We only had one pool and a whole bunch of sessions. And we see diversity kind of going right down as fitness goes up. And that kind of confirmed my suspicions that as everybody's becoming more and more. Question, go ahead. Yes, I would love to. The first one does, but he's the best. So everything's baselined off the best. He doesn't get any boost for being unique. He's the best just based on the count, on the number of hits. So he hit 10 break points, whether it be functions or basic blocks. Essentially that's what it is. 10 equals 10. So you see he had 10 hits, but he didn't have any unique because he's the best, so we don't count any uniqueness for him. So his overall fitness is still just 10. 15 uniques? Well, overall, yes. So what you would call the total diversity, and that's what we see in the next slide when I talk about total diversity peak. See how it's measured like 80, 87, and 107. The diversity peak is the total count. So the count across the entire pool of diverse hits. And that's what you want to maximize, right? Because say if there's 150 total possibles, you want ideally the total diversity peak hits 150 at some point throughout the evolution. You've covered the entire tech service. That's the goal. That's what we're heading toward. So I didn't actually get that. But what I got is that when I added multiple pools, you see the trend of diversity is kind of up and down. Total diversity, what you're talking about, the count of all diverse hits throughout the pool is the green line on top. And it did better than just with one pool. So multiple pools did seven better overall. And when we added that final metric of boosting or giving a boost to diverse sessions, then you see the total peak was 107. So it helps substantially to give a boost to people who are less fit, hit-wise, but more fit diversity-wise. So that was the evaluation. Sections one, two, and three, and four. Four is the actual results. So just some results. After fuzzing, Golden FTP found lots of bugs. I don't know if you guys have heard of that before or use it, but I wouldn't recommend using it. IAS FTP and SMTP did not find bugs there, but did find some instability that could probably mostly be attributed to the debugger. Wasn't able to identify those to any one session. So the kind of moral of the story here is that really more testing needs to be done. I tested against lots of different applications and lots of different scenarios for any real evaluation to be made of how well it does compared to other fuzzers. So this was just some pictures that I had generated a long time ago when I first started using it. I just liked this because I thought it was really cool when the little bubble popped up in the lower right-hand corner saying that GFTP, a user had logged in. I didn't even know that happened when I just installed GFTP and just started fuzzing it without even really using it. So when the bubble popped up and I saw that a user had logged in, I was like, wow, that's really cool. It worked. So I was very happy and impressed about that. And here's just a picture that I'm showing that it found a bug and it's kind of dumping the registers to the screen and stuff, which it also dumps to a file and also dumps to the database. And you can query the database later. And that's how the overall, you can tell how many total crashes you got, right? Go ahead. Well, you mean does the debugger have to be on the same machine as the process you're debugging? Yeah, kind of. So the answer is yes. I mean, if you wanted to fuzz a Cisco router, then this wouldn't work, right? Because PyDebug isn't ported to Cisco routers. In fact, PyDebug only works right now on Win32. So it needs to be running on a Win application. There's people porting, I know Charlie's porting it to OSX, and there's other people porting it to Linux. But if you're on Windows and if your target is on another Windows box, you can actually debug remotely. You could have the debugger running on a different Windows box, as long as they could talk to each other. You can do PyDebug remotely. So it's kind of, yes, it's kind of an answer to both those. Again, yeah, feel free to ask questions any time throughout. So this is just kind of a showing kind of the coarseness of the landscape initially, how it's not particularly smooth, especially in terms, this is called Best Fitness. And you see where the big spike is where it learns how to log in. And then shortly after it learns how to log in, it starts finding bugs. And this is just the same kind of graph smoothed out. So basically, the moral of those two graphs is that after it learned how to log in, it started finding bugs pretty quickly, shortly thereafter. Which is good to know. This graph is just intended to show the same thing as the earlier graphs. The green line on the lower right-hand graph, you see how it kind of keeps creeping up. It has an overall upward trend, as opposed to the red line is one pool and the diversity kind of went down and up. So it's just kind of to show you again that same thing that you can help maintain diversity with pools. And this one just kind of shows that 10 pools did better in terms of finding unique crashes. So in the first case, with only one pool, I didn't even count. It's like 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 unique crashes that it found, bugs, if you will. But with using 10 pools, it found like 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, I don't know, I'd count them up, but it found more unique crashes. And that's really key, that's really important, and it sort of speaks to the fuzzing, the diverse nature of the fuzzing. So let's see, that kind of went through that pretty fast. Good. Future work, importing it to fuzz files now, that's a big one because file fuzzing is kind of a, it's kind of like the bomb right now in terms of fuzzing, right? Because fuzzing network applications is pretty hard, and fuzzing applications that receive files is still pretty doable. Go ahead. Right, so the question was, how long does it need, does the tool run like minutes, hours, days, specifically how many generations is what you ask, how many generations is it? It's not like terribly long, it is long, maybe like a day or something, it's not like months or anything like that, but it takes a little while, it's a little bit slow because if each session takes, you know, say, I think my default setting is a session should take about 0.6 seconds, and if you have 100 sessions in a pool, then it takes like 100 times 0.6 just to do one generation basically, there's a little overhead at the end of the generation when it does the breeding and stuff, but not terribly long. So, go ahead. Right. Yeah, that's an integrate point. Good to see you again too by the way. Yeah, so the question is how do you know that it's unique bugs you're finding, right? Because that's what you want to find in this sort of thing. You don't want to, at the end of the day, you don't want to say I found 30,000 crashes and then have to go and figure out what all 30,000 of those crashes are. And well, I guess I sort of half solve the issue in that they're bind by exception address basically. So as long as they have a unique exception address, like the exception address, I mean the actual line of code, right? As long as EIP is different for each crash, then they get put into a different bin to see that. I'm sure you can't read the numbers, but in the first case see how the big portion of the graph is purple. The actual number above that's like 31,000. So I found 31,000 of that bug. That's just one bug basically. So it only gets counted one time, right? And that's true of all those, right? Because it's going to keep hitting the same bug over and over and over again. So I actually thought about adding a sort of a fitness minus on terms of sessions where you've already found a crash, it's sort of like penalize them if they keep hitting the same crash and don't let them kind of grow down that road. But it didn't really seem to be a problem because eventually they sort of one that doesn't crash will get further and he becomes more fit. So it kind of ends up doing that by itself over time. You just get a lot of repeated crashes. But as long as you can bin them like this it doesn't really matter. You only count it once. So go ahead. Okay, I didn't quite hear at all. I'm sorry. Can you say yes, the basic building blocks you give it in a seed file are critical? Well, it does both actually. So when you give it a seed file, it doesn't just pick building blocks from there, right? It has like, I forget it's either a 30 or a 60. I don't remember what the percentage is. But there's a percent basically whenever it goes to build a token, it's sort of like okay, I'm going to build a token. Should I grab it from the seed file? Or should I build it randomly? And if it grabs it from the seed file it's not just the piece of ASCII or binary data, but also you can show it in that so it wasn't a perfect explanation of the seed file. But if you actually look at, you can download all this code. If you look at one of the seed files that's in there you'll see that you give it like a type too. So you'll like say the length is four, the type is ASCII and whatever, and here's the data. So it might grab that token and just use it or it might just totally randomly create one, right? Like say I want to create a binary token that's 2,000 bytes long, okay, five minutes. So it kind of does both. I completely answer your question.  It still will evolve, like I mentioned Charlie, he actually tried this, right? He didn't give it a seed file against a binary protocol and it actually did get better over time, right? Because it was flipping bits and stuff like that and eventually found that this sequence of bits did better than this sequence, right? It doesn't really know anyway. It's just a way where, so particularly if it's a small binary protocol maybe you don't want to give it seeds, but for ASCII protocols I think you really want to give it seeds because it's going to take it a long time. Like if it needs to find, before it can really do much fuzzing, if it needs to find user, space, Jared, character, turn, line feed, pass, space, Frank, character, turn, line feed, it's going to take it a long time to find those strings, right? Okay, I think I have time for maybe one or two more questions, so let's see, did I, I didn't really finish the challenges, normal fuzzing challenges, um, yeah, file fuzzing, oh I mentioned that early on that depending on your target is going to make a big difference in how you go, so I think I mentioned most all these anyway, so any more questions? Go ahead. Micro economies? Yeah, I think I've heard of that. Yeah. Right, so the question I think overall is sort of like, have you tried other evolutionary techniques, other genetic algorithm techniques and stuff like that? The answer is I haven't, this is all I've tried, probably wouldn't hurt to try, right? It just takes a long time to implement all that kind of stuff, so I haven't done that yet. Any, go ahead, is there another question over here? I think that's it, I think I will be going to the question room after this if people want to talk more, please follow me there, so thank you very much.


Before anything else come with me. I want to show you something. I want to show you something. That's just you see this You see this That is every single Zelda game that has ever come and all gone with Trinkets collectibles and fun things and the reason for all of that is because Zelda is my favorite franchise of all Time so they can't be a bad Nintendo Direct if it has some Zelda in it I want to start with Zelda, but let's let's get there I want to say I loved this direct because of the end because of everything Zelda that we got even a Zelda game and watch Which I told you I told you just remake the Zelda game and watch and start a little series of this because people would drop 50 bucks each to have them all you know, they will you can start re-releasing these that's easy money I thought it was gonna be part of the 35th anniversary, but disappointingly apparently they have Nothing planned for that. They even said sorry. We have nothing well We don't have any campaigns or other Nintendo switch games planned We've been working on this game and watch system as a special item to help mark the occasion a little game and watch I'm happy with that. Was it the best Nintendo Direct? No, it was not the best that said Nintendo's was good and it was definitely the second best event next to that well end of over digital because it was fun Okay, but we're not here to talk about other crappy events. We're here to talk about Nintendo I want to go through everything we saw and there were a ton of really cool surprises and announcements That just wasn't everything I was hoping for and that's my bad Also, a lot of the things revealed weren't things. I'm super interested in but I know that people watching were freaking out We streamed to 30,000 people watching at home And I can't say thank you enough if you were one of those 30,000 and there's a good chance one of you were Thanks for spending the time with me. Okay. Let's take a look at Nintendo's direct I want to break down everything we saw and then at the end I want to dive even more into breath of the wild because I've poured through that gameplay trailer and I've noticed things So the direct stuff Let's take a quick break and talk about one of our favorite sponsors Raycon, what's that? You mean you haven't purchased your own pair of raycons yet by going to buy raycon.com forward slash beat them up What are you waiting for? I've already done it twice Actually, the other pair was for Kim because she wanted her own pair Even I got my own pair Yeah, I mean everyone wants raycon. Well, yeah everyone wants raycon I said that already Raycons have great sound more bass are comfortable fit. They're noise isolating They have six hours of playtime which can be extended by charging them back up in the little carrying case they come with Not to mention they come in a wide variety of fun colors and patterns. I use mine while I'm working out I use mine while I'm cooking and while I'm pooping And I even use my raycons while I was setting up to make this video so that I could move all the lights and cameras around I've just been using my raycons to listen to a podcast the entire time I've been Setting up to film talking about the raycons So one last time click that link below and get yourself 15% off I'm done. I'm done. Yeah, how about you lock keep it down? I'd appreciate it I knew immediately that this event was going to end with zelda Because it started with zelda or at least a zelda fake out And I don't think nintendo would have the gusto to make it look like we're about to see something zelda related And then flip it unless something was coming. I mean that would just be too mean We saw gannondorf being carried by Tekken man. I have never played Tekken And here's where we go wrong I've never played Tekken. I really have no input here. I I couldn't be more well like if there's overwhelmed and there's underwhelmed. I'm not underwhelmed. I'm just I'm just well like I like smash reveals are the one of the most exciting things about the event So it always kind of stings when it's a reveal that you personally don't really care about and that's fair I mean not everyone's going to care about every character. I loved banjo and kanzui I was overwhelmed by minecraft steve it happens. It's a crapshoot But that's just me and I'm thinking out loud and I'm not going to stand here and lie to you guys If I'm not excited about something I'm gonna be honest life is strange all of it. I think and possibly a new cartoon colors Titled life is strange game are finding their way to switch I really loved the first life is strange like adored it And I've tried the others and for some reason they just haven't hit with me the same way So for me this announcement again wasn't like edge of my seat I'm glad more people are going to get to enjoy these games because of this super monkey ball Is a game I've never played one of these either except for at the local arcade when we were allowed to go to arcades There's an actual monkey ball arcade machine and kim and I love that thing It's got like a giant ball on it. You try and get top points It's actually really hard and then you get tickets That's the only monkey ball I played the next one honestly just flat out frustrated me like I'm gonna be honest I am a little maybe too peeved about this next one mario party superstar mix-up mash or something it's a new Mario party But like it's a collection of favorites So it's five maps from older games that people loved But it just seems weird to me because the first mario party was lacking so much content It only had five maps and they got repetitive really quickly because all of them literally all of them were just Around around about like there was nothing really unique about those maps at all And then recently that game got some cool dlc and I was hoping that when I saw this Oh, they're finally adding new maps to that game But no, it's another game that only has five maps It has been four years since metro prime four was first announced to be here on the fourth year of the fourth game I really thought we would get if anything just a little bit of cinematic trailer even like a 10 second splash screen Maybe a coming 2023 something just something That tells me this game is moving along that something is happening with it But instead we were just told no nothing and Bayonetta was nowhere to be seen as well another game really announced around the same time We got to be coming up on that four years to instead. There's a new new Metroid game It's one of those 2d traditional style games like we had fairly recently on the 3ds, which that game was amazing So i'm actually not upset about this at all like i'm excited It stings a bit watching it because it's kind of like it's not the Metroid game I wanted but it's not Metroid Federation force. So that's nice Yeah, I mean i'm gonna i'm gonna like it It's a new Metroid and it comes out in october a couple of days before my birthday. Yeah, i'm excited for that.  I love seeing third party support another look at mario golf because it comes out pretty soon So that makes sense. Oh, I did like this next one. We finally got some gameplay of monster hunter stories too I'm super excited for this game It's more of I don't want to say watered down But it takes the monster hunter elements and makes it easier to consume But at the same time it's because it's a completely different game and there's more elements to it The battles are rpg battles you get eggs and you can hatch little monsties You can't be mad at warrior wear I really didn't think this was going to happen today new warrior wear comes september 10th It's not a game. I'd play on my own But a game to play with him or a game to play online that could be really fun Shin Megami tensi 5 is underrated I feel like as far as a nintendo switch release I think it's been made pop probably because everyone just wants persona 5 But that's a game that's already been out and I know i'm the same way I just want persona 5 2 but this is a brand new game for switch and it's it's very similar to persona the universe is crossover I'm not really sure how there'll be a super fan in the comment section that will tell me it happens every time I and I you fight my comment section can back me up here I've been saying for years that fatal frame should come to switch at this point and I've said this too It's the last Wii U game. They could bring over that anyone would care about I've wanted this for a long time. It makes sense. Uh, it's a very creepy Wii U game All right, then we got a couple things like doc for doom. Tony hawk is still coming to switch, which is great But then strange brigade I'm gonna weirdly focus on this one because I very recently got that on game pass and played it through with my friend And we had a ton of fun It's a really great co-op multiplayer game and I highly recommend checking it out It is on game pass for free Mario and rabbits got a quick mention And then they moved on to advanced wars one and two remake. These were beloved game boy advanced games I had no idea what I was watching while I was looking at it though It's been a long time since I played one of those games or even seen or saw one of those games And I was looking at this as one of the final announcements and I honestly was baffled. I was just How was this one of the last things we're seeing? I honestly thought and this is gonna sound really mean But I thought it was just some budget Shovelware looking game I was like, what is happening? But then as soon as advanced wars came up. I was like, oh Oh, that was advanced wars. I didn't even notice that was advanced wars. Okay. That's cool. That's cute That's cute people like those and I gotta be honest rewatching this event There are a lot of fun things smatted in around here or there and again comparing it to the other events We had during e3. This is leagues above and better than anything else We saw other than xbox, but up to this point. I there was nothing really like Super excited worthy. I mean there were cool things and it's a good event. Here's an example I'm not a nintendo ambassador anymore, but the other ambassadors got sent presents from nintendo They were these like glass cases that you were supposed to break in case of extreme excitement And it was for when they were reacting to the direct if they got really excited They were supposed to break the glass now. I didn't get sent one obviously But my point is up until now I had no reason to break that glass They were like I might tippy tap on the glass and see if I can't loosen it up a little bit Just in case something happens There might have been a little bit of me getting Really excited at the hype and then the glass was like shaking a little bit But there was nothing to make me smash that glass. So when we got to this Hyrule Warriors DLC and they said it was the last thing today I actually thought I was about to have another seizure. This next segment will be our last announcement of the day Please take a look If that had been it, I don't know I don't even know if I would have made a video because I don't think I'd be would have been able to say anything nice Thankfully it wasn't there was a few more zelda things Skyward Sword got another plug makes sense comes out in a month Then we saw the zelda game and watch which I already talked about But I didn't mention that unlike the mario one it actually comes jam-packed with games They've crushed the first two original nes zelda's and link's awakening under this thing as well as an actual game and watch game But that's really neat. It's nice that I'll actually might have a reason to open this one and play it I won't but I could this is the sequel to the legend. Oh my god. Yes Development has been steadily progressing. Yes for today. We've prepared some new footage to show a bit more of the game Please take a look. Okay. Okay. Okay, baby. Let's go. Let's go The sky is this like skyward sword? What is happening? Why is that eggs? Why is his hair so long and fully? I'm still so hype. So the trailer starts with link's arm getting like like all sucked into like goo I don't know. I don't know the calamity the calamity took his arm I don't know but judging by the rest of the trailer It looks like his arm has become the sheikah slate essentially for this game Then it hard cuts to a skyward sword-esque vision of him falling from the sky I love this. It's kind of weird because I thought maybe they would take a more major as mask Root and maybe they still will where it goes a lot darker and we're in like this twisted version of the realm But it actually looks like ironically or coincidentally with skyward sword coming out in about a month They've taken it to the skies and instead it's very light airy breezy And it has that similar breath of the wild fun feeling But yeah link plummets from the sky we see chunks of hyrule floating above A beautiful shot of him running through the field his hair flowing in the wind It does look like it's had some visual improvements, but a very similar looking game to the last one I like this little tease towards some new enemy types. We've got one of the rock Mon's god Why don't I know my zelda names? But he has one of the goblin camps like perched on top of him. So it's like merged the two threads I like that. Oh, and I'm now getting this but we see another shot Of link's arm glowing and then it immediately cuts to his new abilities So I guess unlike the first game where you had to go around and unlock vashikis light abilities now That's tied into his arm and he has new abilities So rather than just a stasis just freezing something in place It looks like he has a reverse time thing where he sends this spiky ball back up the path that it came down Next it looks like he has this weird dragon arm thing that's breathing fire There's a puddle on the ground that kind of gets like sucked up and then link shoots into one of the sky buildings And that's where he is now I don't know if that'll be ability You can just pull the trigger on or maybe you have to find a certain spot But that's your way of teleporting up to the platforms above It would be great if you could do it anywhere at any time, but it might just be a teleportation thing I don't know then we get a beautiful shot of the whole of hyrule and then it cuts to the castle getting like Up into the sky. Oh and a release date 2022, which is next year probably going to be late next year And knowing nintendo probably going to be 2023, but hey 2022. Okay. All that is hype. I love the new abilities I can't wait to mess around with them. I have so many questions Like do we have the old abilities or is it just the new ones this time around my biggest thing my biggest? Oh, I wish we could have seen this or had this explored more Is one of my and a lot of people's biggest complaint from the first game if there was a complaint to be had Was just kind of like Emptiness there wasn't much around as far as other npcs civilizations villages things to do places to go that kind of stuff I mean there was a massive map But it was a lot of open fields and mountains and terrain and the fun you had was sort of interacting with all of that Rather than the people that actually lived there and for all the Expansive field we saw in the trailer for going up in the sky and looking around there was still kind of nothing Like even in this last shot you see these platforms. There was nothing on the platforms. It's cool that they're there I mean it's cool that we're in the sky now But why what what are we gonna do up there? That's my biggest Question right now like the abilities explain themselves I love getting to see those and that'll be fun to mess around with but the whole other element to this is you're in the sky now But why now like why why that really like I don't I hope this is coming across the way I want it to Why I guess what I'm saying is civilizations in the sky would be really cool things to do in the sky people Up there. I mean games to play. I don't know. Yeah, I'm really excited and next year can't come soon enough So the direct as a whole was good. It was a good direct condensed into itself. It was good and I had a good time watching it My god, baby this gameplay what the heck what is happening?  I want more. Yeah Next year no freaking bug when though when If you like this video and you had a good time watching it or you just like the nintendo direct hit the like button Subscribe follow me on my social medias everywhere. There'll be links down below. Check out the sponsor while you're down there I love you all I'm exhausted. Bye


What's good, George Boyros back out again with another video. So we're going to check out how these 20 WWE wrestlers got their names by none other than tap out corner. This should be a good video. I'm interested to see how some of these wrestlers got their popular name. And then I like these type of videos, these videos where they give like some information, some tidbits of like certain stuff that you may not have known about a certain wrestler. And a lot of these I definitely probably did not know how the name came up for a particular wrestler. So I'm going to check this out, appreciate all the love and support you guys have shown on the channel. Let's get right into this one. Roman Reigns actually stole part of his name from another wrestler. When Reigns signed with WWE, he knew exactly what he wanted to be called, Roman. However, WWE told him he needed a last name. There was another WWE development wrestler named Calvin Reigns who had just been released from his contract. This gave Roman the idea to call himself Roman Reigns. Before he made it official, Corey Greaves suggested that Roman spell Reigns R-E-I-G-N-S like how King Reigns over his kingdom. Roman took the advice and never looked back. The Undertaker got his name was pretty unusual. After being told by his employer that nobody would pay to see him wrestle, mean Mark Callis left WCW and got in contact with WWE. The future dead man had a meeting with the company, but they didn't agree to hire him. Then a few months later, the phenom got a call and the person on the other end asked, is this the Undertaker? The dead man had no idea what this person was talking about. But he said yes, which was the correct answer. Vincent Van later told the phenom that he had the idea for the Undertaker character for a while, but couldn't find the right person to betray it. It's safe to say that mean Mark Callis was the right guy. No, that's and I definitely did hear about that story. And it's just crazy to think that Vince came up with one of the greatest wrestling characters of all time, which is once again, I can give credit to Vince coming up with some crazy and amazing things. But that was decades ago. It's not too often now that we get some great creative ideas. Granny, I don't know who knows. Maybe he has come up with some great stuff. We just don't know it, you know. But technically, the Undertaker's original name was Kane the Undertaker. Wow. However, it quickly got shortened, but the name Kane would pop back. That's very interesting. Over six years after the Undertaker debuted, WWE needed an opponent forum. This led to a storyline where the Undertaker discovered he had a brother he thought was dead, a man named Glenn Jacobs, who had wrestled in WWE under a few different names, was given the role. Originally, WWE wanted to call the Undertaker's brother Inferno, since the story was that the brother had been burned in a fire. Jacobs thought it sounded too cartoony and a WWE writer named Bruce Prichard agreed. They came up with the name Kane, which was inspired by the story of brothers Kane and Abel from the Bible. Did you know there are actually two Rey Mysterios? Long before coming to WWE, the Masters 619 went by a few different names. These included Labritcha Verde, which was the Green Lizard and Calibri, or Hummingbird. However, after proving himself, the masked luchador was given the name Rey Mysterio Jr. by his uncle, who was the original Rey Mysterio. Rey kept this name until he debuted in WWE, and they dropped the junior and simply called him Rey Mysterio. That's dope. Also, in case you're wondering, Rey Mysterio means Mystery King. Before he can make his WWE debut and try to come up with a ring name, he wanted to just use his real name Adam Popeland, but at the time, WWE wasn't allowing real names. While driving through Albany, New York, the Rey and our superstar was listening to a local radio station called The Edge. He thought the name Edge sounded good, and the rest is history. Before she was called Sasha Banks, she was known as Mercedes KV. However, once she signed with WWE, Mercedes had to come up with a new name. She wrote down Sasha with a different last name and Banks with a different first name. However, it was WWE coach Sarah Del Rey that told the boss to combine the two. Thus, Sasha Banks was born.  Who's Jericho? That's totally different. That's Sasha Banks starring in Wrestle Me Now, Wrestle Me Later, Tonight Edition 1. Different name in mind. Before he made his wrestling debut, why did he call himself Jack Action?  The future WWE star agreed and remembered that the band, Halloween, had an album called Walls of Jericho. The young wrestler decided to take his real first name, Chris, and combine it with Jericho. Stone Cold Steve Austin actually got his name in two parts. Early in his career, Austin used his real name, Steve Williams. However, when he went to work for a company called USWA, he was told he couldn't be called Steve Williams because there was already a wrestler with that name. The guy running the show, who was actually Zeb Coulter, ended up deciding that Steve Williams' new name would be Steve Austin, because Steve was from Austin, Texas. Years later, the Texas Rally Snake joined WWE and was given a new name, the Ringmaster. Austin hated the character and wanted to do something more serious. After watching a documentary about a serial killer named Richard Kuklinski, Steve Austin pitched a sadistic, cold-blooded character to WWE. The company agreed but wanted Austin's new name to either be Ice Dagger, Autofan Ruthless, or Fane McFrost.  Who came up with that? Autofan Ruthless Ice Dagger, which don't sound good either, but Fane McFrost? Imagine if Stone Cold Steve Austin's name was Fane McFrost. Fane McFrost! Fane McFrost! Can you imagine? I can't even imagine J.R. saying that. What?  Those names were terrible, so Steve had to figure out something else. While thinking about it, Austin's wife at the time gave him a cup of hot tea. She then said, Go ahead and drink your tea before it goes Stone Cold. And just like that, Stone Cold Steve Austin was born. Awesome, man. The origins and meaning of CM Punk's name are difficult to find, since Punk himself will make stuff up. What's the CM stand for? Turtles Montgomery. CM Montgomery burns? However, when CM Punk first started wrestling, he got put into a tag team called the Chick Magnets. Chick Magnet didn't really fit, as the strange wrestler saw himself more as a punk. To get around it, Chick Magnet was shortened to CM, and he added on Punk. As Punk's career grew, the CM eventually started meaning Chicago Made, since Chicago was the city where Punk was born and raised. However, in 2018, Punk was in a $1 million defamation lawsuit with a WWE doctor. Under oath, Punk was asked what CM meant. Since he was in a position where he couldn't joke or lie, the former WWE champion said that CM officially stands for Chick Magnet. Before joining WWE, Bayley was called Davina Rose. However, WWE wouldn't let her use it, and she had to come up with a new name. Bayley threw out a number of ideas, and WWE ended up giving her three choices. One was Davina, but without the N, so it was Davia. Since that wasn't a real name, she ended up picking one of the other options, Bayley. Bayley suggested the name be spelled with B-A-Y at the beginning, since she is from the San Francisco Bay Area. Why is Triple H called Triple H? In 1985, the game joined WWE after leaving the rival of WCW. WWE was working on a new name for Triple H, and they came up with Reginald Dupont Helmsley. Triple H didn't like it and came up with some other names. After reviewing the game's suggestions, WWE said it called Triple H, Hunter first Helmsley. His character was a Stabia Litas, and the name worked well for about two years. In 1987, Helmsley joined Shawn Michaels to form D Generation X, a gang of edgy rebels. The name Hunter first Helmsley no longer fit, so WWE decided to shorten the game's name to Triple H, which is also the nickname that Helmsley's friends and co-workers have been calling him backstage. The story of how Hulk Hogan got his name is really fascinating. During the 1970s, Hulk Hogan was wrestling as Terry Boulder. Hogan appeared on a talk show where he sat next to Lou Ferrigno, an actor most well known for playing the Incredible Hulk. Ironically, Hogan was actually bigger than Ferrigno. This caused Hogan to start calling himself Terry the Hulk Boulder. Later, the Hulkster joined WWE, and the owner at the time, Vince McMahon Sr., won Hogan to have an Irish name. They decided to give him the surname of Hogan and make Hulk his first name, likely due to alliteration. Ironically, Hogan means youth, but the Hulkster was already losing hair when he made his WWE debut. Dolph Ziggler was almost called David Diggler. That was the name WWE came up for him, but the showoff didn't feel at work. The company gave him 15 minutes to come with a new name. However, it had to have a D. Ziggler thought of his great-grandfather, who was named Rudolph, so Ziggler shortened it to just Dolph. He couldn't think of a...  Damn! Last name, but asked that not be Diggler. The creative team then came back with the name Dolph Ziggler. The showoff still thought it sounded too cartoonish and even went to Vince McMahon to try and change it. The boss wouldn't budge, and said it was different and people wouldn't forget it. Nah, he... This is another situation where Vince is correct. Dolph Ziggler works. You wouldn't think. And initially hearing it, but it's something that people instantly remember that name. So it does work. The Miz came up with his WWE name before even stepping foot in a wrestling ring. In 2002, the illustrator appeared on a reality TV show called The Real World. There, he introduced an alter ego called The Miz. You think that c**t worked?  It doesn't f**k her up and The Miz does not break! Do you understand me? Stop! As you can see, The Miz loved wrestling, so he eventually tried to join Diff the Bee and brought along his character. This is the true story of when The Miz came to the million dollar tough enough challenge! Once he made his debut, Miz would still be called by his real name, Mike Bizanin, for a short while, but that stopped not long after he started wrestling. You might think that Killman Owen's name is a tribute to Owen Hart, and it is, and it isn't. Like many wrestlers, once K.O. got to Diff the Bee, he had to come up with an original name. He kept his real first name, Kevin, but the name Owens was inspired by his son. Kevin Owen's son is named Owen, and Owen was named after Owen Hart. We all know what RBD stands for. But where did it come from? Long before RBD wrestling in Diff the Bee or ECW, he was performing at smaller shows, a wrestling promoter so that Rob looked like the actor Jean-Claude Van Damme. Damn, he does! I always figured that's where he got it from, but oh my, yo, he does a little bit. He resembles him a little bit for sure. The actor Jean-Claude Van Damme, the promoter then told the young wrestler to start telling people that he was Jean-Claude's cousin. So Rob took his real first name and added Van Damme to the end and never looked bad. That's crazy. AJ Styles did not like his name. When he played football as a kid, everyone called him AJ because his real name is Allen Jones. However, when he became a wrestler, he was put into a tag team with a guy named Steel. The idea then got pitched to give AJ the last name Styles, so the team would be Steel and Styles. AJ disliked it because Styles didn't sound cool, but as AJ Styles began to establish his career, he grew to love it. Unlike a lot of wrestlers, Kofi Kingston basically got to keep his name when he went to Deb the Bee. Before making his wrestling debut, Kofi decided to have a Jamaican character. He took his real name, Kofi Nihaji, and he used the capital of Jamaica, Kingston, as his surname. The name was near perfect and when Deb the Bee signed him, they simply took out Nihaji and shortened the name. How R-Truth got his name is a bit more complex. In his early career, Truth would ban a few names, including K-Quick and K-Crush, depending on what company he was wrestling for. In 2002, while Truth was working for T- Y'all see that old footage? It looks like R-Truth has not aged at all. Well, that is insane. Insane. He is the founder of Youth. I'm telling you. In A or Impact Wrestling, he got on the mic and said that he had been held back because of the people above him. He ended with this. I'm not an ancient black man. Soon after that, the wrestling rapper started calling himself Ron The Truth Killings. After leaving TNA, Ron Killings went to Deb the Bee. In an interview, Truth said he was backstage with Vince and Stephanie McMahon. They wanted his new name to start with R, elected because of Truth's real first name, Ron. Stephanie then blurred out R-Truth in that celibate. While nothing was confirmed, it's hard to imagine that Stephanie wasn't inspired by Ron The Truth Killings. As a kid, Ryback, whose real name was Ryan Reeves, got the nickname Silverback after a friend said he looked like a gorilla. The big guy liked it and kept using the name. He eventually got a tryout with WWE as a competitor and tough enough and used the nickname there. However, Ryback would not win a WWE contract and later, Mark Henry started using the nickname Silverback. So once the human wrecking ball made it back to WWE, he needed a new name. He decided to combine his first name, Ryan and Silverback, which created Ryback. WWE accidentally revealed how Jeff Hardy did his jump off the raw stage. To see it, watch- Aw, man. That was interesting. Very, very, very interesting. I love these type of videos where you don't, you don't really get the full backstory of, or, you know, well, not, well, you do get like the backstory. You get the backstory of certain wrestling tidbits or facts that you may not have known. I definitely did not know about the whole Kevin Owen situation. I know he used to be called Kevin Steen and on the independent scene, but I didn't know the Owens part came from his son. Since his son is named Owens as well, or what not, in his tribute to Owen Hart as well. So I think that's awesome. That is dope, man. Comment down below. Let me know some other wrestler names. Like, if you know how they got them, or, you know, if you know the information on how they got some of their names, or if you didn't know some of these in this video, let me know which ones you didn't know about. If I appreciate all love and support, you guys shown on channel Road to 150K, and I'm still getting this video. YouTube Wrestling Champion World, I appreciate you all kicking me. See you next time.


Oh, Nelly. I'm gonna stay back here cause I'm gonna accept otherwise. I can open in 3, 2, 1. I'm buffed, I buffed, I buffed, I'm in. Nice. I got roguhite. Okay, don't press. Cleaning him, cleaning him. There we go. Couldn't leap. This shield is like 10 dispels deep, man. Stop priest if you can. I'll burst. Okay, I can't really. Double cheap shots. Up here, yeah. That's late. I protected the field ward at least. Yeah, I watched. They're both. Nice fair. And I still have my field ward. They're a really good spot. Rogue trinket. Gage me. I'm fed. He's out. No, no, no. Already used. Okay. I'm gonna get fair ward. No, I didn't have that. I'm fair, I'm fair. I poked his vanish. Next fair you innovate me, okay? Care for it. Care for it. No, what the hell? I sat me. There we go. It's my badder. Actually, I don't think it's that bad. It just gives me my fair back. Keep shielding me. Should I take this one off? Yeah, you can tell him we're just dead. Fuck. Fair both. Gouch. You can innovate me. Nice, nice, nice. I'm in gouch. Gonna innovate you now. Gonna clone innovation. I'm lying. I'm just gonna delay him. As much as I can. He's gonna fair watch out. He's not doing it. It's actually nice. Clicked on. Yeah. Just keep road if you can. Gouch. Got both. Got both. Really nice. Blimey, blimey. I saw you. He's full dots though. He banished at me. You can die, okay? You can die. Just live, live, live, live, live. He has dance for you, okay? 10 seconds for my PS.   Trink it, trink it, trink it. Take the priest. Nice. He dead. He's open. Yeah, sorry. I'm here. Yes, we just go warlock. Yep. Oh, I could have died. This is my bad. Picked it. Yeah, we're good. I'm gonna clone lock here. Go for pet for a bit. You can leave. This is fine. Yeah, fine. Cloning the pet low. I'm gonna do big damage. Can I get pet? I got you. Big damage. Kappa. Coming to you now. Let's see. Coil. Penetrate now. He has DR on me. He doesn't have DR on me. He just did it. DR on me for a couple of seconds. I'm gonna drink it. Quartet. Yep. He wants to shadow fury. Maybe you can... Oh, he missed it, missed it. Quick, quick, nice. I'm getting drinks. He's huge. Cloning the pet. Cloning the pet. I'm free. I just bowed you. I got one burn early. Okay, you again. Sacred's off. He might shadow fury me. I'm gonna MC this warlock. Yep. I should get it. Oh, he said he's won. I thought I'd position him well. I drink this stuff. Okay, okay. Lightning's there. Where am I? Got you. He's gonna shadow fury. Watch out. I'm seeing him. I'll get him off that again. Yep.  I'm gonna drink. I'm gonna drink. He's gonna shadow fury me, okay? Almost one shot pet. Nice. Still drinking, okay? Yep. I'm a Berserk Christian. Cloning warlock. I'm going back to trying to drink. Palad's man is not amazing. He doesn't have play right now. 15 seconds. I'm gonna drink again. Just watch yourself. Yep. Pet's coming for the juice. Palad's drinking. He has to stop. He has to stop. You got it. You got it. Nicely. Play him. Play up. Hold on. I won't get the play. Okay. I can fit Palad. Almost one shot. Can you stop? No, you won't get it. Oh, it's Phil. You got Phil, aren't you? Yeah, yeah, yeah. He wants shadow fury. Watch out. I'm just aligning. Missed, yes. I faked it. Hold on me. Warlock. He shouldn't get the fit. He shouldn't get the fit. Nice claim. Do you want the innovate? No, yeah. I'm gonna MC lock him. Maybe I'll get it. I got it. I got it. Jumping him off.  I'm gonna go on him. It's not bad, I think. Okay. I didn't even finish my nuke, dude.  Oh, Nelly. I think so. I don't know if he's streaming right now, though. I appreciate it. Yup. I'm rushing for combat. Okay. Three, two, one. Go. Saps. Exactly, man. I'm gouged. I preserved the shield. It's good. And I got refueled. I'm in fear right now, but it should be fine. Evasion up. Kidding me. He's gone. He's not bought the shield. I'm gonna clear off the back. Okay, nice. I'm gouged. It's building red. Canons. Nice claim, dude. Maybe we can kill him. You're kidding me? I'll try, I'll try. I don't have rockets here. Nice. That was a really clean game. Yeah, GG.