Hello everyone and welcome back to you for the Minions episode 41, the weekly show where we track some of the games that can possibly replace Paragon somewhat, replicate the experience at least. I am your host, TheMangus, joining me as always is the illustrious Mandy Mal. How you doin' Mandy? I am just as illustrious as ever. I'm doing good, I'm really excited to introduce our guest host this week, Mr. Foolish Blood Hunter. How are you? Hello. Hello. Hello. So we usually start by asking our guest host their favorite hero to play in Paragon, how they got started with Paragon, all that kind of stuff, so do tell us all the juicy details. Well first my last favorite hero is Polari and maybe Greystone, but my history is that I started on PS4 when it first came out, played legacy, played all this, played the beta and then the early access and open whatever, but I like legacy a lot better. I started, you know, getting to know how to play, it's my first MOBA.  I had to learn it all on my own, play solo, but without it. Yeah, there's a lot of people's first MOBA in Paragon. Yeah, it was my first as well. And yeah, we've discussed that before, there needs to be better training in these games.   Just stand there and tell me if anybody's coming. I've never had that, but that's funny. Run the jungle and sit in the corner. But let's go move on to the news and updates for this week starting with ethereal. Before we get into ethereal, I want to let you guys know that their new website is up and running. This was announced after we recorded, so we'll talk more about it next week. Be sure to check it out to find out all sorts of new information about ethereal like class passives, a tropos and a ton of lore. They dropped some more voice lines whenever they played the Lea voice lines last week. So I'm going to go ahead and play those this week, but we're also going to go ahead and take a look at Kallia's voice lines, which we will do right now. When I awoke, my lands have been desolated, each pure heart is a seed that will blossom anew. The forest is a strange way of revealing your true intentions, anger, joy, sadness. It is all laid bare for me to see. Alright cool, really loving their voice lines, really digging them, really digging the voice acting especially. Mandy, what did you think of Kallia's voice lines? She sounded like, again, a perfect fit of voice to character. The voice actress did incredible, very childlike and precocious, and I especially liked the ooh, a butterfly and hi Marina, it's your best friend.  There we go. See I listen to you sometimes, but yeah, she sounded good, she sounded amazing. What would you think? Oh, she made me feel young, young again, like yay, I remember this, this was the fun days when you're young, you're 16, you don't know anything, you're just like in an airhead sometimes, but it's okay, I'm just young. I like to look over, I like her look since I hadn't seen it before, but it's just, I was just listening earlier to some of the voice lines and I was like, these are kind of cute. Yeah. You know, it's just, I'm not really into young girls, but I'm like, it's just to remember that you were that age again and exciting for me. Yeah, well that's one of the things too that I wanted to bring up, like she replaced the Kaika, well I shouldn't say that, she was originally slated for the original 14 that they want to have on their roster for their Alpha, but they had to completely rework her and that's when they brought Chief of Kaika in. They were going to go and reverse that, Chief of Kaika is going out, Kali is coming back into the original 14 and if you've ever seen Kali's original artwork, she was basically in her underwear and I think they're really trying to avoid portraying her as a loli. We were discussing that term earlier, which Mandy had never heard before. So they put some more clothes on her, I think she looks a lot better with more clothes and it's more fitting for her voice because she does sound like, I mean she's supposed to be thousands of years old, but she does look and sound like a precocious, I think was a great word for it Mandy, just a precocious child that's on an adventure, not like in a life or death situation.  So I think that was a good call on their part to completely rework her character model to make it fit a little better with her voice lines and her attitude. Yeah, I agree 100%. That was good either way. Either put more clothes on to her or make her sound older. Or make her sound older. It was like you could do either. I don't feel dirty when I play her. Yes, yes, that's that sums up exactly what I was looking for. They had to rework her so you didn't feel dirty when you were playing her. So well said, well put, well put. And then the other thing with them, the community Q and A's, I've mentioned that before that they just kind of come out of nowhere, but they do have a method for their madness. I didn't I wasn't aware of that this till this week. But whenever they do drop voice lines, that's when they're going to be doing these Q and A's where everybody can go in and interact with the developers and the development team. So yeah, if you're so say if you're sub sub sub if you're subscribed to their YouTube channel, whenever you get a notification, just go ahead head over to their Discord and there will probably be in there. And you can ask them a question, drop them a compliment. Just say, hey, whatever you want to do, you can just go ahead and interact with the developers over at Undying Games. What do you guys think? What do you guys think of that? I think it's so cool. I really used to enjoy the community corners that Epic would put out for Paragon. And I just think this is a way better version of that because it's immediate one on one. So that's like just so cool that they would take the time out to do that. And it shows that they care what the community is thinking and what they have to say and they want to answer as many questions as they can. So I think it's really cool. What's got. I'm with her. I like the community corners on Paragon. It was just something fun for somebody that was new.  Sometimes I'd watch them twice. What were they saying about this? It actually the only thing I was able to learn. And then when they finally introduced a little bit of onboarding, you know, teaching me a little. OK, but I learned that part. Give me some more dance. Because I it took me months to learn like, oh, you're supposed to hit these at the end at the last. OK, last thing. Now I was like, God, that's what last hitting. Yeah, you hear the words, but you don't really equate it to what you're supposed to be doing unless somebody tells you that's what you're supposed to be doing. Yeah, people used to yell at me all the time, push this lane or, you know, freeze that lane. I don't know what we mean.  I can't freeze this lane. Yeah, right. So almost what I said one time. But then I finally I learned. And then when I learned they it was. Model lift came out. I was like, now I got to learn different timings. Yes. It's like, oh, wait, this is actually easier now. And then I just got tired of them changing. I want these new ones to exceed because I want to play that third person mobig. But I also just got tired of them changing their core formula on paragons. Like you went from this map, people let and then you went to a smaller that all this and you went this and then you did this. And then they changed the card system the third time. I was like, give it a chance, but no. That's what broke the new car system. Yeah, that's why I was wanting these people. I was actually digging some of the core stuff and they even predecessor. But my real favorite is serial. I've always liked that since the beginning. Yeah, it's a really cool step up from from a typical mobile. Way better in smite. And it's a we've definitely seen the fruits of interacting with the community. Definitely from Omeda as well, which we're going to head into now. Smokey's always been very open and willing to field questions from the community during his streams. And it really shows in the final product. They really they take a lot of that into consideration. They don't let the community make the final decision and they make that final decision themselves. But they do allow the community to influence those decisions, which is always good to see because if you like, if you're if you're if you're creating a product, you don't want to committee tell it, you know, creating that product. You want one person to have the final say on how that product gets created. So it's good to have that community feedback, but you do need somebody to keep directing and steering steering the the game towards its final product. So. But for for Omeda for their for their news, they're going to start using the Amazon GameLift. I think it's called for servers. I'm not sure if that's the correct name for it. But anyway, they're going to be using Amazon to host their servers whenever the alpha comes back online, which is great news because during the alpha gameplay, they would often have to bring the servers down when a lot of people wanted to play simply because none of them were going to be there to monitor the servers. With this way, they could just walk away and it takes another thing off their plate. They can stay more focused on bug fixing and and testing the game themselves and not have to worry about servers. So I think that's really great news, far more important than it may look on the surface. So good for them. But did you get you get a chance to take a look at the at the alpha and all that for predecessor? Basically, only when y'all showed it. Yeah, maybe some from British. I watched Brick play it a little bit for a while. I remember, I think, weren't you streaming it once? I don't remember. Was that you streaming it or? I did stream it a little bit, which I wish I would not have. I think when you're both playing it, I think I saw a little bit of each one of them. Yeah. But I kind of liked it. I was like, well, it looks like the timing is way off. That first one I saw when they showed it. They're most like, well, y'all shouldn't have brought this out. You should know how people are. Yeah. After they started calling Anthem up a demo. It didn't do well. Yeah, we'll talk about a little more about that later on. But I don't know. I think there was a lot of fixes. They did make a lot of fixes rather quickly in the alpha. I think if they have somebody else looking after the servers, they'll be able to make those fixes even faster and get them pushed out even more quickly or so. Maybe that's great news. And as far as what they're working on, they're still working on the minions. We I know we've been saying that for like the last couple of weeks. But if you if you watch Smokey's streams, you can see that he's very quick. He's very good at what he does. He can very quickly change things about a hero's kit, how it works, implement bug fixes very quickly. The minions, the minions are kind of getting him. The minions are holding him up. And there is a lot more that goes into the minion AI than I ever thought. Mandy, you got to play a little bit, right, with the with the minions in in predecessor. You got to play test that. Yeah, last week, I think it was they invited me. They needed some people to fill a spot. They invited me to jump in on a couple of test runs, basically just testing out how is the game performing with minions on the field? There there wasn't a lot of AI programmed into them. And he basically told us at the beginning, we know they're dumb.  Just we need to know, are you getting any lag? Are you getting any rubber banding? Are you getting anything like that? And actually, everything was great. We we play tested both on an NA server and European server. And it was great. I mean, I thought it was I thought it went as smooth as a baby's bottom. So that was really impressive. And like you said, there's so much that goes into these minions and what they have to interact with. And just I mean, it's it's incredible if you know the nitty gritty of it, which I don't. I barely, you know, have scratched the surface of what it must take to program these minions in their AI. But yeah, it's definitely reasonable that they're going to take a little a little TLC to get them working right. So. Yeah.  It needs to de agro and then return to its original position and just getting the correct radiuses for that, like how far they should chase you, how far away out of the lane they should go and where they should return to after they de agro. It's a lot of stuff involved that he's really trying to get worked out. Then some other news from a minus three years of predecessor this week. We discovered that sometimes this month, we don't know what day is Smoky's birthday according to the Queen Elvira Smoky's wife.  That is the exact quote.  Tell him we say yeah. Yeah, make sure you say Mandy and Mongoose and Jeff. It's specifically said to call you. Oh, you're right.  It's been about what, 20 years? 20? No, that hasn't. 10 years. About 20 for me. Oh, yeah. Well, I mean, if we if we were talking about resumes, then we would have 10 more years of experience. That way. We'll look at it that way. Gamers don't age. We level up. It's fine. Nice quote, Mandy. Nice. I totally stole it from the internet somewhere. Got to steal it from somewhere, right? But that's going to be it for Romeda Studios. We got nothing for made about. We got nothing for Phoenix Rising. Let's move on into the poll results. This is another one where people were pretty pissed. The comments were got a little angry, which is which it's good. It just shows that everybody's passionate about the game. Just remember, people, these these polls mean absolutely nothing to anyone ever and never will. So, you know, it was the it was Aurora versus Muriel, who whose ultimate was the most impactful. And it seems like I think people like if they in Aurora one, it was 76 percent to 24 percent. I think people would like vote on Aurora, see that she was winning and be like, cool, and move on. Now, if they wanted Muriel to win, they voted on Muriel, saw that Aurora was blowing Muriel out and like, oh, hell, no. I need to explain to these motherfuckers why Muriel was so much better. And I think there's cases to be made for both. But blood, what did you think about Aurora versus Muriel as far as the most impactful ultimate? I think it was a good decision. You know, Aurora is pretty good at, you know, getting in there and just freezing everybody real quick for her ultimate. Muriel, you have to be in danger for hers to ever even work. So that's true. Hers, it's like what is she just going to tell the port user ultimate on someone that doesn't need it when there's no reason. They're not low on health. So her ultimate was only a situation. Well, Aurora, as you could say, it was situational. But you could sometimes use it to get away, you know, sometimes and initiate, sometimes, you know, just because you panic, depressed and happened to freeze people in the air as they were jumping on you, just stuff like that. That's why I voted for Aurora. Yeah, some good points. I ended up, yeah, I ended up voting for Aurora. I do have to say, though, I'm I'm not surprised. Aurora won, but I am surprised at the margin. Yeah, yeah, I figured that Muriel would it would be a closer race than it was. But yeah, I just I kind of. I always fall back on if it's two heroes that I either didn't play much or didn't have a lot of experience playing with. It was like, which character did I get messed up by? Which which all messed me up or, I guess, in Muriel's case, helped me out or FaZe's case when she if is she in it? Is she in the no, she's. Yeah, he's got me down a long time ago. What was that impactful? I mean, it could be. Yeah, but I mean, not she or so was but the ultimate wasn't wasn't not not in the sense that we're speaking for like team fights and everything. But yeah, Aurora, I just remember getting frozen and killed so many times. So I went ahead and voted for Aurora. And I felt like I felt like it was such a strong, such a strong and useful kind of like Blood Hunter said, you could even use it to escape. You could use it to initiate. So so sorry, Muriel mains. I think I was speaking from a silver rank, so called, you know, silver rank, that's where my experience come from. I think I think a lot of it comes down to ease of use. I think Aurora's ultimate was a lot easier for everybody to use. You didn't need to be an Aurora expert to know that, hey, everybody's gathered around a whole group of friendly of enemy minions. It's time to go ahead and ultimate and do as much damage as possible. Whereas Muriel, you really had to know your Muriel. You had to be a very good Muriel player to know the best time to use a reversal of fortune. I mean, it could also be somewhat used as an escape or an engage.  They're just they're in there up against another just, you know, one on one. Muriel could quickly turn that into a 2v1 and take them down. Or, of course, you know, if Muriel is in trouble, she could just ult to somebody to get away. But I think the good Muriel player, I think I think reversal of fortune used by a really good Muriel was more impactful than a cryosism used by an Aurora, no matter what her skill level was. However, I think it was a lot easier for everybody to use Aurora. So in the end, I do believe cryosism was the right choice here. I think it was more impactful in the game overall than Muriel. But I don't tell people what to use, what standards to use whenever they they do these vote on these polls. And it's like I said, it doesn't really matter. So it all comes down to to Muriel. You know, she used to have a knock up on her ultimate. And that was it was far more impactful than then later on, especially if you put shockwave on her. If you had Muriel with shockwave and steel with shockwave, you could just keep the entire team bouncing around until they were all dead. Sorry, after the new hard system, I only used grugs with that one move that gives me my one shot kill.  What was that? It was like, that's something. Yeah, that's the only car I just did that. I didn't care if I win. I was just I just wanted to play with that card.  So that's going to close it out for the poll results for this week. Let's get into the highlight reel, speaking of people killing off their target before Muriel could get to him. That's going to be in the highlight. So let's go ahead and check those out now. We've got knocking the streamer here, formerly known as I am gamer. Going into the shadow. Well, he's in legacy. He goes into travel mode and gets immediately bricked out of it. That was the problem with travel mode right there. Rampage T and off of and got the red buff. And now Grim is in the party, gets this placement blast away. He's going to try and sneak around and use his cue. You can hear him saying cue, but they are just dogpiling in on him. He's got his sword up there and the wind up. He gets away with just a sliver of health. You can hear how excited he is to have actually escaped. Once again, it is travel mode. You hear the rampage rock smashing behind him.  Good on you for getting away. We've got James Ganko with two clips this week. Starting out with the Sparrow, the deceptively named 10 year old girl like that. We're going to start a big play here. Moving in on the red team starts the channel.  Take somebody out immediately, takes down several. Deckard just cruises through, pins him in with the containment fence, but it's not going to be enough. Aurora, I don't know why she stepped up, but she just stepped up through her death. Murdoch steps up into the black hole, gets taken down by Sparrow.  Sparrow is going to charge in and try and take her out. She's taking tons of tower damage. She's going to have to get out. But Deckard stuck between a rock and a hard place and she gets pierced by the Sparrow for the pentakill. Kalari cleaning up the lane here, but there is a massive push from the enemy team coming straight up mid lane. She's going to slink over to the side, enter the shadow realm and see if she can deal some damage and you know she's going to.  Starts teeing off on Wraith. He's juicing and jiving. Muriel comes in to save him, but it's a little too late. Takes her down, takes crunch down. He tried to save her too. The death sentence comes out to pick up a fourth kill. Now Muriel is in trouble. She's trying to shield herself, trying to get away and it's not going to happen. Enter the shadow realm. Once again for that big beefy slice through the face for the pentakill. All right, guys, I hope you enjoyed the highlights this week. Now, for the topic of discussion this week, it's not really anything set in stone. We just want to talk about some of the games that we've been playing in the meantime while we're waiting for these other games to come out and just some other general stuff that I wanted to address. But let's start off with talking about some stuff that we've been playing in the interim. Something that Blood just got me into was Destiny. If you don't know, Destiny is a free download right now. I think it's just free period. Some of the other you need to unlock some of the other stuff in it. But if you want to just try Destiny out, it's completely free. I've been trying it out. Been having a lot of fun with it. Mandy, I know you used to play Destiny a lot, right?  I loved it so much.  I haven't played since Forsaken. So there's a couple of expansions, DLCs that I haven't played. But I always look forward to jumping back in.  It is one of my all time favorite games and will always be. So I'm very excited that I'm excited when it first was announced that it was coming free to play on PC. I was like, well, what do I care? I have it on PS4. But now I have a chance to play with my friends that maybe only play on PC, like the man goose. So I'm excited. I'm, you know, I'm pretty good at the bow. So yeah, I mean, yeah, there you go. So I can I can get on and carry you goose anytime you want me to. I got bloodhunter to carry me. I don't know what you're going to do. You should get on PC there and play with us. Yeah, one of these days will definitely do that. The way they did it on PC is they the way they did the whole game on the with New Light called New Light is the free version. And they took the original mission that was the best mission before they fired a guy. Destiny, the original Destiny one mission that you had. They put that in the first light so people get that sense of wonder where they wake up. They just don't know what's going on. They put that there, which makes it better. And then they have like, I think they have another mission that leads you into this, too, right away, like the next. And then it leads you to this. You have to learn your that you did lose your powers and you have to learn to get them back kind of like to do in Forsaken. And then after they just put you in the tower and like, here's the tower, here's the hub. And yeah, explore. Now they did it where you don't it's not forcing you to do anything anymore. It's not like you have to finish this campaign. You have to go here. Now it's just like, here, here's all the things you can do. Now go choose. If you want to choose to do the campaign, you can just go to this vendor and start the campaign instead of doing it like it used to be. So they turned it into more of a MMO, basically an action MMO. But I've never liked those because they always low res or kind of top down almost or or a third person. But this one's actually pretty good. It keeps you involved. Yes, definitely. After the story. Yeah, I didn't expect to enjoy it so much, but it is it is pretty fun. Compared to right killer, though, compared to Paragon. Yeah, yeah. But compared to Paragon, like it is like, you know, a hero shooter kind of kind of thing, which is where Paragon was going towards. It was going towards pretty much as being a hero shooter that happened to be on a MOBA map. So, I mean, it is kind of like Paragon in that aspect.  So that's kind of fun and kind of cool, and I like that. Overall, enjoyable game. I'm digging Destiny. Mandy always used to talk about it and Blood's been trying to get me to play it forever. Now it's free. I have no reason not to play it. And I tried it out and it's a good time, man. It's a good time. Good stuff. What did you play, Mandy? Did you play Hunter or what? Oh, of course. Of course, I was a Hunter. I always play Hunter. Absolutely. Actually, by my name, a blood hunter. Yeah. So any time you see my characters or my spare or my ghost, everything has a blood hunter kind of look. Yeah, they're really cool looking.    Another game is speaking of other games that I've played besides Paragon. Well, since Paragon probably was a No Man's Sky got so much better this this year on their Beyond Update. I used to hate No Man's Sky. And then the Noom No Man's Sky.   It's just you can go straight up in the space and then they have it in real time. We have to go to another one, another like planet or something.    Did you know Paragon was my multiplayer? I was at somebody if she didn't play No Man's Sky. I played it before the update. Gosh, this was a while ago, because this was when I worked at GameStop and you could actually like check out games, basically. And then GameStop, weird flex, but OK. I'm just saying. So yeah, I just checked it out for like three days, played it. I was like, meh, not for me. But I have been watching a couple of my friends stream it not too terribly long ago, back when the big update kind of first hit. And man, I was like, I kind of want to get back into that. Like, I think it would be super fun to play with a group of friends. But also there's so much to do that it would be such a cool thing to do on your own and just go and explore and see what you can find and make your way around. Everybody can jump into like everybody gets a freighter now. Like you see the big stars destroyers in the sky. You can actually own one of those, but it costs a lot. There's a certain place in the game where you'll get one. But I want to say anything about that.  There's there's billions and billions and billions of actual planets you can go to literally. You can get lost in the game if you're. I still haven't finished the story and I've been playing it for a few months. Yeah, really. It's it's it's you can build your own base together. It's just it's almost like a second life if you really get into it. Plus with the beyond update, they released the VR and then they completely read the game for the VR. Which is you never played VR. That's a really good game to play VR. Crazy. That's really cool. I didn't play it. I just I saw all the negative reviews for it at the beginning and then just moved on. Yeah, it did kind of get pooped on there. Yeah, I really haven't been playing anything with any kind of regularity. I just haven't kind of. I just bought my time to one of these games come out. Really? Yeah, same here. I think the thing I've spent the most time with consistently that I tend to bounce back and forth is Red Dead Redemption 2. I was like, oh, my gosh, I was in an absolute funk with gaining. I was like, I don't know, nothing is nothing is paragon. Nothing is like I want to play this all the time. I want I think about this game outside of playing it. I was in a total funk and I don't know why I had never I didn't try Red Dead when it first came out because I'm like such a I don't know. I love cowboys and horses and stuff like that. So it's like right up my alley. But man, just like the story, the online, the game is beautiful. Every time I play it, I like stop and look at the scenery. And I'm like, this game is beautiful. And now they've had they've had a big update. They've added roles now you have different roles you can pursue. It's just it's amazing. It's another one that has had some polarizing opinions. Some people kind of poo poo on Rockstar. But I've had nothing but amazing experiences with it. And it's the the thing that I've been most consistently returning to. But like you said, Goose, I've been hopping around a lot from game to game just kind of seeing, you know, what will what's fun for a little while. So I heard someone call that game racist one because that depicted slaves and stuff like that in it. It's more towards Native Americans. There is it's like right.  But yeah, I mean, I like it. Yeah, I just I kind of grew up that way. And it definitely portrayed the I would say the white man as the enemy. You know, it well, actually, it did a good job of not making either side seem like the enemy. But of course, I I sided with the Native Americans. I'm like down with the white man. It looks just one more thing about that game. It looks so good in 4K. I put I had a 4K TV and I played it on my PlayStation Pro. Oh, and I was like, man, this looks so good. The snow, the aiming, the the very first scene that you're in. It's like, man, this I was sitting there not playing the game. I was looking at the scenery. Oh, I have taken so many screenshots of that game because it's just beautiful. I've only ever watched Mandy play. I've never played it before. Oh, you're missing out. You've got to play it. I'll play the original. The only other game that's good that I think my best experience with gaming just with all the, you know, I have a point system that like it has to have this, this and this and this. I had a game that just had nine out of 10 of the things that I love. And I was obsessed with this game. And that was a PlayStation exclusive. One game of the year last year. God of War, God of War. Even if you didn't like it as a kid, they changed it so much. And then that one camera system that basically keeps you in the action, it never goes to a cut scene. It just camera just kind of scrolls out or it spins around. But then when it's when the cut scenes over, the camera spins back to you and then also you're playing. Sometimes I didn't even know I was supposed to be playing or wasn't playing just because of how good the kind of like Uncharted did a little bit. But this one did it the whole time from the time from the time that it shows you on the title screen. From that moment, it never breaks away from your character or around your character unless you die in the game. Then it has to do a load. Even when you get to the end of the game and the credits start playing, the game still it never fades to black. Wow. Until you get to this one, you figure out the last little tidbit secret that hints of the next game. But I don't want to ruin that for anybody. Yeah, don't. Yeah, that's that's when I haven't touched. And honestly, like people have told me, play it, play it, play it. But I've heard it's really difficult, but there's a lot of. Difficult play on the if you play it on the easiest level, which sometimes you don't want to play on the easiest level. But if you play on the call, give me a story. The combat is still OK for new people.  Yeah, it's especially if you're into mythology and all that. And it's nice that they don't make you feel bad because a lot of times the easiest setting is called like a windy baby diaper boy setting. Yeah, you tell me a story. Yeah, that's that's what you're trying to do. You just want to look at the story. Yeah, being bothered with too much difficulty. I like that. Yeah, the hard, the hard one that keep people keep saying that was hard is people tried to start that game when it first came out. And it's called Give Me a God of War. That's their hardest difficulty. People started on that difficulty. So it's like, no, play the story first. I did not go well for it. Yeah, right. So that's going to close it out for this week.  Or do you want to plug anything? I got a Twitch channel when I don't have a constant streaming time. But if I just feel like streaming, I'll stream when something new comes out, which is Blood Hunters Legion. A little different from my main name, Twitter. I don't know. It's the same thing. That's about it. I'm all my links are on my profile in your disco. Anybody wants to hang out with me? I'm always here to play, especially Destiny. I'm here if people need help. I haven't played in a while. I haven't leveled up that much because I've been helping, I guess, Mongoose and a couple other people. I was like, it's a I don't I don't think it's a burden. I just like playing with people. Yeah. Mandy, anything to plug? Um, I kind of want to do a special plug for my stream this month because I'm going to be doing all like spooky games and I have special Halloween alerts, so we're going to play nothing but like scary, dark, gritty horror games for the entire month. And you can come scare me with the alerts. So right on. I got nothing to plug this week, so that's going to close it out. Appreciate everybody coming out, especially everybody that seems to join us every single time. I just really appreciate you guys and people that don't have time to join us every time, but you still show up. You still leave those comments. Thank you so much. And even the people that show up and just never comment because, you know, maybe they're maybe they got a little anxiety. They don't want to say anything in chat. I appreciate you too. Just I don't want to leave anybody out. Everybody that watches the show. Thank you so much. But for now, this is the four of the minions crew signing off. You guys have a good one. A man goes.


And now, Ned Williams talk attacking Chrome Interprocess Communication reliably finding bugs to escape the Chrome Sandbox. He will be talking about finding bugs in the Chrome Interprocess Communication in order to escape from the Sandbox using a filing method to enumerate the attack surface of the Chrome Interprocess Communication. Ned is a vulnerability researcher, he likes CNC++ vulnerabilities, did research for consoles and browsers and now started to work on mobile devices. Please welcome a huge round of applause. Hello everyone, my name is Ned and today I'll be talking about Chrome IPC. And actually as I was writing this talk, I kind of came up with this idea to make it more useful to everyone. The way I ended up doing this was by trying to start really general and then kind of going more and more specific all the way down to the Chrome IPC fuzzing. So if you're really technical, the end will be still interesting and then if you're new to this stuff, hopefully the beginning, the power will show some of how to get started. So just a quick overview about me, I've mostly been spending the last several years on low-level vulnerability research and my particular interest is on any kind of critical bugs, meaning kind of the more severe individual bug, the more interesting to me. So I'm trying to kind of solve this problem of how do we make the bug finding process effective enough to bring out these really rare hidden bugs? And you will see an example of how to do that by the end. But just an overview, I've basically worked on four things. The first being CTFs then went to 3DS and Chrome. Now I'm starting on XNU but just a month ago, so not too much yet. Before we get into it, I just wanted to give a little recap of what happened since last time. So I was part of the Nintendo hacking talk two years ago here and I presented two exploits called Sound Hacks and Fast Hacks. Not to go into it too much, but I did want to share what happened here because I was actually surprised. I put Google Analytics on the Sound Hacks website and I thought maybe a thousand people use it or something, but I just looked at the stats a couple of weeks ago and then turned out like 800k people use it or something. And then I searched YouTube and found these huge videos where they were copied. I wanted to have a screenshot, but it's copyrighted so didn't do that. But basically it looks like something on the order of about a million users, which is crazy because this is one of my intro projects really. So I think this should show you that you don't have to be all the way up onto Chrome or whatever it is to get into this and do some huge fun project. And then I just wanted to publicly talk about the donations because I had a donation link on the Sound Hacks website and we fortunately received about a thousand dollars in donations and then half of that went to the emulator people because they that's how I eventually wrote my exploit for that Sound Hacks. So it made sense to repay that. And then the other half went to buying switches for the toolchain developers who couldn't afford it. And so just wanted to thank everyone who used this or whoever donated, just shout out. So we'll get into the actual meat of the talk. So basically I want to focus on the bug finding process, not exploitation necessarily because this topic is pretty well explored. And I think the bug hunting aspect is kind of what's the most prohibitive for people to join in. And when I look at the number of people who I play CTF with who are really good at exploitation and then a number of like these prolific bug hunters, it just seems like from what I see from how smart people are, there should be more people doing the bug hunting. And I hope that if I can talk about it, more people can come over. So with that, the agenda will be just overall, how do you make a process to achieve any goal? Then next, how do you apply this kind of some kind of strategy to bug hunting? Then this new fuzzing style, I've been kind of developing some other people out in the industry I've been working on. And then finally, how does this all tie back to Chrome IPC? So also just to mention, I should mention that the bug I'll be showing in this presentation was used in a full chain exploit that I developed with a couple other people. And the details of the exploitation of that will be discussed at offensive cons. So that's also here in Germany and hopefully people will check it out. So how do you become an expert at anything? And I kind of was thinking this before I even started anything and I was in the CTF stage and I was just kind of curious, if I approach this with the mindset of there's this arbitrary skill I want to learn and if I approach it strategically, like what's going to happen. So I looked into this expert research and then there's kind of this idea of pop-psych, you need to study something deliberately for 10,000 hours to get good at it. And there's some debate about this number it's kind of made up I guess. But the essential idea of deliberate practice I think is very useful and it's exactly how I structured my studying. And so what deliberate means is when you're learning you want to be thinking purposefully like I want to make sure that the project that I'm doing is making me get better. I want to be actually thinking about how I'm structuring my training and then you want to make sure that you're kind of always struggling because that's just how you're growing. So essentially to do this you just need to keep picking projects that have some like success and failure feedback mechanism that's tied to the real world. And you know with bug hunting like this is very obvious you know you're either finding a bug or not. And like as I mentioned you want something difficult but achievable and so this kind of order that I did the different projects I mentioned in the beginning was like specifically chosen so that each stage would be achievable to me but also like really really stretching what I could do. And there's a funny antidote there's this guy named Ben Franklin from American History and I read the story that he used to be really bad at writing and wanted to get better so the way he did it was he took an essay that looked perfect to him and then he took notes on it and then a week later he rewrote the essay from the notes and then he would just compare the the goal versus what he had done and basically saw all the shortcomings and so that kind of just stuck in my head and so I'll show like how do you apply this kind of trick to bug finding practice and then just another thing with setting goals for bug hunting. A lot of it is psychological I think it's almost psychological more than intelligence like for sure and basically you want to iteratively pick harder and harder projects so that your tolerance for failure goes up and up and so by the time I was working on Chrome I worked on it every day for six months like right home from work until 1 a.m. sleep up and then all day every weekend and found nothing the whole time and then just one day found something and then from there all that accumulated like struggle and effort when the bug precipitated it was just like a sign that all these like necessary skills were there and then I was able to repeat it and so so now I'll talk about what that actually looked like for bug hunting so when you think about how to train the skill I think there's kind of two constituent skills that are important and those are knowing where to look and then recognizing the bug when you're looking at it and this first part is just from my own experience it seemed like just being a developer it's pretty easy to get a sense for you know you can look at the git logs like I'm mentioning here you know are there crashes happening somewhere in the library are bugs getting reported you know publicly does the code look bad you know it's not hard to tell that something looks sketchy but I think what's really hard is getting the bug to kind of come out and so that's where I'll talk about strategy kind of directly and so I kind of have this training idea where essentially once you have this kind of target in mind where it's a little bit out of your skill range but you think it's doable you try to enumerate all the existing bug reports and then look through each of them and then this it's this like ben franklin idea like you take the bug and then you look at usually there'll be like you know this block of text and they're mentioning like the file where it's happening and stuff and you can kind of skim it and sense like where the bug is and so we know without actually looking at what it is and so then you go over and you try to find it yourself and you know it's really important that you actively try to look for the bug yourself and kind of strain yourself and when you've given up essentially then you look at you know what was the bug and then through that struggle it's usually pretty clear like what was the fundamental thing you were missing and you know just by repeating this process constantly this is how you train and so this is actually how I first ever started on bug hunting was you know some of you may know Yuru he's this like really talented researcher he's been at it for a long time and I remember seeing this blog post from him showing all these IDA pro bugs and it just kind of blew my mind like wow someone took IDA and found like security vulnerabilities in it and then when I looked at the bug reports they're pretty small so I thought okay how do I practice and how could I have done this myself so basically the first day you know they're all like integer overflow bugs I could barely even like I knew what integer overflow was but I hadn't like actively looked for it before and so you know I was looking at the function I couldn't find it and basically I went to sleep feeling like oh god like I'll never be able to do this stuff and then the next day I looked it again I was like oh yeah that's actually easy and then kind of filled the second one and then kind of by the third day I was like able to just see where they were once you know I knew where to look so that kind of made me think okay I'll just keep doing this for a long time and keep doing harder and harder so this is essentially the strategy like I think you know I'm probably the perfect example of someone who was like an intermediate CTF player really like insecure whatever like and just you know wanted to get into this but I had no idea what I was doing and I just kept thinking if I just believe in this kind of process you know hopefully it works out and so here's just like a little really basic roadmap if you want to try to replicate what I did which is to focus on CTF because if you can do CTF binary problems these are perfect examples of a kind of training where you try to do something yourself there's a write-up and like once you can do these problems you know all the kind of low level details that are needed you know you know what a bug is things like that and then from there you just kind of progressively do harder and harder targets and so there's kind of this component where like you know I don't you can't really assess your own ability like how much of this is innate or something and it just seemed to me that regardless of that you know this like I'm saying here you know this isn't chess where you have people like trained from birth with like perfect study and like decades of you know like we're barely figuring this stuff out and it's just kind of a huge mess and so there's plenty of room for new people to join in and then also like there's a lot of these kind of stories about people who are just insanely naturally gifted and stuff and I tried really hard to like look into what these people are actually doing and I haven't found a case where someone wasn't working extremely hard and so you know just keep that in mind so just for the sake of time I won't go into this too much but if you're looking at the slides later I just kind of give more detail on like how I picked the mini projects and got down to chrome so now let's talk about fuzzing and so before I get into it I should emphasize that you should really know how to do auditing and the first couple of years and like not until into that six months of failure on chrome you know I was doing auditing the whole time and I think fuzzing gets a bad rap because people think that these are unrelated strategies and people are only a fuzzer person or an otter person and really I think these things are extremely like they work really well together but you can't really know why fuzzing is failing or how's it even apply it or where to apply it without being able to audit yourself and part of this was like I noticed on chrome that I could audit things but there essentially the bug density was so low on the sandbox attack surface that I needed a way to kind of automate what I was looking for in each each subsystem I was looking at so you know you have like 20 subsystems that you want to read well you know it takes about a week each minimum to learn it's a lot faster to try to fuzz for like a day or two each thing and then I don't know like it's I can't explain it it just did random things and then this is what worked so so how would you practice fuzzing it's it really the same idea that I had about auditing where you take a bug and just ask yourself like how would I have written a fuzzer in the first place to write to hit the bug how could how could I have known to write the fuzzer that would have triggered this you know am I lacking something in auditing ability am I not able to write fuzzers well enough and actually took me probably like a year of of fuzzer writing to get good enough where I could actually act on my ideas like just it's it's kind of tricky and so we'll get back to it later but this exact idea of practicing fuzzing on something that looks unfuzzable is how I found this real exploitable sandbox escape so really quick just for those of you who don't know too much about fuzzing at least in like the current meta essentially there's this tool that called AFL that came out in 2014 which I think really shifted how well fuzzing worked and the idea is essentially that you have some corpus of inputs that you want to fuzz and then as you're mutating them you're looking for coverage feedback which is compiled into your code and then as you're mutating and running new test cases when you find new coverage you take that input and put in your corpus and over time your corpus kind of grows and grows as more coverage is hit and so there's this just seems to work really well and then there's another version of this basically called lib fuzzer and this is just written by the LLVM project and the same people who wrote address sanitizer also wrote lib fuzzer and just in my experience it's written in a way that's a lot more extensible and like easy to understand and play with and so it makes it kind of easier to audit and fuzz together and so you know if you want to think about what fuzzing is essentially you're trying to replicate the normal testing process but kind of parameterizing like what a unit test would be doing with some input bytes that you're just feeding into something and seeing if it crashes and so what's interesting is there's kind of this gap in the middle of like an end-to-end test which AFL will give you just feed the binary or like the unit test which lib fuzzer will give you where you just keep stuffing bytes into a parser and real security vulnerabilities are kind of logical in nature and I think that's why people think that fuzzing isn't applicable and I think there's actually kind of this part in the middle where if you see a few components that look suspicious and then you can integrate them and fuzz them in isolation but have the complexity that you'd kind of see in the real program that's where a lot of bugs come out and so how we do this is using grammar and so essentially it's combining generative fuzzing with coverage guided fuzzing and so we'll touch on how that works in a minute but just for some more evidence on you know why does this work well like I'm not the only person who is doing this kind of simultaneously myself and two other people I guess seemed to have stumbled across this idea last year or two years ago and those are syscaller and lokihard so syscaller is a kind of fully automated linux kernel fuzzer and if you guys haven't seen this it's kind of hilarious like essentially they are automatically generating zero-day bugs like tens per month at least and they automatically generate the test case like submit the report when the commit comes in it's like automatically tracked it's basically the zero-day generator sitting there and yeah no and I see this I'm like okay there's 3 000 the bugs that are being found there's a web app for it and you can just download it you know and I saw the linux talk from the author of syscaller and the youtube video is like 100 views and stuff I'm just like okay so people need to I gotta reiterate how important this stuff is so then there's lokihard as well who's like a famous extremely talented kind of canonical auditing fuzzer person and he seems to be doing a very similar thing with chakra and v8 and he's finding like tens of interesting exploitable bugs and then there's me who applied this on the chrome sandbox and found over 30 bugs about half of which are security relevant and then five of which were like sandbox escape without render code execution so you know this is just to emphasize like we're finding really important things with this technique and since I discussed this the first time a couple months ago at psc conference it's been used by someone in their chrome security team to fuzz SQLite and they're already finding new bugs like in the first week so just more of the evidence like here's the kind of the breakdown of some of the bugs I found with this strategy so just to highlight a couple of them or maybe three of them so the first one was like an out of bounds read just an integer overflow in blobs and this lets you you can make a blob and then ask to read part of it and then the offset could have been negative and there's an integer overflow they got the check wrong so it was a full like memory disclosure from the browser process there's also this like app cashews after free which is what I used in the exploit this year and then finally I guess the critical bugs are pretty interesting so two of these I guess the first pair are in quick and the first one is a stack buffer overflow with just a bad packet that comes in over the network so you just browse to an attacker site and they stack buffer overflow chrome browser process which is outside the sandbox and it jumped over the the stack cookie so that was bad and then then these block file cache problems these were in the HTTP caching mechanism which is also in the privilege process and um these were actually crashing in the wild for three years and uh the they didn't know how to I guess I don't know if they didn't have resources or they didn't know how to address the problem or something but I sent them the test case and then they closed like four bug reports and you know ancient bugs so you know it just goes to show that uh this kind of technique works in the variety of really interesting places that are really important and so now let's get to the boring stuff so uh what's protobuf well protobuf is uh this data serialization format uh from google and it doesn't really matter that it's protobuf just this idea is uh you want some kind of um you want to encode like a little language for yourself uh that expresses uh what you want to fuzz a kind of a higher abstraction layer than just fuzzing bytes randomly and so if any of you have done functional programming like uh I had been doing stuff with okamal and like uh quick check for a couple years and then when I saw this it was just immediately recognize the pattern um essentially what you can do is you can create this little tree structure of just basic types like enum um you create these uh messages and like you can just kind of specify um actions you want your fuzzer to take and then um what uh this next little lib protobuf mutator will do is it will take this specification you've written and link it into lib fuzzer so that it will automatically fuzz and create these like trees that are um you know these kind of random ASTs from this little language you wrote and then you can kind of parse this language which sounds crazy or more hard than it is but you essentially um you can generate this highly structured input which makes it a lot easier to explore like a logical type of uh bugs so um yeah just really want to emphasize that this strategy can be used to fuzz anything and so um kind of this same exact idea as being used to find bugs and like caching apis encrypted networking protocols uh kernels sandbox uh like serialization code um stateful systems that have ipc and network interaction and timing as part of it which is what we'll show at the end um and so like what's what's common here you know we just fuzz all of these different systems in the same way uh the idea is like as an auditor what what you do is you kind of notice like okay there's some subsystem like uh some caching mechanism with a simple api and you uh look at how it's implemented it looks complicated so you think okay you know if i can write a fuzzer in like a few hours for this you know it seems like high value so um once you kind of play with the api bit and understand like uh how the api works you know you can just write this little specification for the api and protobuf and go ahead and write the fuzzer so uh uh basically uh i'll show how this works on chrome so um just to make sure i cover all of uh kind of the background knowledge you know for those of you that don't really care about fuzzing or don't care about anything else you know at least you can get bootstrapped on uh chrome ipc research um the basic idea of how uh the chrome sandboxing situation works is when i'm saying i'm finding bugs in the sandbox like um it's really finding bugs in the browser process which are reachable um you know from a sandbox process and so um the sandbox itself you know it's just constraining these render like tab processes um so they can't really uh do much and then what you want to do is kind of jump from there to the browser process which can do anything so it's a very common model like almost you know like on 3ds you know you have like user land kernel then security code processor you have a linux like you might have a user land process and then in the sandbox there's some apis in the kernel you can hit like syscalls you can hit and basically everything just keeps boiling down to there's some api that you can look at from the less privileged context and then if you can trigger a bug in that api you escape and then you just kind of you know this kind of applies everywhere and so this idea of understanding like self-contained chunks of like uh you know syscalls in linux are hundreds but being able to look at and say like okay here are 10 related syscalls um you know this is like a subsystem that i want to fuzz in isolation like this is kind of uh how you want to think about it and so um if you just want to get started on uh chrome uh you what you want to do is look at okay what are these end points in the browser process that i can reach from the renderer and then um you don't really have to understand how ipc works to do this you just have to be able to recognize where um what you're allowed to hit from the renderer to the browser and um what's actually in the browser and so fortunately the the chrome code base is pretty well organized so they just tell you if you just generally go into any folder that says browser in it like all of this is outside the sandbox and uh prone to sandbox escape and so uh most of my bugs i found were in this content browser sub system kind of thing but you can look anywhere and i think like um all these results i've had the last year were just like in one folder and so um you know there's so many other places where bugs can manifest that i didn't even look at so uh basically there's plenty of room for more um so just to go in on what i did is uh in this kind of content uh stuff is uh you just want to see where kind of the apis that are reachable from the renderer are enumerated and those are in um this render processed host empl in it function um so yeah like c++ kind of wordy but you get used to it uh basically there's there's two places where the the apis are set up and or the interfaces are exposed uh those are create message filters and register emojo interfaces and it took me a while to realize where these were like a year or something but like that's those are the key functions to look at and so um i'll skip over old style ipc because it's going away but it's pretty uh easy to figure out what's going on if you look at it so i'll talk a bit about mojo so um essentially this is a new ipc kind of platform that the chrome team has developed and um the idea is they want to uh i guess simplify this process for developers in terms of defining a interface that you want to expose to a renderer or some other client somewhere else and um essentially you write these little interface files called mojon and then the build system will generate all the c++ glue for you that you can just like subclass something and then it handles all the the mechanics of actually exposing this to other processes and and so on and so as a security researcher you know you don't really care about that all you care about is like what can i reach and uh how do i know like what to fuzz or something so uh what i guess i looked at is just you know what are some of the mojon files that have um that are subclassed in this content slash browser and you can just do a little crap to check this uh so um essentially the apcache was one of the bugs i found this year and um here's the api that the renderer can um you know these are all the messages that the renderer can send to the browser and along with the types of uh the arguments and so you know that's pretty straightforward um so in the browser process this is the code that we're trying to attack which is the actual c++ like implementation code for this api and so you can see their subclassing there and then they just make sure to override all these virtual functions that actually implements the api and so um i won't go too into detail in this part because it's a little boring but essentially um you know how does the renderer get from it to all the way over to this kind of browser c++ code well it essentially goes through this like request mechanism where the renderer tells the browser process like hey i have this kind of request uh to access this interface and then um it'll actually just uh create that uh like dispatcher implementation object and um you know just feed in that request over there so essentially stuff gets glued together somehow and um you know then there's uh this stuff which is kind of ugly but i mean here's where um you're actually exposing the ability to to do this so here's here's where we're actually like the request comes in and then where um this kind of request handler function gets fed in is that thing i mentioned earlier to register mojo interfaces so it's pretty it's named pretty well it's kind of easy to follow um and they're adding new stuff constantly all of this stuff is on the attack surface like i think um and i stopped Chrome a couple months ago i think i looked and there's like you know five new apis in there or something like they're just constantly adding things so um just a quick point about this uh essentially you want to do fuzzing in process with this like libfuzzer protobuf mutator uh strategy and um you don't want to be like actually doing ipc and like it's just very brittle and weird so what you really want to do is just like here's the cbless plus object i want to just instantiate it and call those functions uh myself and then um this whole thing is just very lightweight and easy to uh play with which is um you know having a lightweight and like very easy to rebuild tweak something and play with it print things like kind of the faster you can iterate the better so all anything that's too complicated like the success rate goes way down so essentially like uh you know the uh the fuzzer that i made open source is like the way you should do it but the way i actually did it was i just um like made the object uh like commented out the the private like i don't know if you can see it on here yeah so just like commented out private created the object started calling these things randomly it would crash and i would just hand fix things and you know it's kind of sloppy but um you know you're testing something in a very small unit that's not really exposed to that kind of testing so um now let's kind of put together everything i've talked about so far um so this exploitable abcache used after free i found this year was found using this same idea of deliberate practice so um i looked at uh this abcache subsystem in the browser process and i noticed that there were three old bug reports uh that were triggering memory corruption and they were pretty interesting because they involved different uh kind of ways of attacking and these things had clearly been audited and i had actually seen these bugs a couple years ago and um i i kind of used it as evidence to myself at the time that fuzzing doesn't work and you need auditing but um it kind of stuck in my head and i kept thinking someday i'll come back to this and um like i'll overcome it you know and so essentially uh what's interesting is you know i've already talked about you know it's easy to specify the api and just feed ipc messages into it and i think you know everyone kind of understands that who does any ipc fuzzing but then there's also this idea that you've got some remote server that the abcache thing like creates a network request some server is you know serving that request and doing different things and so in the second bug it actually matters when things were like when the server was returning data um because some jobs like stay alive and then if you send an ipc message to you know close your session and then the job is still alive there's like a raw pointer somewhere and you know something going on that it matters that the server keeps a connection open and then the last thing is just kind of a logical issue and if the server returns these htp codes in the headers of the response in this kind of weird order you trigger some logical bug that actually leads to memory corruption and so um you know i looked at this and i said okay well so what do we need to test to cover all this uh basically ipc network and um that timing and so uh not only that but this is kind of a state full thing so we want to make sure that um for each fuzzing session that um we kind of reset the state completely and fortunately in c++ this isn't too hard because um you know you just destroy the object and if it doesn't exist anymore what state is there so you know you just make sure that like you don't leave things lingering um so yeah so i just had this basic idea we'll call random ipcs with this fuzz input uh we return uh random data from the network and then uh we reset the state of the cache on every iteration and then part of it was thinking okay like if i can repro these old bugs if i reintroduce them by editing the source um this is kind of appealing to this like deliberate practice idea that like i could have written a fuzzer that would trigger these old things and this is kind of the idea i was pursuing when i actually triggered a new bug so um now what's tricky about this is if you just return random data from the network you're not going to make much progress and this is kind of where the auditing background comes in is uh you know you want to think about what is expressive enough of uh like how do i make my fuzzer expressive enough that i hit uh can hit everything but then not so generic that it's just spraying like it's just noise and so um i'll show how i did that uh in this specification and so at a high level my kind of root node in the ast or the tree of uh uh the fuzzer message is this session message and then this just contains a sequence of commands and so commands are something i also made up and so the first 10 of them are all the different ipc calls i can do the 11th one is handling any pending requests or pre caching uh like a response to any new request that comes in um so that handles like both the asynchronous case where it makes a request that's waiting for the server and also like the synchronous version where the response comes immediately and then lastly this run until idle thing which um essentially just uh it helps you like if you kind of place these uh run until idols uh randomly as you're as you're kind of doing these ipc messages um you're kind of flushing the queue of accumulated uh work and so what this lets you do is kind of identify these race condition uh type things uh because uh you can do something like uh do a bunch of ipcs that come in and are handled at the same time without like actually serving like actually doing the work yet and then you do this run until idle and then like all the work happens um and you know i didn't like think of this a priori in some like smart way yeah i just looked at like the unit tests and i just try to think about like okay how are these developers already testing it and this is just what it looked like they were doing so um these messages are very easy to write essentially just provide um for each ipc message uh that i could have sent to this thing uh just make sure all the arguments are are correct and then um there's a little bit of cleverness which is like the host id is um also breaks down to uh just an enum of like zero one two because just from looking at the code you know that if i'm randomly um creating hosts destroying them and stuff over the whole like you know four billion int 32 uh id is like it's just going to fall apart and not find anything interesting so um you know i just constrained that for the url i also that is also a custom message that i constrained to just return a few like pre-made legit urls so that way i'm also not testing like the url parsing stuff um so then you know how do i handle the network well i just read the source and looked at what are all the types of uh hcp response codes that affect control flow and i just enumerated them and then um for any given uh request that comes in from the apcache system uh i kind of just encode anything interesting about the response that i thought of just by um by reviewing the source and it seemed like the things that mattered were those hcp codes um whether or not the headers asked apcache to do caching or just download it once uh and then also the the apcache uh can request from the server this manifest file which has some metadata about um what files that it should be caching and so um you know essentially just all of this is encoded in one message and so how you go from this like high level description to actually fuzzing um is just this so you can see how how simple it is you're really just um you know i looked at kind of unit test code and saw how they set up this apcache service and so um they let you pass in this url loader factory and uh what this is is just this kind of unit testable uh network um request thing so this is how i'm i'm like you know intercepting the network request and feeding data and so i do this little setup and then here i just create the one like render to browser host um this is just kind of simulating how you would do the mojo stuff if uh it was a sorry a real uh the real render to browser interaction and then i just go through those commands that i mentioned and just do these things so i mean this is all it is you're just pull the host id out of this uh protobuf message that we're getting at the top there that session that i um defined as like the top level tree node and uh you know you just go through and you just call the the uh apis that are there and so how to get the network stuff to work you know as i mentioned i have this like mock url loader uh factory also c plus plus e but essentially um it's this uh well okay so this is when i basically handle one of my request messages that i came up with um i just simulate a response this is a built-in like unit test function that they have in their code base and i just uh pass in the relevant bits that came from that message so yeah so this is what it looks like i have some kind of do request uh helper function and then i just pass my stuff through to it um and so it takes that like url factory and then serves responses to anything that's waiting and then um what's interesting here and what's necessary to find the bug is that um you know i mentioned that this is asynchronous so what will happen is when you do like register host uh and then if i go back yeah you like register host select the cache um do some things like like the app cache will make a request to the server and then um get this manifest and then we'll start making requests to download things and then um these things are like pending it's pending like responses that it's waiting for from the server and so um it actually mattered that um you mutate the state further before those responses come in and so by doing this like in between the ipc messages i'm not like pre-loading the the network factory with a bunch of responses i'm actually like um serving things like i'm not making i'm not encoding an assumption about when i'm serving responses and i this is kind of tedious to to go so into detail but essentially uh you know you run this thing it's a maybe 150 lines or something and then trigger this bug with a dress sanitizer and so um essentially a use after free happens and what what's going on here is you can see the scoped ref pointer destructor and it turns out that um when let's see here yeah so when you go to unregister the host like uh it's an ipc that's an ipc message there at the bottom that i that i sent and then it just accidentally um this is kind of inaccurate this uh stack trace but essentially um some ref count goes from one to zero and then it starts destroying this abcash object and then um in the destructor one of these kind of requests was waiting on a response from a server and then essentially um like gives a reference back to that other object and that's kind of um aligning some details but essentially like the ref count went back up to one and then now you're adding a bunch of references uh all over the place to something while it's being destroyed and so um what happens is now you have all these pointers to a freed object and then you can trigger uh access to that freed thing again later and so this is kind of the recipe for an exploitable bug and so um i just wanted to point out that all this fuzzer is open source and it's just in the chrome codebase so if you download it or go online to the code search tool you can just search for abcash fuzzer and it will come up um so then uh real quickly just to kind of cover the the uh exploitation you know i guess i i have more time than i thought so i compressed this a lot but essentially um i did this in part in a in a chain with two other guys uh salo and nicholas and so salo provided the rce bug and so from there um we get code execution in the renderer and then this lets us send arbitrary ipc messages and so um it's kind of annoying to send ipc with mojo uh like arbitrarily so we kind of piggybacked on um the renderer side like glue code for sending these abcash messages so we just like found the c++ object and called into it and then um all in all we end up with this primitive where we can uh deck ref and like release uh reference to this ref counted thing um like after it's been freed multiple times so uh there's two stages to exploiting this like because we're in the renderer and we only have one bug we need to turn this into a memory disclosure and so uh you know fortunately this bug can be triggered repeatedly and so the idea here is um triggering it once gives you this you know decrement by n primitive and so um when you're releasing you know if you ever hit zero you'll trigger the the destructor again and so um essentially what you want to do for the leak is to not trigger the destructor because it will blow up uh but rather find a string somewhere in memory where there's a you know string pointing to the heap and then decrement the string pointer so then it starts like sliding somewhere else into the heap so that when you read that string back you're actually um leaking heap data and so um we did that so you know there's some object that had a standard c++ string in the beginning on windows the first uh q word is like the uh the pointer to the string data so you decrement this uh it was actually a cookie object so we just read the cookie back from the browser and then in the cookie value we see the the leaked uh bytes and then from there there was a uh a vtable um vtable access that we can control um in the destructor so we make another fake object that looks like it has one reference left you know make it hit zero so destructor is triggered and then this app cache uh thing gets confused and essentially calls a controlled vtable pointer and then from there you know those are the primitives you need to write an exploit and then it was just a matter of kind of putting it together and um if you're curious about that again you should look forward to Nicholas's talk and so just a summary uh essentially starting all the way from the beginning you want to be practicing deliberately um keep working constantly and keep identifying gaps and actively working to improve you know it sounds weird but you kind of want to keep that in mind and uh use this new technique with libfuzzer and protobuf mutator i can promise you it's not going to be uh the last time you see someone using this and uh i mentioned i've started on x and u and uh we'll see some initial results pretty soon on that uh it's working so uh yeah and lastly uh never give up just may take months but it's fine so uh with that uh i guess i'll open to questions yeah thank you thank you for that talk sure yeah if you do have a question please line up at the microphone phones in the room and try to limit your question to one single sentence if you would like to leave at this point please do that as quietly as possible so that everyone else can still stay for the questions and also if you're listening on the stream you can ask a question online to this seems there is no question ah there is one microphone number two your question please hello hello um i just want to ask uh why have you chosen uh chrome for back hunting was it just like you picked one random browser and then started uh yeah no i mean it's uh basically uh just kind of the hardest thing i could think of that i could plausibly do you know it's just for the purpose of getting better and so um there's more to it like i think chrome the way it's written is very amenable to research and like i actually didn't know c plus plus before i worked on chrome so like learning looking at a great example of a c plus plus code base and learning from that um was really helpful to me and you know i glossed over kind of my my path but i was actually finding random like obscure like library bugs that weren't even reachable at first so just the quality of chrome makes it so that what you're training is the real talent not just like being able to decipher bad code so yeah highly recommend it i can say that like i definitely feel that the two years i invested on that one project like completely helped me get better so yeah thank you sure single angel quest from the internet uh is it so the question is is it i guess possible to attack using mouth on respecter so i don't know i guess it's possible i was essentially focusing only on kind of application level bugs so things that i could trigger like kind of deterministically um using only bugs in the chrome code itself and so i mean also those things came along like way after i was doing my research so uh you know i can't comment on that but i'm sure someone knows thanks yeah and as i see no more people in the microphones or question into it yeah okay thanks for your talk and thanks for today thank you