Welcome back lecture three. We are wrapping up the review today, kind of an overlap between this course in 141. So if you have questions about any of that overlap, which I kind of picked up on yesterday, that it wasn't complete overlap for everybody in here. So anything you want to ask about that, because we will start with new material on Monday. So here we will use the table of integrals today. So if you don't have yours and you can kind of look off somebody else's, I have one that we can put up here and look at. We're on partial fractions. So let's talk about the two situations that we haven't talked about, at least as far as a review, that address decomposition into partial fractions, and then from that point we can integrate each piece. So these are taken from the appendix in the back of your book, appendix G. This is the first case, we've already addressed that, that if we break up this rational function, this R of X over Q of X, and we break it up into pieces, and each of the pieces is a linear factor in the denominator. Linear factors, regardless of how many there are, this is a linear factor, it gets a constant numerator. Here's a linear factor, it gets a constant numerator, and we'll continue with that. For every linear factor, they'll all get constant numerators. We had an example that dealt with that. The second case is kind of a sub case of this, we have not looked at yet. So we have a product of linear factors in the denominator. Notice there's an X squared in this example, but X squared is really a linear factor, it's X and X, 1X plus 0 and 1X plus 0, but it's a repeated linear factor, as is this. This is a repeated linear factor. Notice every numerator in this decomposition is what? Constant. So if the nature of the factor itself is linear, regardless of to what power that linear factor is being raised, it will command or demand a linear numerator. So here's our first linear factor X, it gets a constant. This is that linear factor X to the second, it's not an irreducible quadratic factor, because it is reducible, it's X times X. It gets a constant numerator. Here's our linear factor to the first, it gets a constant numerator. Our linear factor to the second, it gets a constant numerator. Our linear factor to the third, it gets a constant numerator. So it's kind of the nature of the factor itself, what type of numerator. Now, think of an algebra problem back in ninth or tenth grade, where you were taking this set of fractions, probably one of the first things you'd do would be to decide on a common denominator, what would the common denominator be for all of these fractions that have a constant numerator? Would we need just an X? We'd need an X squared, right? And we'd also need an X minus one to the third. Isn't that the common denominator? Right? So kind of think of that algebra problem that you did in your prior lifetime before you came here. This is exactly the reverse of it. So we're allowing for the fact that there might be a factor of this type. It is possible that A is zero. We allowed for this factor, but in fact, that factor isn't present. And same thing with any of these others. It is a possibility that any of these linear numerators could be zero, which says we allowed for that factor, but it doesn't really end up in the partial fractions decomposition. So we didn't look at an example of this yesterday, but there is a pretty good example of a repeated linear factor. Third case, I think we looked at one of these yesterday. Again, this is from the appendix in the back of the book. Irreducible quadratic factors. So if you've got irreducible quadratic factors, here's one. X squared plus one cannot be factored. Here's another one. X squared plus four cannot be factored. Anytime we have a quadratic, let me rephrase that, irreducible quadratic factor, those kinds of denominators in the decomposition will require a linear numerator. Here's our linear factor. It gets a constant numerator. Here's our irreducible quadratic factor. It gets an arbitrary linear numerator. We don't know what B is. We don't know what C is, but it's kind of generic linear. Another irreducible quadratic, it gets another, potentially another different linear numerator. So we did an example with that in it yesterday. And then the fourth case, we did not do an example with this in the example that we looked at from my 141 exam yesterday. If we have an irreducible quadratic factor in the denominator that is repeated, so I don't know why there's not an example here. Maybe I cut it off too early. So let's say we have an x squared plus four squared and an x squared plus one cubed. And we've got some polynomial of degree smaller than what's in the denominator, in the numerator. So we allow for x squared plus four to the first. It gets a linear numerator. Somebody tell me how to keep going here. What's the nature of the factor that's repeated? It's quadratic. What kind of numerator does a quadratic factor receive? Cx plus d. Cx plus d linear, right? And we would keep going with that till we've got it to its highest power, which were there. So then we'd start with our other factor. This, by the way, I think would be a fair test question to decompose it kind of the first step and then to solve it from there. This would be hideous. So it would not be a fair test question. But this would be, can you decompose it, kind of do the first step? What would it look like in the first step? And then what? x squared plus one squared. And another one, ix plus, I'm going to put another one. Does everybody get the point there? That it's the nature of the factor, and if it's repeated, we allow for that factor appearing. But each time, if the factor is linear, it gets a constant numerator. If the factor is an irreducible quadratic, it gets some arbitrary linear numerator. Questions on that? So all that's in appendix q, if you chose not to write that down. That's where it is in the book. Let's do a couple of table of integrals problems. I've got actually several. Somebody asked me after class, Daniel, was that you? Was that a web assign? That's a trig substitution problem. Actually, let's go to that. I think I have that. I think it's a problem from the book that's also a web assign problem. Does this look like it so far? X cubed in the numerator? So it's a web assign problem. It's also problem 14 in the 5.7 problem set. This is a trig substitution problem.  This actually isn't a full-blown treatment of that. But if I give you this kind of statement, let me find it for this one. Well, I didn't write it down. I think it's this. Let x equal 4 sine theta. Is that right, Daniel? Okay. So that would be given to you. Now, it's your job from that point to make the substitution, change this from an x dx problem to a theta d theta problem, integrate it, bring it back to x's, or change your limits of integration so that you can evaluate it over the original x values 0 to 2 squared to 3. Excuse me. So if this is going to be given to you, then you need to be able to take it from there. So we need to know what dx is because we're transitioning this whole problem away from an x comma dx problem into a theta comma d theta problem. So we know how to get rid of the x's. Every time we see an x, we're going to put in 4 sine theta. But we also have to eliminate dx. So if this is our choice for x, what is dx? 4 cosine theta d theta. Okay? So now we can do this transition. What web of sine problem is this? I think number 4. Okay. Web of sine problem 4. I did it without shake, so it's going to work. Okay, how did you do it? I just said u equals 16 minus x squared. And I took x squared out of the top and made it just x. x times x squared? Yeah. Okay, yeah, I can see how that would work. And you'll see when we do this substitution and start knocking out some like terms, your integrand will look a whole lot like this one. Okay.  I said I'd let u equal 16 minus x squared. And du equals negative 2x. And then I just pulled out a negative 1 half. Okay, all right, I see. So you're kind of stuck with your radical in your problem, right? You didn't eliminate the radical. No, I just meant u to the 1 half. Right. So his approach is not going to be all that much like what we're going to do because we're going to change what's under the radical so that it is something squared and then we're going to take the square root of that something squared which then eliminates the radical. So they're going to not be as similar as I thought originally. So let's see what it looks like when we make the substitution and see why trig substitution and that's common. Different methods are common for problems. The question is, can we get to the same solution from different methods? The numerator is x cubed so it's going to be 4 sine theta cubed. We've got a dx that technically is in the numerator. We've got another 4 cosine theta d theta that's in the numerator and in the denominator which is kind of the real reason for the substitution anyway. We have 16 minus x squared which is 4 sine theta squared and that substitution is going to allow us to eliminate the radical. We're going to have the square root of a perfect square. Questions thus far? Anybody? Should I write in the limits of integration? No, we'd have to change them if we're going to write them in. Aren't these x values? So I'm just going to put a little reminder to myself that we do have limits on this problem. I don't particularly at this moment want to change them to thetas theta 1 and theta 2. I'll kind of bring it back in terms of x before I evaluate. Alright, 4 cubed and then we've got another 4 is that right? 4 cubed is 64 times another 4, 256 we've got a sine cubed we've got a cosine the denominator is going to be 16 minus 16 sine squared theta is that right? Factor out of 16 if you factor out of 16 under the radical you've got 16 times 1 minus sine squared and what is 1 minus sine squared? Cosine squared, so we've got 16 cosine squared kind of getting to why the trig substitution happens to work at least this particular choice for x, why it works in this problem. So the square root of 16 is 4 the square root of cosine squared is cosine what now? Cosine over itself Cosine over itself is 0 what else? Make it just 64 4, get rid of that there divide this by 4 64 so at this point we have I'm going to bring the 64 out in front we still have a definite integral but I'm choosing to not bring that back until we get it back in terms of x Did we do a problem like this yesterday or the day before? No. Odd powers of sine or cosine what was the little procedure that allowed us to integrate it? Pull out one of the sines and throw it out there to the right change what's left to cosines, right? We have a sine squared and a sine sine squared we can change to minus cosine squared I think your method is looking pretty good right now man, kind of got there a little quicker than this but this works and I do want to do an example of trig substitution so we've already made one substitution, right? to get it to this point because we started with x's now we're in thetas I think either we can kind of think through the next substitution or we can actually write it down it's pretty early in the semester let's write it down so I want to let I don't know r the cosine theta and if r is the cosine theta what's dr? So it looks like we need what, negative one here and a negative out in front so this part at the end is all dr and this is 1 minus r squared does that look good? how many of you are doing this kind of stuff in your head? a couple that's a good early goal in this class for you to kind of not have to write this down and say well that's this is sine theta is the derivative of cosine I've got those I'm going to integrate them get back without making the substitution but it's fine so we've got 1 minus r squared dr what's the integral of 1 with respect to r would be r in the integral of r squared with respect to r r cubed over 3 we've got a negative 64 out in front now let's see if we can track our way back everywhere there's an r we should replace it with cosine theta and then to get rid of theta and go back to x we'll probably need a little diagram for that and you'll notice a part of the triangle we're about to label will be similar to a term that appeared in the original integrand I really have a hard time thinking of a time when you use trig substitution that that is not the case so from our original substitution here it is equals 4 sine theta isn't it true that x over 4 equals sine theta is that right so let's label our diagram we'll find an angle theta and then we'll label it as such that when I take the sine of that angle theta we're going to get x over 4 so there's theta so how am I going to label this triangle such that the sine of this angle theta is x over 4 opposite over hypotenuse opposite is x and hypotenuse is 4 does that say the same thing that we wrote out earlier in the problem and now we can solve for the third side wouldn't it be hypotenuse squared minus leg squared square root is that a term that you see in the original problem in fact that was the denominator very common that that's going to be the case so we need to substitute for cosine theta and cosine theta right so what is the cosine of theta this square root over 4 yes and there's another cosine theta so we want to cube that sorry that should be over 4 right yes that whole thing is going to be cubed over 3 now we can bring back our original limits of integration because those are x values kind of a mess right so we substituted a trig substitution to get rid of the radical we're stuck with the sine cubed we pulled one of the sines out we did a trig integral I think is what those are called by making another substitution worked our way back to the original letters which are x and let's see what we can get I don't think the substitution is going to be an issue so let's not take class time to do that but if we distribute the 64 what are we going to get negative 64 now we've got a 4 down here that's technically if you want to put it where it belongs it's in the denominator right it's in the denominator of the numerator so we can put it in the denominator so there's a 4 cubed which is 64 so what's the end result here negative over 3 over 3 64's should that be plus we've got it integrated by using trig substitution and a trig integral and kind of working our way back I don't think that this is going to be an issue you would put in 2 squared to 3 for x and evaluate get a number subtract from that put in 0 a lot of times when you put in 0 you get 0 not the case here when you put in 0 for x you get the square root of 16 which is 4 you're going to get something numerical in both cases when we put in 2 squared to 3 and squared we're going to get 12 just wanted to make sure we didn't get a negative under the radical which we don't everybody feel comfortable from that point trig substitution eliminated the radical little substitution to deal with and I'm pretty sure that's a web of same problem more trig substitution problems this also is in your book it's problem 11 slightly different from what we just did common to these trig substitution problems are that you have a radical that looks like the original problem it's an irreducible quadratic under the radical we're trying to take the square root of it it's just not going to happen square root of the sum of 2 squares we're kind of stuck with that so the recommendation is to let x equal 2 tangent theta I'm going to give you that now you have to be able to take it from there so tell me now that we've done an example of this type tell me kind of what's next dx what is dx d theta right changing the problem from an x dx problem to a theta d theta problem I guess we just go ahead and do that right numerator is 1 and also in the numerator is dx so there's our dx in the numerator denominator is x squared we'll put in 2 tangent theta for x and then square it under the radical which hopefully this is what is going to simplify by the nature of the trig substitution itself x squared does that look right? let's square this stuff in the denominator 4 tangent squared now some ideas I don't know if this happens when you do a math problem but it happens when I do a math problem I've got all these I'm writing stuff down but I'm thinking too much about what I'm writing down I'm thinking about other things that might come in handy later in this problem for example I see a secant squared not only is secant squared the derivative of tangent but what else is secant squared that we worked with yesterday that might come in handy there's a Pythagorean identity that involves tangent squared and secant squared so if I needed to get rid of that secant squared and write it in terms of tangent squared I'd probably do it same thing I'm writing down tangent squared I'm thinking that may or may not be an issue but I know something that I could replace tangent squared with that doesn't look like a tangent squared anymore so maybe you don't like to do that when you work through a problem but those are the kind of ideas that are going through my mind which I've told you is very finite maybe I shouldn't do that 4 tangent squared plus 4 this is kind of why this procedure should help factor out of 4 and what is tangent squared plus 1 secant squared and what's the square root of 4 times secant squared 2 secant so this is transitioned here now we have the square root of a perfect square so where that radical was we have simplified it down to something that doesn't have a radical anymore okay what next 2's are gone and one of the secants in the numerator knocks out with this secant in the denominator can you factor out can you factor out a 2 and get rid of some of the 4 we already packed it out the 2 this 2 and this 2 are gone with a 4 or not we're going to be stuck with a 4 in the numerator we have a 2 in the denominator we have a 8 that's what I would choose to do is kind of get now that we've reduced everything we can reduce let's get all the other numbers out in front so that's not really a 4 it's a 4 in the denominator which is a 1 fourth and at this point we have a secant of theta up here tangent squared of theta because we moved the 4 out in front as a 1 fourth so we did eliminate the radical that appears to have helped but we still got some things to deal with here split them up into sine and cosine that's what I would try I don't really see anything here that's going to tell me that I ought to change tangent squared to what secant squared minus 1 I don't think that's going to be all that helpful to do that but if I had a numerator that I could split up into pieces then I could put the whole problem into pieces but the denominator into pieces isn't going to be helpful which by the way I need to review this this is going to seem ridiculous and you're going to probably think less of me for doing this but let's review something real quickly that you can do and another thing that looks a whole lot like it that you can't do 2 plus 3 over 11 if you have a sum or difference you have a fraction you have a sum or difference in the numerator this is completely legal right? to break that up into pieces and in calculus especially integral calculus doing that kind of thing is sometimes helpful because the original rational function might not be able to be integrated but when you break it into pieces that's the whole premise behind partial fractions here's something that you cannot do let me go ahead and write that there first because you cannot do 5 over 1 plus 5 over 2 I would love to do that with my bank account I would love to do it I would do it repeatedly, daily several times because what's the value of this side right here? that's 5 thirds man, now it's more than 5 in fact it's 5 plus another 2 and a half I want to do that I want that to be a new rule it doesn't work that way you can't say something that's 5 thirds which is clearly a whole lot less than 5 is going to be more than 5 is that okay? we can't do that so as tempting as that is we can do it when it's in the numerator our number system actually operates this way as much as we'd like for it to in the time it does not operate this way so that's why in this problem I don't think it's going to be helpful to break this up into secant squared minus 1 because it doesn't help us to break a denominator up into a sum or difference so I like that recommendation and I need to learn names that is NOAA that made that recommendation let's change everything to signs and cosines and see what that gets us maybe that'll kind of allow us to because secant can be pretty easily written in terms of sine and cosine tangent sine and cosine and then we've got a tangent squared so let's see what happens so secant is in the numerator which is 1 over cosine tangent squared is in the denominator sine over cosine we're going to square that see if this takes us anywhere I have a sneaky feeling that it probably will so the numerator is 1 over cosine we're dividing by this sine squared over cosine squared so we can multiply by its reciprocal is that alright so are we going to get anywhere get rid of the cosine we can get rid of the cosine that's on the bottom with one of these up here let's see if that's going to be helpful before we go that route you can see the next thing cosine over sine squared yes I like it alright so this cosine is gone and one of these is gone so we have a cosine over a sine squared now there's a couple ways actually of approaching this from this point I like that recommendation and that's going to work if you wanted to at this point change this problem into sine of theta to the negative second cosine theta d theta that's going to work isn't it because this is the du stuff and this is the u stuff this would be what u to the negative second du that's going to work or can I multiply the numerator by one would you be angry with me if I did that's legal isn't it so I can do that and then we've got their product what's cosine over sine cotangent and what's one over sine cosecant we don't use this one a lot but isn't cosecant cotangent the derivative of something what's that the derivative of negative so the derivative derivative of the cosecant let me go back another step the derivative of secant is what secant tangent we probably use that more than this one right don't these work in pairs so if the derivative of secant is secant tangent derivative of its co-function which is cosecant isn't it the negative of the product of the same kind of terms but the co-functions instead so for secant we would have negative cosecant for tangent we would have its co-function which is cotangent so the derivative of cosecant is negative cosecant cotangent so if we put a little negative here in a negative here I think we're in business isn't this the derivative of cosecant so what's the anti-derivative of the derivative of the cosecant it ought to be the cosecant so that ought to work this one ought to work also let's show in fact that they get to the same solution here is u to the negative second du what's the integral of u to the negative second du up the power by one divide by the new power u to the negative first over negative one right? our u here is sine theta so that's going to be negative one u to the negative first is one over u and one over u is one over sine theta is that the same answer? same answer two different approaches but they're both going to get you to the same exact solution now we're not done with the problem but the rest of the problem is stuff we have already reviewed this week so I don't know how redundant we need to get when we review the review which reviewed the former review we're going to get a little redundant here but we started with an x problem we need to end up with an x answer right now we don't have that but how could we get that go back to our original substitution make a little picture of it right or you can make a big picture so x over two is the tangent of theta so if this is theta x over two is that correct hypotenuse is what? square root of x squared is that somewhere in the original problem? u square root of x squared plus four is there so it's probably done the right way at least this labeling of the diagram so from that diagram we need to pick off what? the cosecant of theta everybody feel comfortable doing that? picking off the cosecant of theta from that diagram then we've got an answer in terms of x put a little plus c at the end and move on to the next one there are some other good trig substitution problems at the end of 5.7 one we did not do that if you want a little additional practice problem nine is a good one for trig substitution let's look at a couple of table of integral problems today to finish up here's the table of integrals problem that I asked on my 141 exam almost a month ago alright the directions say integrate using the subset of the table of integrals that was provided so the subset that was provided is actually and this is sometimes part of when you're using the whole table of integrals part of something you have to do but you look for an integrand that kind of matches what we have so here's kind of what you look for 4x squared that's the variable part so that's kind of u squared this doesn't have the variable in it but it's going to be something squared that's going to be an a squared so you're in a in a sense looking for problems in the table of integrals that deal with the square root of variable squared minus number squared u squared minus a squared so on the test I provided a subset and those are the forms that involve this particular guy right here 39 through 46 if you're looking at the table of integrals so all of you don't have table of integrals tables of integrals so this is what I provided on the test we can zoom in on that just a little so these are up at the top forms involving this letter squared minus number squared so it's our job then to pick off of this table which one we have so of those which one looks like this original problem the most 40 so it doesn't I mean it is the longest one but that should be an issue it's a matter of getting our problem to match number 40 exactly so here's number 40 it says if we can get an integral of the form u squared that's variable stuff squared u squared minus a squared du everybody agree that's kind of what we have in this problem it's not letter perfect but it doesn't take a whole lot of corrections and compensations to get it to match this exactly so here's our u squared right do we also have that same u squared right here we don't because this u squared is four x squared and this is just x squared so we don't have a literal match right there which we need to have so it looks like our u squared is going to be four x squared that look good okay you could in fact that's good and some people on the exam did that you've got a four here in here which you could factor out in front and then take the square root of that four in front is two so there are other ways to accomplish this but if we're going to leave things where they are our u squared is going to be four x squared so that's not u squared but we can certainly make it u squared by doing what? multiplying by four and dividing by four so now this is a u squared and this is a u squared so although we had this nice looking match it wasn't a literal we're all doing that this doesn't look like it's going to present a problem but sometimes it does because we've got to have d u and you say well this was an x problem we had dx this is a u problem we have du everything's good let's just finish the problem that's not going to work because we kind of like to get it right so if u squared is four x squared then we know that u is two x and if we differentiated this two with respect to x or we could do well let's just kind of leave it as it is what's the derivative of this? two dx I'm sorry this would be derivative of u would be exactly what I just said derivative of u is du and the derivative of two x is two we're deriving with respect to x right so this is kind of derivative of u with respect to x if you want to get real technical about it derivative of two x with respect to x is two multiply both sides by dx so we want to get rid of dx what is dx the same thing as? it's one half du right du is dx so everybody okay that that is now u squared is that alright? this was kind of how we started the problem that was already u squared that's how we started the problem this is a squared that's not going to be an issue a is a squared is sixteen a is four that will be a part in the answer but it's not an issue to correct for and for dx we don't need a dx we need a I want a du there so I'm actually going to use this statement right I have a dx but what else do I need? I need a two so I can multiply by two and multiply by one half and now we're in business so we have a one eighth out in front we've got a four x squared let's make sure that we now have a literal copy of integration formula forty from the table this is u squared this whole mass is u squared minus a squared and what is two dx? du it's du so don't we have it now exactly yes we had a couple of corrections we needed a four we didn't have it we manufactured it compensated for it we needed a two we didn't have it we manufactured it compensated for so now all we need is to put a one eighth out in front of everything go to the right side of the table of integrals now that we have a literal match and everywhere we see a u what are we going to put in? four two x and everywhere we see an a we're going to put in a four might help if I had a table of integrals so that one eighth is because we didn't have a literal match so that's part of the answer there's also another eight right we've got u over eight two x over eight two u squared u squared is four x squared is that right minus a squared that's sixteen this is the easy part it's the part to get the literal match that requires work now it's a matter of you know every time you see a u you put in a two x every time you see an a you put in a four watch me miss it after I say that u squared minus a squared minus a to the fourth what's that 256 that we don't even know what it is it could be hyperbolic signs and cosines anybody dealt with those in a math class before anybody hyperbolic signs and cosines it could be a bunch of those it doesn't matter what it is all we're doing is plugging in every time we see a u we're plugging in a two x every time we see an a we're plugging in a four u two x u squared minus a squared it's got to be because we're out of time have a nice weekend I will see you on Monday


Okay, so, by this point I think most of y'all understand this format. It's a bunch of books that I don't really have enough to say about them to each give them their own individual review, so I'm just putting them all together. And this one doesn't really have a theme or anything, but let's go for it. This is the introduction song. It's not very good, but it's not too long. Up first on the chopping block is Prey by Michael Crichton. Now, this one is a techno thriller by Michael Crichton. So you already kind of know, if you've read any of his books, you already kind of know exactly what this is going to be like. This one's about nanomachines. There's a factory out in the middle of the Nevada desert that's been making them, and luckily it's far from anybody because a swarm of them got out and it's no longer under control and it's reproducing on its own, so that's not good. And then they bring in the main character, Jack, to try and help them find out a way to stop the swarm before it gets out of control, kills more people and animals. And from there, well, like I said, it's a Michael Crichton story, so it goes into a lot of depth about the machinery, how it works, how it was built, and that's all fascinating, don't can be wrong. Like in this, I'm not sure if it's really a spoiler, but the way that they create the nanomachines is really interesting and involves the use of bacteria and stuff. And when you see how it was able to get out of control, that also ties into it's programming, so it talks about computer programming and stuff, which I'm not all that well-versed in, so I can't really say if it was accurate or not, but knowing Crichton it definitely is. And the thing is, you kind of know if you've read any of his books, whether you'll like this one or not.  And this one is the same. The only real difference I can think that it has is that it's told from first person POV, like it's from the perspective of a guy named Jack. And the first almost 100 pages or over 100 pages is mostly about his domestic life, because his wife is one of the people that's working on the nanomachines, they're initially made as a piece of medical equipment basically, so they're supposed to be injected into the bloodstream and then they can look around in your heart and stuff for cardiovascular issues and that's really cool. But yeah, we see his wife and how they're having marital problems and how he's lost his job so he's a stay-at-home dad for a while, and while we do still get hints about what's going to happen and how some shit's going south, I did find myself wanting the story to hurry up and pick up and once it does pick up it gets a lot better, trust me. But the fact that I was ever sitting there going, okay, come on, let's do this, let's hurry up, let's get the plot moving, that is unusual for a Michael Crichton book. But once it does get going, really good, really solid adventure, nothing mind-blowing I don't think, but if you're a fan of his other stuff then definitely check this one out. Okay, so this next one is actually a did not finish, but I had had it on my to read list for a very long time and then once I actually got to reading it, it was, I guess a word is disappointing, but it's praxis, the dread empire's fault. Now I'm a big fan of space operas, you know that, I rarely find any that are, that I like all that much, but that's just because I'm kind of picky. But this one had an amazing setup, basically what it is is there's this gigantic galactic empire ruled by these immortal beings called the Shah, and the Shah, like I said, they live forever, but they can still die if they get shot or catch disease or anything, and they've had this massive empire where they're ruling over a bunch of other races including humans, because this is a couple hundred years after humans were taken over, and the Shah have been slowly one by one committing suicide, and by the time, we don't know exactly why, it's just they feel that nothing should last forever or something like that, but they are leaving instructions for after they're gone how to run the empire, and by the time the book starts there's only one left, and so when I heard that, that immediately drew me, and I was like, yes, let's have some shit happen, let's go, and then it immediately goes to the life and times of the main character, Gareth Martinez, and I'm not saying that you shouldn't have a little bit of time before everything gets started to get us into the world and get us into the main character's lives and what they're like, but my god, this one was, I got over halfway through the book, so it was, I think I got around 250 pages, and nothing had happened, like the summary of the book, at least on Goodreads, literally says that Gareth uncovers a plot from the Naxids, who were another alien race who were ruled over by the Shah, the Naxids were apparently planning to take over the empire, and over halfway through the book there had not even been a whisper of that going on, like there's a slow pace and then there's nothing happening, and this one is the second very much, so maybe you've read it, maybe it turns into an amazing space opera, like 10 pages after I stopped, but the fact that it has that high of a barrier to entry is a massive problem, and I don't think I can really recommend this one to anyone, even if you are a big fan of space operas. Next up we have The Darkest Minds, now this one, this one kind of got swept up in the, you know, young adult dystopia craze that was happening after The Hunger Games got big, and I think that's kind of a shame because this one, while it isn't perfect and while it's not that different really, it does definitely go out of its way to try and be a little bit different, because this one is all about this girl named Ruby, and she lives in the United States when this plague goes around that only affects children, and most of the kids who catch it die, and a small percentage of those that catch it get powers, get superpowers, you know, some of them get telekinesis, some of them get mind control powers, some of them get, can shoot electricity out, it's it's a, okay, it's kind of stupid, yeah, I'll admit, especially because they admit later that only the United States was hit, like every other country has only had small outbreaks, and that's just, okay, that's that's kind of dumb, in fact that's really dumb, but other than that, I liked this setup because all the kids with powers wind up going to these concentration camps, and Ruby is there for literal years, and she has to hide her powers, because she actually is what's known as an orange, which is, because they give color coding to all the people that have powers, and she's an orange who is someone that can control minds, which is super dangerous, so they usually get wind up getting a euthanized, or I don't know if euthanized would be the right word, but they kill them, and so she has to hide that, and eventually after she's been in there for years, she manages to escape, and after that it's just her going on a journey with some friends that she meets, and trying to, well, trying to not die, trying to not get caught by the government and the people who want to send her back to the camp. Now, as I said, this book doesn't do a whole lot that's new, but it did intrigue me, you know, there is a totalitarian government that is running everything, at least running everything in the eastern part of the US, because they mention that on the western coast there is a different government that is not abiding by the rule of the eastern government, so that's probably going to come up later in the series, but they mention that their world has not completely been destroyed, but huge sections of the country have been abandoned because, you know, a bunch of people died, and there's been a lot of fighting and rioting and terrorism and stuff that's happened, the economy has collapsed, so people have mostly left the smaller towns and gone to bigger cities, and I kind of liked that, because rather than being, you know, dozens or hundreds of years after an apocalypse, it's we're kind of in the middle of an apocalypse, and that's not something you see a lot, so I thought that was really interesting.  She has, you know, a romantic relationship in this, obviously, but it's actually, well, there's actual chemistry there, you know, it's kind of weird that that's rare enough that I have to mention it, but yeah, her and the dude, his name is Liam, they form a genuine relationship, they actually like each other, they aren't just constantly arguing and being at each other's throats, you can actually tell that, yeah, this is why these people would like each other, and I'm not gonna give it away, but the end of the book is really heartbreaking for what it does to them and their relationship, and they don't go too quickly and just drop the L-bomb, you know, they've only known each other for a couple of weeks by the end of the book, and so it makes sense that they wouldn't just come out and say, I love you, and they don't do that, so I was super grateful for that, and this one, it is kind of shaping up to be a standard dystopian young adult thing, where they rebel against an oppressive government, and we'll see if that changes at all in the next books, but I am actually pretty excited for them, and I will probably read them at some point. So then after that we have A Darkness Forged in Fire, which is book one of the Iron Elves. Now this one, I was intrigued by this one because I saw it at a used bookstore and I had never heard of it before, and I like doing stuff like that, I like going through like old cheap paperbacks and finding stuff that I've never heard of, that no one's ever read, and then just picking it up, because a lot of times, yeah, there's a reason no one's ever heard of it, it's because it's crap, and other times it's, you can find a little hidden gem, and I have found hidden gems before, but this time, yeah, this time I found crap, so A Darkness Forged in Fire, this one, I'll admit I also didn't finish this one, but I did get over 180 pages in, and the whole time just, I was kind of waiting for things to start happening, and I was like, okay, can something start happening now? Can something start happening now? And then I realized I was 180 pages in, and I realized, you know, I'm just a little, I'm not looking forward to reading this anymore, you know, I have to kind of force myself to do it, so this one I might get back to one day, but yeah, I wasn't a fan of it, and the plot of it is basically just that there's this big, you know, Empire that rules over stuff, and the main character Kanawa is an elf who was a commander in the Imperial Army, he killed the viceroy of the Empire for reasons that were not made super clear by the time I stopped reading, and so he was thrown out, but then they need him to complete another mission, because this powerful spirit being called the Shadow Monarch is waking up, and so they reform his regiment, and they're supposed to go off and stop her, and so, you know, it sounds like pretty standard fantasy stuff, maybe a little bit modern in how dark it is and how violent it is, but it didn't really do all that much to draw me in, like, because the basic plot is decent enough, I was kind of into that, but it didn't do anything else to draw me in, you know, the world isn't super unique, the characters aren't super great or anything, it just, there was nothing new that was able to get me past all that nothingness that was happening in the plot, so yeah, this one, if you're a super big heroic or epic fantasy fan, you might be able to get into this, but I don't think I can recommend it to anyone, and the last one for today is actually a super popular book that I only finished reading last night, it's called One of Us Is Lying, and this one has gotten basically nothing but praise over the past three years, since it's come out, or three or four years, something like that, and even people that normally just hate young adult and want nothing to do with it have been saying that, yeah, this one's really good, and it's a, it's a murder mystery, it has an amazing setup, I will say, and the solution to, you know, who done it is also really good, and the climax is halfway decent, it's just the way we find out who did it is really boring, because the middle section of the book has basically no investigating. The plot to this one is that there are five kids who are stuck together in detention one day at their high school, and then one of them drinks some peanut oil, seemingly by mistake, and he's allergic to peanut oil, so he has an allergic reaction and starts choking, and then his EpiPen is gone, and then it turns out all the EpiPens in the school's nurse's office are gone, and even when they call an ambulance, they're too late, and so he winds up dying, and so it looks like somebody murdered him, and the four people that are in the room with him are primary suspects, because he, they all had secrets that the kid, Simon, who died, knew about, and he was about to release, so the question comes up is okay, one of us did it most likely, so who was it? And that's, like I said, amazing setup for a murder mystery. The thing is, the story is told from first person POV from all of the characters, and it constantly switches between them, sometimes multiple times per chapter, and man, I've said it a million times, I'll say it a million more. If you can't make the narration between different characters actually feel different or sound different, then it's hard to tell them apart, and you may as well not do it. Like this one would have been much better served by being in third person, because not only is it hard to tell the characters apart in their narration, but sometimes they'll say stuff like, oh, I have a secret, but I'm not gonna reveal it to the reader quite yet, and it's just, it's kind of cheating when you do that. Like you can get away with it a little in third person, but in first person it is cheating, and it's only done so that we wonder, oh, I wonder which one of them did it, and like it just would have worked a lot better in third person. But like I said, after the initial murder investigation starts, the main characters don't really do any investigating of their own. Like they wonder, oh, I wonder who did it, and they talk about it a little bit, but they don't really go off looking for clues, they don't go lead chasing, they don't go snooping around, they're just concerned with their own personal drama, and I get, you can put that in. Okay, in fact I think most of the drama is good enough as it is, but it's just distracting from the mystery, and well the mystery is what drew me in here, it's what drew a lot of people in here, and so by the time we get to the end, the answer is kind of just given to them, and in fact a lot of the answers along the way are just given to them, so it just feels shitty. And last year I read a book called The Cheerleaders, which was also kind of similar, it was also a young adult murder mystery, and that one also focused a lot more on personal drama than it did on investigating the murder, but they did still investigate the murder there, and it was all told from one person's perspective, so it was just her investigating things, you know, she was never really a suspect, so I have to say, but one of us is lying, the fact that it's told from the perspective of four suspects feels kind of gimmicky, but anyways in The Cheerleaders we didn't have to deal with that, and even though it had its shortcomings, I was just so invested in the personal drama that I was able to kind of see past some of the weaknesses that the mystery had, so it wasn't the end of the world for me, whereas one of us is lying, it's almost entirely a murder mystery, and the murder mystery part just isn't that great, so this one, I don't want to say it's awful, because it's not, it's solidly above average I'd say, but I just don't get why all the hype was there, I really don't. So that's all for today, all five of those books, I know I didn't really sing their praises all that much, but you know, none of them were super terrible either, so yeah, please subscribe and all that. So this is the ending part where I tell you all to subscribe and everything, and I thank everyone for watching, as well as thanking my patrons, including Apo Savilane and Alex Humva, Ashley Watson, Ava Tumor, B Quinn, Brother Santotys, Christopher Quinten, Emily Miller, Evan Stigall, Joel, Johnny St. Clair, Madison Lewis Bennett, Ronnie, Tobacco Crow, Tom Beanie, Tofer Wheeler, Vacuous Silas, and Bay Victus, as well as all the other names on there, you know, you guys can read, you know how much I appreciate y'all, especially if you've come this far, and anyways, I'm rambling on a bit, so please subscribe, smash that like button, and I will see you later, bye.


Let's get started. But before I get started, I do want to give one final warning to you guys. This is a talk about defending. I'm not breaking anything. You might have stumbled into the wrong room. So I'm trying to build something here. This is not like an Android malware thing that people are going to patch tomorrow. I'm trying to make something that could potentially change the game in a few ways and help a few problems that are very, very dear to my heart when it comes to defending blue teams and things like that. Also, there's a lot of math involved. I mean, not as much as people would start rioting and throwing things at me, but if you haven't had lunch yet, you might experience some difficulties. It's not good to do math in an empty stomach. But anyway, if anybody is living, I guess we should get started then. Just want to give just a quick note to where I'm coming from. And a bit of this presentation is going to be a bit ranty because of this experience. So I've basically been in information security for like 12 years. And one of the reasons why you've never, ever heard of me is because I did most of this in Brazil. So I used to work with one of the largest information consultancies there. And where I come from, where I did the most, was actually leading socks, putting secret operations centers together, building sim solutions, building log management solutions, either inside of a company or in this huge, massive sock you guys see in the movies and of course in all your organizations where we're trying to make things like day to day of log management and defending work. And after I was done with that, I got into machine learning and I found it to be a very, very interesting subject. And I was quick to try to find ways that this could potentially help the problem that we have been facing. Okay? And this is my first presentation. I do want my shot. Okay? So, man, perks of the job, man, perks of the job. I mean, don't tell my employer but I used to drink before the presentations all the time. That's the only way I could get them. Excuse me. Anyway, here's an overall structure, right? I'm going to talk about the problem. I'm going to talk about machine learning so that I want to be sure that we are in the same level here.  Okay? I just wanted to get the basic feeling. Of course this is not introduction to machine learning one-on-one. We're going to be like four hours here talking about this. Not at all. I don't have the time. It's not that cool anyway. But I just want to make sure you understand where machine learning is coming from, what are special considerations for information security which is something that's very, very important. And most machine learning experts or data scientists or however you may call them, they will not understand this point. And a little bit of the case study which actually goes through the process of one of these algorithms that I have designed. Okay? So let's get moving because we have a lot of ground to cover. First of all, you've all learned of log management, right? You've got a bunch of logs. You've got a certifications. You've got a best practices. It's everywhere, you know? You just have to deal with it. Everything is generating logs for you. But the problem I have is, and bear in mind I've worked with a lot of SIM solutions and log management solutions over the time. And nobody's happy. And everyone's deploying this. They have the state of the art. They're doing everything they can. And everybody talks, I mean, and I say everybody, I'd say, you can quote Gartner, 93% of organizations had breaches they couldn't catch with the state of the art deployment that they had. And everyone is unhappy and looking for other options. I mean, can't be all that bad. What's going on here? You know, I got this great graph from SANS. And I do apologize. They don't have the horizontal axis markings. I have no idea what those numbers are. But that first one seems big. Okay? So I'll take their word for it. So, but anyway, what really caught my eye here was this, is that what's the biggest problem in SIM deployments? Well, it seems I cannot identify key events from normal behavior. Maybe it's a pronunciation. This huge thing is like, yeah, that thing. Security information and event management solutions. They have correlation rules.  They put some graphics on. Try to pretend that you understand what the logs are talking about. That's that. Anyway, the point here, I cannot tell, my biggest problem is that I cannot tell key events from normal behavior. It's like if you bought a car. And your key complaint is that I can't seem to get from point A from point B. I know the car doesn't start. I cannot find the accelerator. This is what we're buying these things for. And we're not able to make them work. One of the reasons why I believe we cannot make them work is because how do you use these things? Okay, you're already coming up with some magic numbers where, okay, I want to be alarmed if someone sports scanning me, if they hit me like five times. Okay, so if I hit you four, I guess I'm home free, right? Absolutely, no problem with that. Or I'm trying to make up these arbitrary rules where if something happened and then other things happened, okay, that's an alert. That's something that should be brought up to me. And in a way, it's all permutations of this. All this 300 and I don't know how many rules people put together. It's so that, you know? And when you're on an engagement, it's actually, you're just iterating on this until the customer is satisfied and you see the strike there. Or you run out of money, which is much, much more likely. And I don't know, it sounds like a Ponzi scheme to me, you know? Like this thing was built in order to make sure that we generate the most consulting out of that. And I don't claim that scene solutions are the hugest offenders. If any one of you ever worked with identity management, okay, these guys, they take the crown. But anyway, anyway. And this is the point I want to make. So one of these vendors, they have like this huge S curriculum, right? You get one month of training. I want to be an expert in this solution. You get one month of training, four weeks, 20 days. There's not a single hour of this training that is about explaining to you what is the content that comes with it. It will teach you four weeks on how to build new rules, on how to create new dashboards, on how to create new things. But I mean, aren't you telling me that you have all these amazing dashboards? How do I use them? How do I apply them to what I need to do? So it seems that things are harder than they should be. And I don't want to touch about behavior rules. I'll talk a little bit more about them in the future. They do help a little of this configuration. But I'm not really to bash the tool, right? So, I mean, some people are very, very good at log management. You see? It's my log management. But there are very few, right? I've worked with teams which are very, very good. You gave them like six months, you know? They would make these things sing. But the problem I have is that this does not scale. We do not have enough people who are good enough at understanding all of that.  But they're not ready. And most people buy them thinking that they will solve a problem, they will be ready. And it's not there. It's definitely not there. What really, really got me scared was big data. Because we are, I mean, the point is we have this smallish databases, okay? So if you think about SIM solutions, anyone here who is in general IT, this is nothing more than a highly vertical business intelligence data warehousing solution. I mean, from the 90s where they never invented columnar databases, right? But now they are starting to catch up. And now you're going to have integration, you're going to be ingesting petabytes of data, petabytes of data per day. Who's going to do that? Who's going to analyze this? We got to create the whole rules ourselves? I mean, usually when you handle this kind of data, you really have to work with statistical analysis. You got to know this. It's a whole different discipline from actually from your traditional information security analysis. And I'm not saying, I've met people who are very good at both, but that makes the pool even smaller, okay? And I mean, given all of this, the only solution I can come here at this stage and tell you about is that we need an army, right? Let the robots talk to themselves, right? I mean, if there's a machine generating data, let's have a machine reading the data for us. We're hopeless. We're not going to be able to keep up. There's absolutely no way. But that's when we start talking about machine learning, okay? And I want to make sure that we get the basics right so that we can understand a lot of what we can and can do, okay? We really haven't got at the skynet stage as of now. I know there's a little bit of discussion, although I've met some people who are really intent of building this, yeah, let's put all this machine learning thing together. It's going to be asthma. Not so sure. But the main point when you think about machine learning is that you're not writing the code. You're not really writing the code for the decision-making. You're writing a code that will identify some data for you and then it will start making inferences based on the data, right? So it's pretty much as if I went to the computer and I said, computer, this is a chair. Chair, okay? And then the computer will look at the chair. Okay, yeah, it's got some metal rims, it's got this stuffing on the back. Got it. It's a chair. But okay, this is another chair. Oh, that's a bit different. But okay, I think I got the idea. And as you show enough chairs for the computer, it will eventually be able to generalize what a chair is, okay? And the secret is all in how you tell the computer what a chair is. And that's the real, in a way, science slash art around this is how do you build what we call the features in order for the algorithms to be able to take this in and make decisions based on what they saw if something is this or that, okay? I'm going to give a lot of more examples. And this is everywhere, okay? Absolutely everywhere. And I think it's a shame that we don't use this more in information security.  You go to Amazon, okay? Amazon is one of the big examples here. They'll use this thing, this technique called collaborative filtering, which will pretty much say, okay, if you bought these things, and there's a lot of people just like you who bought these things, if they bought something else, I'm probably going to suggest it to you. You guys are probably pretty similar. So it will make decisions.  And they come up with, you should be reading this book. It's actually quite awesome. But I mean, this is something that has been studied. It's something that's very well understood by the marketing community and the sales community. When you talk about trading, and this is a very, a very good picture, which is one of the cautionary tales here. This is the flash crash that happened, I think it was two or three years ago, where actually most of our high-frequency trading right now that's run by these large quantitative funds, it's algorithms. They're just talking to each other. And they're selling and buying based on what's happening. And some of them got confused there and almost crashed the whole economy. But I mean, someone was watching the monitor, no, no, no, no, wait, wait, just back up a little bit. And that's, okay, you guys play nice now. Let's separate this fine. But it's everywhere. You know, this is a very sensitive place, what it is. On the other hand, you got people doing some really, really cool stuff with image and voice recognition. So this is actually a picture. That's good. Should I continue? No. All right, so we, we, evidently he's the only one that does not know why we were here. So what are we called? Shot the noob, that's right. So your first time speaking at DEF CON? Yes, sir. Congratulations. Thank you.  You in the blue shirt, because you were faster than everybody else. Sunday morning at DEF CON. You got to love it. Thank you.  Oh my God. It's an evil cast. It's an evil machine learning. That is, that is really cool. All right. Data visualization of some kind. And everybody first time at DEF CON. Cheers. We'll see you soon. Okay, I'm happy now. I guess we can continue. Anyway, what's this care about? So the guys from Google, they actually set up like a, a little 16,000 cluster machine. And they told the machine, okay, find me cat pictures on the internet. Right? They really know their audience, right? So the, they did. And it did. So they use a technique that's called deep learning, which actually creates, I don't know, an arbitrary deep and complicated neural network out of the blue. It's, it's, it's, I don't claim to understand it, but it's, it's mighty fancy.  That's awesome. I mean, I can see a cat there. Man, this is the future, man. The computer is watching cat videos now. Anyway, one or more serious note, okay? What, what is this being used for security right now? So a lot of the fraud detection system, they will use some type of another. The most basic technique they use is called clustering, which I'll talk a little bit more further on. But anyway, they're trying to find deviations on a pattern. So if they can create a baseline and identify where you are in a group of customers, they would be able to, to see if, if you're not, if you're not you in the dimensions that they can, they can look at.  And here's what I touch about behavior monitoring. Behavior monitoring, no matter what people tell you, it's not machine learning, you know? My, my, my, my economics calculator, you know, the, the, the HP 12C can do rolling averages, you know? It's not learning. Rolling averages are very easy to do. So they, the statistical analysis is helpful, okay? But it's, it, it's a first step in understanding maybe a burger scope, but that's not machine learning in any shape or form.  You remember the Bayesian filters? Yeah, that's it. That's actually, that's actually the algorithm that they use. And the point I'm trying to make here is that, how many talks did you see this year? Or one of the past two years about spam? Nobody seems to be doing research on that anymore. I, I know I have my, I have my, my Gmail account. I opened it in 2004. And it's a long time ago. I mean, I'm pretty sure every single spammer on the internet has a hold of my, of my account. I don't see spam. I don't get spam. Do you get spam on Gmail or something like that?  I do get phishing emails, of course, but phishing emails, they are, they're specifically crafted to look like a normal email to get past all of this. This is a problem that we don't really look at anymore, because it took, I don't know, 10 years, maybe 15 years. Actually, I got, I got, I found, I was doing a research for another talk. I found a paper from 98 that talks about some techniques of using Bayesian learning for spam filters. It seems like the problem is solved in a way. And that's, that's one of the things, one of the messages that I want to leave there, okay? Maybe the work, if we start doing this work now, maybe we'll get very good at picking up those stuff in about five to ten years. And this is the power. Once you have enough data, and I believe, arguably Google would be the one who has the most data. We, we probably can agree on that. You can get pretty good at this. Anyway, now we start getting a little bit more, more technical. So the idea here is that there's two big kinds of machine learning. You get your supervised learning where you are actually telling the computer or the program what these things are, okay? And there's two major groups, which is called classification, which I'm telling, okay, this is a chair, this is a table, this is a chair, this is a table, bunch of chairs, bunch of tables. Okay, here's an object. What is it? Is it a chair? Is it a table? So you're pretty much giving it data to train on, okay? And you're giving it labels, which is, this is a chair, this is a table. And you're, and then you present new data in order for it to predict to you, okay? And that's, that's the word that we use, if it's a chair or it's a table. When we think about regression, we're actually not looking for a binary answer, okay? But we're looking for, okay, how much of this is a table and how much of this is a chair? I know it doesn't sound to make sense, but if you look at a stool, or maybe one of those, of those benches you have from pianos, you know, that actually look like a table, but it's like, we, hi. So, yeah, the computer would get confused into, yeah, it's a dot for chair, you know? And, yeah, I mean, it's a very general example, but this is the kind of stuff that we do. We're trying to analyze where this lies. And based on this, we can either use this to, okay, now this is definitely a chair or definitely a table, or we can use this data to make the humans better informed on what the decisions they should take. And then you've got unsupervised learning, which is, okay, I don't know anything about this. Look at this and tell me what you can find, all right? So, you have two big groups as well. You have what's called clustering, which is the one I mentioned before in fraud solutions, which, I mean, it's by far the most abused machine learning technique for anything ever, okay? So, it's not always applicable, and it has a very, very fatal flaw that you have to actually tell the computer how many clusters you're looking for, so, so that it can be able to guess, which would be the separation of the data that you send it. So, of course, there are techniques to discover that, but then it starts getting very complicated and most of the people don't actually do all this legwork, so it can get a little bit fuzzy. And finally, the composition, which in a way is a tool for, for you to design better algorithms in a nutshell. So, you've got, I've got a chair, okay? And I need to tell the computer what a chair is, and I have all these different things that I could tell the computer about, its height, what's it made of, and all these things, and so I get like a, I don't know, a hundred possible variables out of a chair, and I'm trying to find which ones matter the most in deciding if something's a chair or it is a table, okay? It could be argued maybe that for chair and table it could be the height, you know? So, probably if I ran this chair table model on a decomposition thing, and try to give it, tell me that the PCA there is principal component analysis, which are the guys who are really making a difference here, it would probably tell me that the height is one of the guys that should definitely choose for my model, because it really makes an awesome difference. Anyway, by the way, if you like this, this is, this tutorial is, is awesome. So, it's not magic. You still have to train the computer, you still have to, to give it data. But one of the basic principles of machine learning is that if you design your algorithm well, if you design your model well, it will generally get better with more data, okay? So, there's, there's a lot of mathematical proofs that you can show that the more data you have, and this is the, the, the, the drawing on the, on the left, your E in, which should be the, the, the arrow you have inside your training model, which is the data that you know you're using to train will, in both the E out, which is the error of the stuff you've never, ever seen before in your life, they will converge to an expected error.  It will make mistakes. We make mistakes as, as we look at it. And it's all part of the way that we make sense of reality, and we're trying to emulate that into a computer. There will always be errors, and you can always hope and work your model for it to be the, the least that it can be. But the point is, you're all, sorry. Okay. The point is, you have to be careful with what data you're taking in. Okay. So I'm going to give you guys for an example where I've used the sans D shield data, which is this, this viral data that, that you get from them, about viral blocks. And I mean, I, they wouldn't tell me, but I would assume that it's, it's pretty US centric. So there, there is this, definitely this bias on, on the, on the results that I'm going to show you guys here. But you always have to be cognizant of that. And you always have to look out for adversaries. And this is one of the points I wanted to make in this talk, that everyone who is into building these models, and they do the machine learning, they do not understand that people would like to fuck with them. Okay. So I mean, we understand that in a very deep personal level. And one of the first things that, one of the first things that came to my mind is, okay, even if I build something here, how can I potentially exploit it? How can I send some random noise and things like that, and that will render this completely useless. And I'm, I'm going to talk a little bit like that as, as, as, as one of the, one of the weaknesses of this specific model that I built. But I mean, it is there, right? People will try to mess with you. And if, if this, and I really believe it will, if this become a valid method of, of actually defending and helping defense, there will be a lot of talks in this conference with some really crazy math guys on how they will defeat this. It, it will be the crypto wars all over again or something like that. But anyway, the point I tried to make here just to exemplify this, remember spam engines, all right? So when we started getting this Bayesian thing going on, the spammers just started posting whole sonnets of Shakespeare at the end, you know? And then the model would think, hmm, I haven't really seen many spams with the word thou. So that's probably a legitimate email. And that's pretty much what it was. That's pretty much. And then people refined that and people made sure that they would understand the sort of thing they evolved the model as well. And we are getting to the level that, that we're getting to. But people will always find a way to break things. And I think especially on these kind of applications, this is something that we have to be very, very careful about. Anyway, enough introduction. Let's get to it. Okay, let's chew on the logs. And the idea, the most of the talk here is about the feature engineering, which is the really important part here. There are, on that slide about the kinds of machine learning, there were some names of algorithms. There's a whole bunch of them. And you just try them all, you know, you create a process for you to try and see which works best for your data, because there's no one size fit all. The problem is what data you're feeding it. And that's the difficult selection process and what everyone will tell you that is the real hard work around anything on this. Anyway, I was telling you about this shield. And what I did with them is I've been collecting their, their bulk logs. So I mean, if you go to the website, you'll have a lot of top 10 ports that are being attacked, top 10 places, bad people are coming from, from the internet and things like that. But if you ask really nicely, and they were completely awesome about that, I really would like to take the SENS Institute for their help here. You can get the bulk data. And I just started mining it starting January.  And this is very basic stuff. And this is one of the points I'm trying to make. This is firewall data, blocked firewall data, okay, and it's summarized. I get to know how many kind of blocks we got for each one, which I use as one of the ways to decide who I'm going to, to select to the model. But that's all there is, right? But you know that for this group of people who are submitting the log files, these are the guys who were potentially attacking them, right? I mean, you always start with a port scan. You always start with something like that to see which machines are up. So if people were hitting them on the firewall, and this is one of the points I'm trying to make on machines that didn't have that port open, well, maybe if they hit the right port, that would be an issue. That would be something worth looking at. And this is just a summary of the amount of data, okay? So roughly by day, I got a million observations. An observation is pretty much this IP address attacked this port, okay? So from that, I would select the behaviors that I would be looking for. And when you summarize that, sorry, when you decompose that and I didn't get to decompose logs, I just got a number there, you would get roughly 30 million log events per day. And one point I wanted to make out, the thing I wanted to point out here is that this is not big data at all. This is like nothing. I'm running this on my laptop and it's not even a good one at that, okay?  So, yeah, you know it is. So I'm really trying to bring this to a tangible level and this is one of the objectives I have in this talk. So I'm just getting some data and I'm doing mining with it on my own laptop. So one of the intuitions here of this model is the proximity. And anyone who has ever done real sock work, they kind of developed this instinct where, okay, I've seen this IP address before. Or I've seen coming, people coming from that side of the woods before. And it's interesting because I've seen a lot of things being caught out of mistakes. So people would look at IP addresses, oh, I remember this guy. Let's look at this. But actually, no, the guy had never appeared. It was something that was an IP address that was similar to that one. Okay. And you follow the, yeah, there's actually something there. And so I started doing some research on that. And there's actually some anecdotal evidence and some really, really hard statistical analysis facts about this. And one of the things I like to point out is the Spamhouse cyberbunker thing. And I don't mean the DDoS, I don't care about that. But I care about the fact that Spamhouse actually stood up and said, okay, forget about that ISP. Anything that comes from there, it's bad. They're just going to try to spam you. Just block the whole thing. I can't take it anymore. That's an interesting conclusion. And they really fight to be, can I say, level headed. Sorry. I don't know anyone. I don't know anyone. I can't really comment on that. We'll have to take your word. So, but then you got the Google report, the Google malware report, where they said it blatantly, okay, there are places where they're more likely to have malware than not. And there's this paper from a researcher in Brazil called something more. I forgot his first name. He actually did a statistical analysis for the past seven years of logs from the Brazilian research network, the one that connects the universities.  This is not random at all. There is some information we can potentially extract from that. So then what I started doing to create my features, I started grouping those logs by arbitrary net blocks, okay. Let's pick some things, okay. Let's choose and see if it comes out all right. And of course by ASN, which really, which is what really shines, which pretty much the synchronous net, autonomous system they're coming from on the internet. And for that I use a lot of team key services. They have an awesome who is service that you can pull and things like that. So here's a visualization of that, okay. And it seems like a lot to take in and that's understandable because that's the internet up there. And the point here is that this is a projection that tries to maximize the proximity as you draw the IP addresses. So I put the drawing there. This is called the Hubert curve, okay. And this is the kind of transformation that it's trying to do. If you think of the IP addresses as if they were on a straight line, okay, from 0, 0, 0, 0 to 255, 255, blah, blah, blah. I'm actually twisting this like this to make sure that the IP addresses are as close as possible to their neighbors, okay. And this is actually data from accumulated data until the 20th July on people that were trying to attack the shield group on port 22. I don't think that's random. And you can see, of course, some places on this map, they are dark by nature. I mean, the DOD is not really doing anything there, although they have like two slash eight blocks, you know. Even IBM, they have a whole, I think it's eight dots something. They don't really use that. But even on the other places, you can see that there's some density going on, okay. And by the way, if you're wondering, if you're in the death connect work, you're there at the start, right? So I'm trying to make, there is some clustering here. And you can start even if you look at this cluster, you can start to jump to conclusions, right?  And then you start doing some more mining of this.   But the point I'm trying to make here, it's very easy to do this and start jumping to conclusions. And we really have to see this thing to fruition at the end. So if you look at it, what actually happens that it's us, right?  So anyway, I just want to make a point that if you're just blocking for Oh, I hate this country. I'm not going to let it in. That doesn't really mean anything. Just have to be careful with that. Anyway, let's let's let's get moving. So we get this, there's proximity. But I want to be able to decay that because the neighbors, the neighborhoods might renovate. Okay. So I want to make sure that if people are changing ISPs, they're changing their anonymous proxies. I want to be able to accompany that otherwise I just have a massive blacklist that goes on forever. And I'm eventually going to block the whole internet off. So as time passes, I want to be able to forget these things happened and do some sort of exponential decay here, right? So you choose your metric. And you can see that I mean after a few after three to four months, it doesn't really matter anymore. You completely forgot what if people were attacking you, which actually pretty much mimics what an analyst would do, right? You have so much memory, right? You go out to party and things like that. So if something happened a month ago, you're not as likely to remember this as if you were it happened yesterday. So I'm given those two intuitions, we have to start calculating this, okay? And the point here is that we create some sort of rankings which are going to be the features to our model by IP address, by some net blocks that you choose, and by the ASNs, okay? If you're missing data, I took a shortcut that just said, okay, if I can't discover what your ASN is, or if you are a bogan, you're bad, man. I'm just going to put you with a high score, which is a I mean it's a shortcut, but it's just to it's just to make sure that we don't ignore these people, which is maybe would be bad in a scenario like this. And the point is for each day that we have, each day that happened based on the on the time series that you saw, we're going to have a calculation. So this is what happened these days. So the following day, we will actually decay this by the function, the exponential function, and then we add what happened the following day. We're pretty much adding one. So this happened this day, this happened that day, and we go decaying the rest. And the importance of us having this history is that if we're if you're using log data to detect stuff, log data will have dates. So I can't use few future calculations I used with with things from yesterday that doesn't make any sense. The model won't make any sense. And it will it will tend to survival, survivorship bias, which is I think that something happened in the future influences in the past. And that's a very rookie mistake that you can do with this sort of thing.  And this is just an example, right? So the vertical scale is actually in log scale.  It's a million, right? And if you get one on this score, which is the horizontal axis, it means you were hitting me every single day. So I mean, I I found it pretty interesting that you could get at least another 10,000 guys from this is three, three, this is RDP that were hitting me every, every single day. And even if I was using that as a blacklist, which is a perfectly valid way to do this, look at all the guys that I'm ignoring. That doesn't mean anything compared to the rest of the data that I have. And I have another example here on 22 when you can see pretty much the similar behavior, right? There will be some guys who will be able to get you. But if you only look at the daily blacklists, there's so much information you're leaving behind, you know, that you could potentially leverage to have help what you're doing. We good? Too much math? Yeah.  I'm sorry. What's all right. All right.  It is time to drink. I can wait a second. Did we already share dusha with you this morning? You did. No, not this morning. You just came to this room before you can do another one. There's no problem. Matt, he wants to do another one. I'm sorry. Is that a Russian accent I'm hearing? No, it's Brazilian. Really? Yeah. Oh, my God. Wait, just wait. Maybe we just want to drink. I don't know. Some of the audience. So wait, wait, what? See, maybe I saw you first time right here. You've done a shot. Yes.  It's great. All right, for the second time, we've got a few shots. Wait, wait, what's your Steve? Clearly, we've lost track at this point. Steve, this is everybody. Everyone first time. It's Andy. Here's to see. I guess we'll be back in five minutes. Okay, let's make sure we get close to finishing this. The point here is we've got a bunch of numbers, right? But we know if this bunch of numbers that these guys yeah, sorry guys, we should do a drinking talk. Anyway, so we get this so we get these features are calculated and we pair them with the data that we had. So the assumption here is if someone hit the firewall, we they're all to get you and they should be considered bad. Okay, so you put these guys on your training model. So pretty much the IP address, the features that you calculated and you feed that to the you feed that to the model. Okay. And you usually have to take more than one data, more than one day. I'm sorry. And that's what that's why the separating the ranks makes sense on daily basis because you want to make sure you're pairing the right date with the right data that you're using. All right. So rule of thumbs and all depends on the algorithm that you're using. Okay. The point is you can't always have malicious data. You have to have good data as well. Otherwise, the computer just say, yay, everything is a chairman. That was an easy job. So the point is you got to find something I did. I took a bunch of IP addresses from Alexa from from chromium to make sure that we we we got enough data to pair it up with. So it was at least 50-50. So as far as algorithms, I'm not going to go into this because I don't have enough time. But support vector machines are awesome. Okay. They do math that people don't even understand about. Okay. It's like the the the mathematicians. Well, I think there is an infinite dimension where this makes sense. I don't care. I'm just going to calculate this on the matrix and it should sort itself out. And it works. Scary.  Anyway, here's the point. Okay. So we train this every day. Okay. So I got a bunch of ports. Okay, what's happening on port 22? What's happening on port 389? What's happening on port 25? And I would get something around 20 83 to 95% of training accuracy. And accuracy is what I got right. So bad is bad and good is good. Divided by the whole number of things that I gave it to to process. And this is good. But it's it's not really accurate because especially if you're using time sensitive data, what it is a technique called cross validation, which doesn't really work very well for that. So what I did to actually train it was okay, let's look at the following day. So if I have all and this is the point, right? I did all this calculation for today. Who are the guys who are most likely to be attacking me tomorrow based on all this data that I got? And I would get something from 75 to 85%. And I'm going to break that down for you so that you guys can really understand what 75 79 79 to 95%. Man, this is hard. Anyway, just just an idea of the progression. Okay, you have to run. Of course, if you have a model from February, you try to run it from from something that happened in July, you're going to have a terrible time because a lot of things has happened from them. And it also illustrates a little bit about the moving about of the of the of the environment. And blah, blah. Here's the point I'm trying to make.  So here's the point I'm trying to make. Okay, it's 79 to 95. Okay, what's the good? What is the best stuff I got bad? And what's the good stuff I got good? So it's true positive and true negative.  And I think once I get different data, it should reduce a little bit, should even it out. The point I'm trying to make here is that if you calculate this, given this, this error rate, which means this, something that this picks up on your logs and tells you to look at is about 13 times to 18 times more likely to be attacking you than all the rest. Okay, so the point here is that wouldn't you have a randomness look at that first? I'm not saying I'm this is not catching everything. And neither are the analysts, you know, spoiler alert.  Labor loss. But let's try to make this a little bit easier, right? And this and this is where this is coming from. Okay, so this is an idea of prediction. Okay, so I'm just simply it's the same curve you saw before. I'm sampling 100K IP addresses from each place. And just making sure so the brighter the tile, the more likely it's for people to be attacking you from that. Okay, and this is a logarithm rate, so it can go like 10, 10 to a thousand things like that. Anyway, these are the deal guys, you see it's, it's everywhere. Anyway, challenges IP addresses are bad. They're the worst stuff you can have to try to do real incident response. But it's, it's just like point I'm trying to prove here that even with the real, really shitty stuff, we can actually get some interesting results. Okay. But I mean, anonymous proxies, okay, it holds well. But Tor, there's not a lot of clustering. If you're coming from Tor, it starts messing it up a little bit.  But I believe that if you can reduce the cycle, so this is this is a daily resolution. You could start getting smarter about that. But then I wouldn't just go on IP addresses. I would go on different stuff. So anyway, what am I trying to take this? As it is, it could potentially help security analysts. Like I said, it brings a new priority dimension to a SOC work. You want to do the like, okay, these are the most important assets, but these are the guys who are most likely to be able to get me. So yeah, let's, let's see who are the guys here and let's invest some time in that. What I think it's really cool is that I created a model for, for firewalls, right? I could do exactly the same thing for IPS. I could do exactly the same thing from WAFS. And if I start taking the inference that each of these individual guys is giving me on these IP addresses, and I combine them, I could become to you and say, okay, this guy has 200 times more likely to attack you.  You know? So, uh, and this is the kind of confidence that we're trying to build. And, uh, I don't know, blocking is a very bad word in information security, especially if you're defending, but maybe we can get that confidence. And the way I'm trying to do this, there's actually a project which what I'm pretty much is begging for data.  I'll send you reports based on what the model is sending back. And the point is that the more data I have, I can start fighting the bias that I got from the, from the, from the sense database. Okay? So there's some URLs, there's a Twitter feed. If you look at, if you look it up on, on Google it might come up, not sure. But anyway, I'll be, I'll be around. Uh, anyway, take a ways. Machine learning is cool. And it can help. Okay? It's not a monster. Of course, there is marketing hype and stuff, but it can really help. And I think 13 to 18 is, is pretty, pretty good. Thanks. Uh,