Thank you Robin and welcome all here to this brown bag about tools that we make here to to make our life easier at Enviso. So if you're familiar with my presentations I usually take questions during my presentations but here because of technicalities we will do that at the end but I will make sure that we will have time for questions. Now also another thing, if you already watched this presentation a year ago, about a year ago I also made this presentation, know that this one here is a different one. So the slides are the same, I haven't changed the slides, but the demos and that's the main part, the demos are completely different. So if you have seen this last year, well you won't see it again this year. So I'm Didier Stevens, I'm a senior analyst at Enviso and one of the things that I do at Enviso is malware analysis and then also in my personal time I like doing malware analysis and I also do that for Sense at the Internet Storm Centre. Now what we are going to do today is tell you about some of the tools that I have and then explain the design principles that are behind and where you can find my tools, where you can find me and also how to start making your own tools because that will be part of one of the demos that is a small introduction on making your own tools to solve your own needs. I have more than 103 open source tools and I have them all documented on my blog and I have to document this because otherwise I wouldn't remember all of the tools that I have. So you can imagine if you have that many tools that you need some system to organize them and one of the methods to do that is on my blog. You have several entries on my blog here. This page for example gives you an alphabetical list of all the tools that I have. So for example one of the tools that we will use today here is B64 dump. This is a tool to extract B64 strings from files. That is one of the tools that is listed there. Okay so without further ado let's get started with the demo. So I have here my command line and what I'm going to do is the following.  Now what I'm going to do here now first is show you what you could do if you don't have any tools. So my malicious word document is here inside sample.zip. That's a file that was given to me and by the conventions for malware analysis it is in a password protected zip file and the password is infected. So I will extract this. Now this is already the first thing that you shouldn't do. This is something you should do in a virtual machine or in a sandbox that you shouldn't do this on your own machine. Now I'm going to extract the file here. See I need to type a password infected and now I have the word document. Now if you don't have any tools the only thing that you can do is open this with word. Again that's something that you shouldn't do in a production or test machine only on virtual machines or sandboxes that are well protected. So I open this and as you can see as with typical malicious documents that contain executable content like VBA, macros there is a warning here above. Security warning macros have been disabled and I can click enable content to execute it but again that's not something that you want to do when you do a static analysis. What I want to do here is now look if there is VBA code malicious VBA code inside this document. So let me do this. So I can go to the VBA editor by pressing Alt F 11 and now here I am in the editor and as you can see I can see some code here. Now this is code that I prepared this morning. It is pseudo code it's not actual real functional code but I made it kind of pseudo code so that it is better understandable. So I have auto open here auto open. That is the name of a sub protein in words that will execute automatically once execution is enabled. So there is no user interaction required for that macro to run and you can see here execute shellcode is executed and then message box infected is displayed. Execute shellcode what does it do? Extract shellcode and then you have a base 64 payload decode base 64 payload with as input to the base 64 payload and then you have shellcode and then run the shellcode is executed. If you look at the individual functions like extract shellcode you see that there is no actual code I kept it simple there I put in simply a comment to tell it what it does. So what does extract shellcode does? It extracts the code the shellcode from the comments property. So that is something that you would see in the VBA code like application document if I'm not mistaken and then properties like comments so okay so you have this open and then you understand yes the payload is inside the comments so let's take a look at the comments. So I come back to words file info and as you can see here now I have the properties comments and here you see this thing this is base 64 so this is our shellcode that is base 64 encoded and this is something that is typical for malicious documents that is one of the places where the height payload what do you have to do then here since you have no other tools well you select this and I copy this now I have the base 64 code now I need to decode this again you need a tool to do this you could for example use Cyberchef a very useful tool that can do all kinds of decoding say you have it online or you can save it locally here I am going to use an editor binary editor the 010 editor so I'm here in this editor I create a new file and now I say edit paste from base 64 so this will do the base 64 decoding and then paste the decoded shellcode into the editor like this and then I see this yeah this probably doesn't make a lot of sense to you but this is the actual shellcode usually my font isn't that big but I made it bigger here in the in the preparation so that it would be easier to read now a lot of things that are not recognizable but then you see something like here user agent Mozilla so there is a string inside here and if you go at the end you can also see an IPv4 address 185, 205, 210, 179 now if you have shellcode that is a downloader and it's not obfuscated then you will often find the URLs or the IP addresses at the end of the shellcode in clear text and that's what we have here this is an actual malicious sample this is an actual shellcode that I took from one of our diary entries on the sans internet storm center so I have this file here it is in my editor I can now for example look at the strings that are inside so let's do this if you go here in the search function you see here find strings and then I click find strings find strings that are minimum five characters to a find and here you see all kinds of strings it finds but here you see the user agent string that is found so that is one of the methods that you can do here again if you don't have any tools so this is how I actually had to do it when there was a revival of malicious documents so by the end of 2014 there was a revival of malicious office documents with VBA code they were very popular at the end of the 90s and the beginning of the 2000s and then they declined and then by the end of 2014 there was a new revival and unfortunately for us I call this a revival but it is still going on end of 2014 that's a bit more than six years ago but we are still dealing with malicious office documents so what I started to do back then because this is not practical at all and also quite error prone you can easily make a mistake and infect yourself and like for example if I go back here to the document and for example I click by mistake here enable content then it executes and you then see my machine would be infected because of the message box so that's not a good way to do it and what I started to do back then is develop a tool to help me with the analysis of office documents and now I'm going to show you how to use that tool I have a command line here and it's in folder one here that I have my sample so I already have my doc that I created let me delete that one because I just extracted this okay so I already had from the late 2008 2009 had PDF tools because back then PDF for malicious PDFs were very popular but now their popularity has declined a lot now it's office documents so I started to develop a tool to help me with the analysis and that tool is OLE dump it's a Python tool you run it in the command line and to execute it you give it the input now here I will just give it sample dot zip run it and you can see already I have output now what is happening here and that is one of the design principles of my tools that is that most of my tools can handle samples that are stored inside a zip container and even with a password protection and password infected here my tool I developed it it's one of its features that it recognizes when something is inside the zip container and then it will try the password infected so it managed to do that it will then extract the document the word document in memory it doesn't write anything to this it does all of this in memory and then it displays you the analysis of this document now I'm not going to go into the internal details of a word document but this is a binary document so a doc and which is an OLE file and an OLE file is actually also kind of container that contains items of binary data these are called streams and that is what you see here this is the list of streams from one to 11 I have 11 streams in that document and the streams that are indicated with an M they contain VBA microcode so these are the first streams that you want to look at how do you look at that streams well you run the tool again so remember this year is number seven this number is not actually something that is present inside the document inside the OLE file but it's something that my tool adds it's an index so that you can easily select it so yeah let me already type the command here sorry OLE dump select seven so option S seven this will select stream seven and then again I give it a sample and then it will select that sample and do an hexadecimal asking dump of that sample for you now that is not very useful when it comes to VBA macros because they are compressed inside that stream so we want to decompress them and that's what you do with option V VBA decompression because it's a compression specific for VBA for office when you do that you see exactly the same code as we saw when we opened the word document so this is the advantage here when you have to do a manual analysis like this is that you can do this with a tool that is not relying on office you don't need office or anything you just need Python and then you can use this tool and you can see all the code again then we do our analysis here and we understand that it is in the comments property so the next thing we can do is go to look at the metadata and with option M uppercase M you can look at the metadata and here you have the metadata and you can see the base 64 string not only the metadata but also my user account that I used here to create this test user also the data that I created and safety so you can see it was I prepared this example here this morning and for the ones that know me I'm not an early riser so I didn't make this at 724 I'm not Michelle or something like that I'm still asleep at 724 you see this because this is in UTC the data that is displayed here by my tool is in UTC in Belgium here we are in summertime so we are 2 hours ahead of UTC so this was done at 924 this morning ok so coming back here now we have our base 64 so exactly the base same base 64 that we had here in 010 editor now I'm going to analyze this base 64 string and I'm not going to copy paste this again I'm going to use one of my tools what I'm going to do is use a very powerful feature of operating systems it started with Unix sorry not Linux it started with Unix as far as I know and that is that you can pipe the input the output sorry of one command into the input of another command and that's what I'm going to do here and you do that with the pipe character and so that line that vertical line that is the pipe character and it means the output of only dump pipe that into another tool now I have the metadata here that I want and now I'm going to do pipe this into one of my other tools base dump 64 base dump 64 here well it's actually base 64 sorry it's a tool that can do base 64 decoding and also many other decodings hexadecimal and other kinds of decodings it's not only base 64 dump but it does not only do the decoding it will also search for such strings into its input so it will search for anything that is syntactically a valid base 64 dump with the input that you presented here so you can see there is a lot of output now so each string that it recognizes is listed as an entry one two three and so on you see for example there is code page that's one of the words that's inside there syntactically code page is a valid base 64 string but that's not a base 64 string that we are interested typically our base 64 strings are longer than just eight characters and that is an option and that illustrates one of the features of my tools that is I have many options to change its behavior with option and you can say a minimum length that you want it to process so we want here at least 10 bytes and then when you run the tool you can see that you only have one input sorry one output and one string and that is our base 64 string here you have the start of the value here a start of the decoding and here the md5 hash so if for example you could take this md5 hash now directly and look it up in various tools to see if it is present there and then continue your analysis there now if you don't like md5 if you say now I want something stronger then it's something you can also configure in my tools so I have here that base 64 now I can again do the same principle select one and then again the two tools will run so all of them will do the analysis again extract the basics the metadata that contains the base 64 and then with base 64 dump we see the base 64 so I select that base 64 string the decode base 64 string and I'm going to do an ASCII hex dump like this and now you can see what we actually saw here in 010 editor that is the code it's hexadismal ASCII representation and here you see for example again the string so the IPv4 address now when I have this I can also do a binary dump like this and you can see what can you do with the binary input well you can write it to disk so for example shellcode.vir that's one of the things you can do or you can again pipe it into another command so let me first clear the screen like that and now for example I would like to extract strings do a string extraction so I'm going to use the strings command from sysinternals and you will see unfortunately this will not work you get this message what is actually happening here is that the strings command from Mark Rusinovich from sysinternals that string command it does not accept input from standard in it always expects a final standard in so that's unfortunate but I found this to be so useful that I started to code my own strings in Python so with strings.py I can do this and then I have this output I could also pipe this into another tool for example this assembler the netwite this assembler tell it that it's 32 bit the code and then sell it at the input is standard in with the dash and then here you see the decoded basics disassembled shellcode now reading shellcode that's not easy you need skills to do that now this particular shellcode here that I took again from the sense internet storm center diary entry this one is actually coming from and the cobalt strike shellcode is very similar to the meterpreter shellcode and I also developed tools to help you with that so I'm just going to run it not giving any more details but if you pipe it into that tool here that's specific for cobalt strike and you have the dump but then here as you can see it will give you information like the IP address it will extract the IP address also the address sorry the part and here to use the agent string that's what I wanted to illustrate here with this demo that is that I make tools first of all that I don't have to use the actual applications that are vulnerable that I am not relying on existing tools that I can do this on different operating systems I'm doing this here on windows but this works on any operating system that supports python and then I also do this in the in the Unix Linux philosophy that you have small tools with options that you string together so let's come back to the slides so how does it all started with me in the 80s I was using an Apple II computer and back then there wasn't that much software if you wanted to hack that computer and by hacking I don't mean breaking into it because we are talking about the 80s personal computers of the 80s there was no security at all because there was no need for security back then there was no internet it's only later in the 80s that I had a modem connected to the BSSO I mean there was no security because security was not an issue by hacking I mean making my own stuff in that computer so I had to develop my own tools so that's the origin that I started to make my own tools and then later also in the 80s I started to use Unix I got familiarized with Unix and I was really impressed really by that system of having commands small command lines tools together and that's what I tried to replicate here later also I started to use the Windows CS internal tools at the early 2000s and that was also a source of inspiration to me now why do I like to do this well first of all the very main reason is to solve my problems so back then in 2014 and later on I was working at a place where I had to analyze malicious documents because they were coming in through email so I had to make my own tools so that I could do the analysis without relying on anything else a very useful thing about making your own tools is that you have to research and learn for example when I started with PDFs back in 2008 I didn't know anything about the PDF language the underlying internal structure I had to figure that all that out and by trying to make a program that understands all that you have to understand it really well so that you can code it so that's also an advantage of making your own tools another thing is that making your own tools also gives you some kind of documentation of what you are doing if I come back to my command here so this command here if I copy paste this into a document for example with notepad here and paste this and then save this as a text file I have already documented the exact steps I took for the analysis I could for example add the hash here of that file and have a bit more documentation so that part is already easier to do to document compared to the first manual method that I presented that you are using the tools itself and then you have to really take screenshot for examples and write down what you do and that's also another thing is actually for me writing code is relaxing so when I'm tired or stressed writing code for me it's just an easy way of getting to relax I can't assure that it will work for you it's an individual test but with me that's the case some of the design principles behind my tools I want them to be generic, solve more than one problem and that's who options I make them flexible by providing options I make them portable I try to make them without the least amount of installation required, Python and then sometimes one library but I try to keep it like that also that they work on different operating systems that they are not tied to one specific operating system modular that I can combine tools and reuse tools let's do a second demo what I have here now and let me also take the explorer view so what I have here now is something that I started to see at the Internet storm center a couple of years ago and from time to time it reappears and what is it, it's malware that is emailed to a victim but not directly but it is stored inside a container that container is an ISO file a DVD file so the binary file that represents the content of a disk, a DVD that is an ISO file, that is something that wasn't feasible to do by the malware orders by the attackers before the arrival of Windows 10 because until Windows 7 you had no internal tools in Windows to mount ISO files but since Windows 7 you have that so I have that ISO file now to analyze this back then 2017 I didn't have any tools so what I did back then was use the command line version of 7zip I'm going to use here the GUI version of 7zip but if you open this file you can see here two files inside that ISO file beacon.dll.vir and invoice.doc.lnk so this is an ISO file that contains something malicious so you want to look inside and do the analysis here I would for example extract this now what I'm going to illustrate here is how you can make with a couple of lines of code how you can make your own tool to do this and for that I'm going to start with process binary files now since a couple of years I have published I have open source two templates, Python templates so that's actual Python code that I used to make my other tools so often if I start a new tool I will use these two templates. This is the template for binary files and you also have a template for text files. Now this template for binary files in itself it already does something if you give it a file like this it will read that file and do a hexaski dump of the first 256 bytes of that file so that is already in that code I can show you that here with the editor so there's a lot of code in here but the code that you actually have to change if you want to use my template to start your own template is this. So this is just a code that does what we saw there, well processing the data and then doing a dump you can see here the line that does the hexaski dump what I'm going to do now is replace that code to do something with ISO files that we can read ISO files. For that I have ISO dump but I will show that later I have here a very first version of that tool ISO dump where ISO and as you can see when you run it it just gives you the name of the two files that are inside so let's take a look at that sample so this is the code, you can see it's actually just three lines so what I'm using is potlap potlap is an open source tool a python module that you can install with pip pip and potlap will allow you to read different kinds of container files like for example ISO files and you do that through the ISO accessor so with that module imported I can already start analyzing ISO files and what I needed to give it is the file the input that I gave it and in my template that is data so if you are here data that is the file that was read and it will also work with standard input all kinds of features and also the zip file with password protected you have all that features here in the template now you have to do some conversion for potlap I'm not going to go into details here but that is some conversions to necessary for potlap that it is actually a file object and then once I have that file object I can say I'm going to the root directory so slash the root directory ISO part and there I want an iteration of that directory so the content that is here a list of output files and then I just iterate over those output files and I take the property part and I print that out I don't use print you can use output dot line and then you also have options for your outputs that you can redirect your output through command line and other things so you see here just three lines to achieve the result. The next step is that we want a bit more information so not only do we want the name but we also want the size and the index so let me show you the lines the code for that here is that code so that first line it's just the same the second line has changed a bit now we are enumerating the O files if you are enumerating enumerate then you not only you have each object but you also have an index an index from zero to the number of objects minus one and then I will here do an output line to print the index the size of the file and you can have that to start size and that is information that you find in the documentation of patlap you can find documentation and read the docs and then you can see which properties you need to access to have the size and here the part and this is my index now remember I start from zero but here I like to display one so I am going to increment this with one index plus one and that is all there is to it this to achieve this result so I am going to add some more information to see the size for example not only the name next of course we want to extract what is inside there we want to look at those files so that is step 3 if you run example 3 you have again the same output but now I also if you go to the help I also included the option what I can do now is say, sorry, select I am going to select two that is a small file and then you immediately have an extra day small ASCII dump of that file that was selected so let's take a look at the code and how to achieve that so here I added one line for the select option so I added that option and then now the code as you can see again it is quite similar so here that line hasn't changed to get the content of the file of the ISO file and here now I test did I select anything if I didn't select anything so if select is an empty string then I do just what I do before list the content if select however is different then what I'm going to do is convert that select value that I gave to an integer subtract one from it because objects in Python are indexed starting from zero and then here in my files, my object files that I have to convert to a list here I'm just going to index this and I'm going to say I want that all elements in the file in the list here of items I want that element so in my example it is two minus one so that's one so integer is one and then I select the second file in that list and when I have that second file here again if you read the docs you just have to do read bytes and then you have the content of that file so that variable content and then I just do an ASCII dump like I did in the previous example so and this is then what the tool allows you to do look at the inside of the list of files or have an ex ASCII dump so with just a few lines here by using this template and I also have a more complete version of ISO dump that I will release now so you give it the malware it gives you this output you see it also displays the name now so the code is a bit different this one is also recursive so if there are directories and nested directories it will also read that content and then you can do a select again of number two and then you have an ASCII dump but you can also change this and do for example a binary dump like this if I do a binary dump of that LNK file here and I look at the strings inside I can see rundll32.exe beacondll.vir this is intentional so that it wouldn't execute I added this so you can understand that this link file contains a link file and when the user clicks on this then this link file will actually start rundll32 to execute the DLL so let's take a look at this DLL so the DLL is the first file so select one I do a binary dump and I'm going to look at the strings so this is a quick win often strings will be obfuscated then you will not find any useful information but it's also a quick win you can also try it and here you get the output, a lot of output and then you have to browse through that output so that's not so convenient and one of the features that I have added to strings is to sort the output by string length so that I have the longest strings at the end because often the longest strings are the ones that you are interested in when you do malware analysis not always of course, always exceptions but that's a useful trick and let's do this so now it is sorting the output by string length and now you see this, you see already something readable now a tip here if you see a lot of lowercase letters I like here, here that end there is a very long sequence of letter I lowercase letter I if you see something like this you might be dealing with a cobalt strike beacon because the configuration of a cobalt strike beacon so the behavior of a cobalt strike beacon is encoded in a configuration and that configuration is stored inside that configuration is binary representation and it is also XOR encoded but with a simple key either a lowercase e or a dot and so if you have a lot of values I then you might have a cobalt strike configuration with a lot of zero null bytes that have been XORed with I and so you end up with long strings of I and of course the name beacon that tells it too so let's pipe this into my tool 1768.py, this is my tool that I developed to analyze cobalt strike beacons, the name 1762 is a small joke, 1768 degrees Kelvin is the melting point of the metal cobalt and that's why I called my tool like this so let's try this out and then indeed you get output like this so you see here all kinds of information like the C2 this is your IP address and then the part it will use for GET requests, this is the part it will use for POST URISE this is the USE Regents string that the beacon will use it is an HTTP beacon it will connect to port 4321 so with that tool you can quickly analyze beacons if you are indeed dealing with beacons and that is the tool that I also started well at least it wasn't public when I did my first brand back here on this topic, but now it is public this tool and so it is based on my process binary file template I started with that template to make this tool here, of course to do all the decoding because there is also different obfuscations that tool is quite more complex than the ISO dump that we saw here so where can you find the tools information, well first of all on our blog, on EnViso we have a lot of very good writers there on our EnViso blog post I invite you to take a look I also write blog posts there from time to time on the SENSE Internet I am a handler at the SENSE InternetStone Center there you can find my tools on my blog and then also on my YouTube because I also make recordings of my videos the software you can find on my blog, on my github also on the EnViso github and also on other tools because I didn't mention this yet but I think it is in the description my tools, some of my tools are also used in SENSE trainings and that is why for example you will find it in Remnux distro by Lenny Seltzer to do reverse engineering of malware and also in Kali you can find my tools and before I will start taking questions I also would like to make a call here to an invitation to start my own tools, even if it is difficult to get started take a look at my templates and start with that as you saw here it doesn't take many lines to get started to do an ISO dump tool. Of course you need to understand what you are doing, you need to find that library to parse ISO files you have to read the documentations, try to understand it to try an error but the great thing about doing this and trying to make a code to do that is that you will far better understand the inner workings, the internal format of ISO files through that library that you are using. Ok so let's see for the questions now Yes, thanks a lot Didier, we did have a couple of questions Let me share them as well so people can follow along in the presentation Alright, so I can handle the first question actually, will the session be recorded? Yes, the session is recorded and you can watch the recording simply by going to the URL you originally used to join So is the file so simple available anywhere so we can follow along a bit late but it's not available now but we will make it available and that you can use it and then you can just follow along with the recording and also it is based on the Sense Internet Storm Centre sample so you can also find them the description on the Internet Storm Centre but what I wanted to say also is that what we try to do now also at the Internet Storm Centre when we talk about malware or samples if we are at the liberty of sharing that sample we will upload to Malware Bazaar and then you can find it on Malware Bazaar and you can download it from there, I don't think you'd even need an account on Malware Bazaar, you can just straight download it if you have the hash or the link No, I usually use Pile Tools, I have some Pile Tools but as far as I know I didn't publish them and it's actually a compiled version of Ole Dump, B64 Dump and a couple of other tools so that I could use them on a Windows machine where Python couldn't even be installed we had to do that on Python and then if you go to our Enviso blog you will also see that we made a Python Decompiler written in Python so Python code, well actually not the Decompiler itself but an interface to a library that does the compilation and the thing about that library is that it changed regularly and by changing I mean breaking changes so you do an update with pip and yeah it no longer works and then we decided to actually compile that tool into an XA and if I'm not mistaken we also compiled it into Linux so that compilation would contain the complete modules complete libraries that are dependent on it so that it would always work even if there was an update well if there is an update of course you wouldn't have the update but you would have the old version that still works okay when you encounter obfuscated strings do you tend to reconstruct them or do you try to tackle the file in different way skipping strings analysis it depends one of the tools that I have will do an XOR small XOR brute force and small XOR dictionary attack it is called XOR search sometimes I will use that and then also yeah the thing is I regularly do malware analysis I do it often and I can also just recognize some of the obfuscation schemes by seeing them for example a very simple obfuscation scheme that I can see easily through is that if you add you have your string for example a URL and inside that string you will add a lot of characters that have no meaning whatsoever into a URL so let's say for example what can I say yeah an asterisk let's say that we use asterisk so at random places you put one or more asterisks that URL to do the obfuscation that's one of the techniques that they do well I can sometimes just recognize that and see that and when I see that then I will just use for example TR the translate tool from Linux TR to just filter out the asterisk and then I will see the clear text URLs so it depends I will not spend a lot of time on string analysis if it doesn't lead to anything okay do you have tools for binary firmware analysis no I did have tools for iOS and by that I mean iOS with an uppercase so the Cisco iOS it is called the network analysis forensic toolkit but I no longer maintain it because I don't have any because I don't have any new well I don't have any new hardware that's just it so that tool can do some analysis of Cisco firmware and also memory dumps but for firmware analysis you can for example also use my base 64dm tool to look up all kinds of encodings there are many encodings in that tool not only base 64 so sometimes that can also help with firmware analysis but I don't have sorry your last sentence was broken a bit for me so I mean yeah so no I don't have specific tools for binary firmware analysis except for iOS but that's no longer maintained so Cisco iOS but you can use my tools base 64 to do some simple things like looking for encodings like base 64 these kind of things okay alright thanks a lot that concludes all of our questions so I want to thank you for presenting did you I want to thank everyone for joining as well I've put the link to your scripts on the screen as well if you have any more questions about this talk or in general you can always reach out to us on our Twitter and on our LinkedIn as well these talks we organize these talks on a wide variety of cybersecurity topics almost every Thursday at noon next week we'll be hosting Michelle Kuhne where he will be talking about purple teaming and these talks are organized by Mvisio we are a cybersecurity company based in Europe we offer a wide range of cybersecurity services so if you're interested in that you can find them all on our website mvisio.eu we're also hiring so if that's something that you're interested in you can find that as well on our website so once again thank you everyone take care oh before we leave I saw here some last comments coming in but it was a thank you message not another question alright so we can end the stream here thanks everyone thank you didier


Hi, I'm John Little and you're watching the Introduction to R instruction series. This series is part of the R Fund Learning and Resources website sponsored by the Center for Data and Visualization Sciences at Duke University Libraries. In this session, it's going to be a quick demo of how you can do a data analysis project in R using RStudio. All the topics I'm going to cover will have longer videos if you want to dig into more detail. So quickly, let me explain what I'm going to cover. We're going to take a comma separated value or CSV file, move it into a folder on the local operating system, make an RStudio project, import that data into the project using an RStudio markdown notebook or an R markdown notebook, we'll introduce libraries, attach some onboard data, do a quick visualization, implement some other exploratory data analysis. Then we'll do some quick data manipulation demonstrating that the plier library, which is part of the tidyverse, will make an interactive chart, do some quick linear regression and save the notebook. So let's get started. First on my local file system, I want to make a new folder. I usually put my projects in my Documents folder, so I'll just type new folder, and it'll be down at the bottom. So I'll make a new folder called Test. Into Test, I'm going to make a new folder called Data because that's where I want to store my raw data. And now I'm going to take that CSV file, save CSV, and I'm going to drag it over into that folder. So I'm going to put it into the data folder. The other thing I'm going to do, I have a script to keep me on task. So now in my new directory, I have a data folder with a data file in it, and I have a script that I want to use. So I'll launch RStudio. I'll go to New Project, and then I will make that existing folder my RStudio project. Notice you can make a new directory right from here, but I have an existing directory. It's in my Documents folder, and it's called Test. And I'll click the Create button to make that project. First thing you'll notice is there is the quick start R&D file that I have moved in there, and there's the data folder, and inside the data folder is that file. So I'll go back to the root in that directory. Let's open up a new R Notebook. I'm just going to go here, File, New, R Notebook. I'm going to put my name here, author, date. I'm going to get rid of all this information, but it's useful information for beginners. Load the library, and then import some data. So I'll make a second model header, and I'll type, um, before that I'm going to put a little explanation. Okay. Give my project a name, test one, and save it. Notice as soon as I saved it, a derivative report file got reported, got created. That format of that report file is identified right there as an HTML Notebook in the output field. All right. Load library packages.  I'm going to type library. I'm going to turn the warnings off on this because I know that there will be warnings. If you're new to this, I recommend you not do that, but for me it's going to be easier. I'm going to use a couple other libraries. All my libraries that are available to me have already been installed, and they're available right here in packages. If I needed to install a library that I didn't have, I could click on install and type in one of the libraries available to me and install. Okay. So library tidyverse for data manipulation and visualization. Library skimmer, which is good for skimming data, for EDA or exploratory data analysis. Library plotly, which allows me to turn my visualizations into interactive visualizations. Library modern dive, which will give me some convenience functions to look at my regression analysis. And I'll type library room, which actually does something similar to modern dive. I probably won't use them both. Okay. So I'm going to load my libraries. And then I'm going to attach some data. So import. Remember how I told you, or you watched me import this move data into this data folder using my file system, my Windows Explorer. So I'm going to insert another or code chunk. And I'm going to create a project called fave data. I'm going to type the assignment variable, which you can read as gets value from on my keyboard. I can type the alt dash, but you can type by hand less than dash. I'm going to use a data import wizard read underscore CSV. Double quotes and hit my tab folder so that I can, my tab key so that I can use tab completion and navigate to the data folder. My tab completion again, and there's only one file in there. So it completed it for me. And when I run this bit of code, I'll get an object up here in my environment variable. That will be the import of this data. And now I know that I have a data file with 14 observations and two rows. Now it might be interesting for you to know that you can, you don't actually have to have the data on your local file system. If it's accessible on the internet, you could have another, you could have another object name and import more data just by putting in a URL. And the skip argument skips the first 11 lines because in this case the file has information about where the data was in was originally found from 538.com. So I can run that and now I have two objects that are actually identical in my environment. I also want to attach some onboard data because we loaded the tidyverse library and the tidyverse library includes it's really a mega library that has it consists of importing or attaching eight other tidyverse packages or sub packages if you want to call that one of the packages we're going to use is Deployer, it's all time data, Star Wars part of the Deployer package. So now when I execute this kind of junk, now I have all three. So let's go ahead and look. So that's the Star Wars data and favorability data. They're side by side. Favorability or data is a favorability rating for Star Wars characters. There's 14 of them Star Wars data. There are 87 characters with various bits of information about each character. A quick visualization. In this case we're going to use ggplot which is another one of the tidyverse libraries. And I'm going to paste some code in here. And you can get this code from the GitHub repository plot object. The plot function. We'll get value from ggplot function where the first argument is the data frame Star Wars which we just loaded. And we have to identify and map the x-axis to a variable inside of Star Wars hair color. You can see it right there. We map that with this argument. The aesthetics maps x to hair color. And then we visualize that with a g on bar to make a bar plot. So all that creates an object named plot. It has this function in it. And then when we call plot that function gets run. And if we execute this code we see this. I'm going to make my screen bigger. Let's just do one improvement. Control I for the keystroke to create a code chunk. You could type it out by hand. And in this one improvement I just want to sort the bars from the most frequent to the least frequent. Using another tidyverse package called four cats which makes vectors which enables you to make vectors into categories. And so I'm doing that right there and using the forecast function called that factor infrequent, which will order the bars by the frequency. And when I execute this code chunk. You can see that I'm now a slightly better bar chart. It actually needs a fair amount of work, but this is a quick overview. Right. Going on with my exploratory data analysis. Let's insert a code chunk. And use the skimmer library to skim star execute that. And we get some basic information about this data frame some that we already had like their 87 observations and 14 variables, or 87 rows and 14 columns that we go back to the full view and you can see that right there, right. So we get some calculations 14 variables. It tells me some other information the first character data list data and numeric data gives me some information about the character variables and information about the list variables and some information about the numeric variables, along with some spark graphs down here at the end. Help me see the distribution as a histogram of the numeric data. Another way to get a summary. Insert a code chunk is to use these are just type summary. I'll use the other data frame and I have availability. And that is a data frame that only has two variables, one called favoring and one called name tells me stuff that's nice to know the name. The first variable is a 14 length character vector and the favoring is a numeric vector and it gives me the quartiles in the medium in the meat. All right, so let's join those two data frames together using a require function called left join. Star Wars data frame using a pipe. We say and then left join left data frame the Star Wars. The right data frame is favorability. We're joining by the variable called name, which exists in both data frames. And then we're just using a select statement and an arrange statement to present the data easier to read fashion. And what we can see is we've added variability favorability rating. Or eight of the 87 characters in the Star Wars data set. All right, let's transform this data further with the, with the five most common supplier verbs. And first we'll introduce the select data chunk of a select function. And select from Star Wars. If we just look at Star Wars by itself, right, it has 14 variables and we want just four of them. So we can name them. You can also call them out by position one, column one, column two, but let's just call them by name. It's easier. Actually, we're going to select three of them here. Name, hair, name, gender and hair color. And there we have a three column data frame subset by the columns. The subset by rows, we can use the filter statement. So in this case, we're going to say gender equals feminine. We use a double equals sign for equality. And it's a character equality. So we put the characters in quotation marks and we'll get things like Leia Organa and Beru Lars. Everything under gender is feminine. We have 17 characters including Leia Organa and Beru Lars. We insert another code chunk and this time we'll sort the data using the function arrange. Star Wars and then arrange in descending order by height, subrange in descending order by name. Organa arrange by this column. Tallest to shortest. And any place there's a match. And in the same value, we subrange in this case in reverse alphabetical order. So T on comes before Rupert in that case. Now we use mutate, which we can use either to add a new create a new variable or to modify an existing bear. So command alt I insert my code chunk based in some data and the mutate function is listed right there. Do a little bit of modification. So the sub data frame is subset to look like that. And then I'm going to create a new variable called big mass, which is equal to which gets value from the function mass times two. We introduced the count variable as it's actually a special case of a summarize function. But I think it's the easiest way to understand how summarize works. So we'll start by count. Count allows us to create subtotals of variable values. So we could start with the Star Wars data set and count how many characters are listed by which genders. And here we see that we've got 17 characters listed by feminine. 66 characters listed by masculine. And four characters where the gender is not listed. All right, so that is, as I said, a special case of summarize. Summarize is a way to get essentially column totals. So here we're going to take Star Wars and then we're going to drop in the mass variable. Any NAs from the mass variable. And then we're going to summarize by summing all of the mass variables together. And what we'll get is a column total that tells us that of all the characters in the data frame that have mass, they totaled 5,741.4 kilograms. And as you can see, it seems to work similar to count. But you can do more with summarize than what you can do with count, because you can do groupings, special groupings that work for you. For example, I'm going to put in just a subset of the code at first. If I drop the height and then group by gender and subgroup by species, I can then summarize where I have mean height, which is a new variable, which gets value from the function mean of the variable height, or gender and species, and another new variable called total, which gets value by counting the rows that fit those groupings. So when I execute that code chunk, I now have two new variables. I have my groupings, gender, and then species. And I have the mean height for each one of these gendered species and the total number in each of those categories. Now, if I just add a few more lines of code, I can clean this up and make it easier to look at. And so I'm going to list species first, gender second, and I can learn that I have 23 masculine humans with a mean height of 182 centimeters, 8 masculine, feminine humans, 160 mean height centimeters, and you see the list goes down. So let's make an interactive plot. This is using the plot.ly library. You remember earlier in this broadcast, we made a function called plot1. We can call that again, and it looks like that, and it's a nice plot. It needs work. It needs an x-axis and y-axis and a title. Could use some color. It could be converted with x to y and y to x by flipping the coordinates. That would make it a lot easier to read, but then we need to order it a different way. But the point that I really want to show you is that you can make any ggplot object interactive using the plot.ly library and wrapping the plot in a function called ggplotly. And now I have what looks like the exact same bar chart, but this chart, if you include it on a website or an HTML notebook or something of that nature, is interactive. So it has pop-up windows that tell you things about each bar. And you could also zoom in on a particular section in the chart. And if you double click, you can zoom back out. So that works really, really well in dashboards. So let's talk about regression. We're going to do a simple linear regression. In order to do that, we'll create a model. It gets value from the function lm for linear regression. So the response variable is mass. And it will be predicted by height, the predictor variable, or the explanatory variable. Notice that we identify the response variable with the tilde. It's on the left, and then tilde, and then height as the predictor. We have to identify the data frame. And in this case, I'm subsetting the data frame using standard teddyverse conjunction right there, and then filter mass is less than 500. That allows me to leave out the job of the hut, which will make my prediction more robust, although technically incorrect. So model gets value from the linear equation. And then I call model to display the value of the model. I can also then type the summary model. Let's start using some of the deep dive functions. Broom has similar functions. I'm not going to recommend one over the other. I think they're both very good. In this case, we're going to use the get correlation function to get the correlation of mass over height. And we can see the correlation value. The get regression table function, which is similar to the tidy function in Broom. I can generate a nice table that has the information I need to know, or that I need in order to make a useful assessment of this linear model, so that for the average height unit, the average mass will increase by 0.62 kilo. And then I have a whole table that includes standard error, statistic, p-value. So I can see that in this case it looks like it's significant. Now there's some other model data that I'd like to get. In Broom, you could use glance. In modern dive, you can use get regression summaries. So you can see, for example, the r-square and the adjusted r-squared statistic p-value again. One more nice tool that you have in both Broom and modern dive. In Broom, it's called augment. In modern dive, it's get regression points. So you get data for each point. We had mass and height. Now we have predictive mass and the residual difference. Lastly, let's visualize that regression. And we'll do that with our ggplot function that we learned before. So star wars and then filter where mass is less than 500. And then send that to ggplot where the x-axis is mapped to height, y-axis is mapped to mass. And then we're going to use the geogitter, which is like geop point. That allows us to minimize the effects of over plotting. And then we will also use geosmooth. Let's just run this layer first. There's the plotted results, mass over height. And if we run the whole equation, we can also draw a regression line with a confidence interval or showing where the standard error. One more thing, talk about reports. Up in the amyl header, I have a standard default output for an R notebook. And the output is HTML note. When I click preview, we can go back here to the working directory. As I'm operating on the test one markdown script, when I click preview, it will render the test one notebook, which is then a notebook that I could attach and share with somebody. If I click, I could scroll it here, but it's a little hard to see. I can also scroll here and see the report for the document I just made, which includes, by the way, the interactivity. So I can share this file with somebody just like I would share a Microsoft Word file. In addition to that, from this practice report, let me just note that there are many different kinds of rendered reports you can make. Notebooks like we just did, you can make a more polished HTML document. These are basically the same ones for development, ones for production. You can make a Word document, you can make a slide deck with Sheridan, you can make dashboards with flex dashboards or shiny. You can make ebooks. You can make web or which I also would think of, you might call it a web book with something with a package called book down. You can make websites with a package called blog down. You can make more simplified websites with a package called distill, or you can make a PDF document. And in fact, there are many other types of documents you can render. The point I want you to take away is that that is managed up here in the YAML header. All you have to do is configure your system and follow the directions. So that was a quick rundown. Anything we're missing, please feel free to go back to subtopic videos, which will drill down deeper and move a little slower on various aspects that we just covered. Thanks for watching.