Thank you. So I will talk about Kirigami mostly about design of applications done with Kirigami and why we did it. So nowadays on a standard Plasma desktop, most of the applications that you run are still done with QWidgets like they were done in the past 20 years. Why? One, because that used to, so there is a lot of legacy and two, because QWidget is nowadays still the best tool for some kind of, some kind of jobs for very complex applications. You still probably won't use that. But since it's a non-technology, of course, it has many limitations graphically what you can do in the main Chrome of your application. It's pretty limited unless you do very complex spaghetti C++ codes. And that's of course bad because it limits what you can do. But on the other hand, it's kind of good because with this implementation it kind of enforces a coherence between all the different applications that then comprise the user experience of a Plasma desktop. But this is going to change now most of the efforts in the UI side from the Qt company is towards QML that is still kind of young. It still has its own different set of limitations. For instance, everybody that tried to have keyboard navigation on an application just work out of the box knows what I'm talking about. But on the purely graphical side, you pretty much don't have limitation. You can do everything you want and that's of course excellent, very, very good for creativity. But if we don't have a very coherent and even strict design process that could be in the end bad for the end user experience because it would bring many applications that are done with conflicting user patterns. So we ended up thinking about both a design language, a set of human interface guidelines and a framework with the same name with the named Kirigami to have a pre-made set of solutions for very, very basic usage patterns that should be ideally across most of our applications bought on desktop systems and on Plasma Mobile with slight adaptations for of course the input and screen differences that are between a desktop and a mobile system. It had already several standalone releases, but since from the next framework release in August it will be released together all of the KD frameworks pack. So this exists because designing a full feature application is even both for the developer but also for the designer it's a very massive task and it's very easy to do weird things. I'm not saying wrong but things that in the end will not really do user a favor. What can happen we can just look at history on a certain proprietary platform where in the past 20, 30 years many applications were completely their own thing. They looked like their own toolkits, their own operating system almost and I think that was actually a part of the whole Windows is bad and Windows is ugly. So we have potentiality for doing beautiful things. We have risks of repeating a tiny bit of that history so let's not do that. To have a weird parallel, a technique in writing, actually a writing book is to limit your own lexicon in order to have a text structure that flows more and what we are doing we adding this artificial limitation that it's completely opting of course to make design less of a massive task because all of the problems of the base patterns are already solved and to put in a really cheesy way in order to think outside the box, you need a box in the first place basically we are doing the box then you can smash if you want. To conclude with a very fast example, Coco is a very simple image viewer written in QML that was written some years ago, then was a stand abandoned for a while, now has been resurrected as part of a summer of code project by Atul Sharma which I am mentoring this year and this is how it looked that it was absolutely fine but it had some problems and it never really had a designer input so we went with Jens and the rest of the VDG to look about what the alternatives could be in order to make it as simple as possible and really a textbook case for Kirigami so in the end we ended up with some wireframes for how it could work and behave for both the mobile case and the desktop use case, this is how it looks nowadays it's still really a very early work in progress so there are still several obvious missing pieces and layout problems but it's getting there, on the left is how it looks on Plasma Desktop with the Breeze style and also with our experimental Qt Quick Controls 2 style that will paint everything like it's Breeze that will be released with the next Plasma Desktop release and on the right is how it will look on Plasma Mobile. Last thing for Kirigami our first main target is Plasma Desktop then Plasma Mobile with the nice side effect that also works pretty well on Android as well though even if it's not our primary platform we care as much as possible to respect the look of the platform, the look of the design, not so much the feel because being our primary target Plasma Mobile we want to innovate in some areas that we think that current mobile applications are not very good because they hinder it too much from legacy for instance most both on iPhone and Android are very top heavy UIs that make it quite hard to use them on nowadays big oversized phones so we can do a bit better in that but apart from that it will look as at home as possible and thank you very much. Thank you very much for watching and I'll see you in the next video.


In today's exercise, we're going to be focusing on getting your horse aware of their feet. And to do this, we've got poles placed in a manner where they're laying perpendicular to one another with one set of poles on top of the other set of poles. And what we're doing is we're riding just simple figure eights. And in your figure eights, you've got options. So the options would be riding your figure eight where it's just a simple figure eight, or we can ride a cloverleaf pattern, alternating the directions we're having our horse go.


7 Great Resources for Teaching Business English. Business students are most likely already familiar with using computers, and particularly the internet for learning English. The internet can provide a teacher with an enormous array of teaching resources, from non-authentic and authentic materials, to lesson plans and ideas. Here are 7 great resources for teaching business English at no cost to the teacher. 1. BusinessEnglishSite.com. 2. StudyTeacher. 3. Annual Reports. 4. InfoPlease. 5. Voice of America Learning English. 6. BBC Learning English. 7. UsingEnglish.com. Many of the resources mentioned above can also be extremely useful for online classrooms and tutoring environments. If students are using the internet to research a particular company, product, or even career path, they could also use the internet to present results from their research. These presentations could provide valuable listening and speaking opportunities for the students, and would enhance their online learning experience.


Okay, let me pick up on a couple of points and focus on those. I really have no idea what the word success means given its use in the last few minutes. I'm an economist, that's what I do for a living. So one of the ways we measure success, it's an imperfect measure, but we use it all around the world, is economic growth, the growth in your GDP, your total output of goods and services. So I want to compare with you the freedom you get if you're no longer poverty stricken, if you have minimum health, welfare, welfare, health, food, shelter, clothing, you're not a terribly poor person. The People's Republic of China, I'll use as an example because they call themselves socialist. Over the last 40 years that they've been developing their society, their annual rate of growth has averaged between 6 and 9%, particularly over the last 25 years. Over the same period of time, the rate of economic growth of the United States has averaged between 2 and 3%. That is, the Chinese are growing three times faster than we are as a nation, and that's why they're the superpower in the world that you see and read about every day. Over the last 30 years, the real wage of the United States has been stagnant, growing very, very small. Over the same period of time, the real wage of the average worker in China has quadrupled. They promised their people to deliver to them an exit from the lack of freedom that goes with extreme poverty. An extreme poverty that China suffered because in their capitalist division of the world, particularly over the last 300 years, what was left to China was the bottom of the barrel. Every Western capitalist country took a piece of China as best they could and didn't use it for the benefit of the Chinese to put it politely. And then let me talk about freedom and the experience of socialism. The Kibbutzim had a problem. Every system has problems. For every socialism that had a difficulty, I'd be happy to give you a capitalism that did. One of the most successful corporations in the last 70 years, for those of you who may not be familiar, is something called the Mondragon Cooperative Corporation. It's a company in Spain. It's the seventh largest corporation in the country. It's a family of 150 to 200 worker co-ops that reject capitalism. Why? Because when you enter a capitalist enterprise, you're entering a place from which all that individual freedom you've just heard about is pretty well gone. A very small number of people at the top of that enterprise tell you what to do, how to do it, where to do it, and when to do it. And if you don't like it, they'll take the job away from you and you can go home and be unemployed and suffer through that with your family. Lots of freedom you have in a system like that. You don't go to work in a democratic space. You don't go to work in a place that recognizes your individuality. And for those of you who have ever had a real job in the world that I live in, know very well what freedom isn't when you're working on the job. And allowing you to leave and go to another place of work where you'll likely be treated in the same way is hardly a solution. Freedom is the yearning for people to do better than what capitalism has in store, to provide you with an education without crippling debt, to provide you with the necessities of life without humiliating you and subordinating you. And to do that, you cannot be asked to go work in an enterprise whose whole internal structure makes a mockery of the notion of democracy. Look, it is absolutely true that it is a disgrace the rate of growth in the United States as being so pathetic. It has. And the reason for that is the lack of capitalism in this country. The fact that it is so controlled, so regulated, so much redistribution of wealth, so little role for the private sector in this country that economic growth is stilted. But to associate China's growth with socialism is bizarre. China grew when it liberalized whole regions of its geography, whole areas which were left free. Free to have wage laborers, free to build corporations, free to build businesses. Alibaba is not owned by the employees, although now, of course, because it's against socialism, it's being crushed by the central government. The successful businesses in China, the ones that are responsible for the economic growth, the ones that are responsible for the export of all pseudo private companies that behave like private companies in a relatively free economy. And many businessmen in the United States have claimed that in China they can do business with less regulations than they can in the United States of America. So to posit China over the last 40 years and to ignore China when it was truly socialist and where tens of millions of people died of starvation. I'll give you one short story because I think it indicates a lot even though it's going to eat up most of my five minutes. How did China go from a position of starvation to a position in which it can actually feed its population? It's a little village. A little village and I've got its name here if somebody asked me in the Q&A I can pull up the name of the village, it's in my notes. This little village in 1978 decided because they were starving, because they had communal farms and everything was grown for the party and for the community and there was no private property. They decided to create private property. They had a document in which they assigned different pieces of land to different families and within a year they were growing four or five times more than they had grown as a communal farm and they were scared because they knew the communists would come down and they would probably shoot them all and luckily for them at that point Deng Cha Peng decided to be a pragmatist and not insist on communal farming on socialism in farming and he said wait a minute this is amazing. You mean if they divide these parts up into private property food production goes up four to five X maybe we can repeat this in other villages and slowly this idea of private property of employees and employers of owners spread through China and production of food went through the roof a country that was starving its people was suddenly feeding them and more. So no China is no example. It's not an example of capitalism is again it's a mixed economy but with the balance of some socialist elements and the growth is all happening. If you look at the numbers the growth is all happening in the private sector. Look the state of the world today sucks to put it in technical terms. It's not a good situation where we just went through COVID we went through financial crisis. It's been a bad time for a lot of young people. The solution to that is not to go backwards in terms of economic success the solution is to go forward the solution is to look for a system that can enhance our lives that can make our lives better better what I mean by better living longer fewer children dying having more wealth getting greater education comparing the somewhat partially capitalist world with any other world socialist otherwise in terms of success over the last 70 years that is success. So we should reject today's system but to move towards socialism is suicide and it's unoriginal if you want to be a radical if you want to be somebody who thinks differently thinks outside of the box don't be a socialist people to be socialist last 150 years to know the veil the veil was pretty destructive be a capitalist embrace an idea that is new that is radical and that is actually focused on individual human well being flourishing freedom happiness success. Thank you. Thank you for listening or watching the Iran book show. If you'd like to support the show we make it as easy as possible for you to trade with me you get value from listening you get value from watching show your appreciation you can do that by going to Iran book show dot com slash support by going to Patreon subscribe star locals and just making a appropriate contribution on any one of those any one of those channels also if you'd like to see the Iran book show grow please consider sharing our content and of course subscribe press that little bell button right down there on YouTube so that you get an announcement when we go live and for those of you who already subscribers and those of you who already supporters of the show thank you I very much appreciate it.


this is how to shot web, better web hacking and mobile hacking in 2015. I know Dan Kaminski is talking right now, so I really appreciate everyone being here. He's way smarter than me. This is me. I work for bug crowd. I'm the director of technical operations. I manage a team of hackers who validates the bugs behind the scenes of a bug bounty, large scale bug bounty program. In 2014, I participated as only a researcher. I didn't work for them yet. And this talk is about my methodology that I used there to do web hacking as well as a little bit of mobile stuff, as well as stuff I learned from other researchers while doing this work. So what is this really about? It's just how to hack stuff better and practically. And I put a lot of memes and some of them are not funny, apparently, my wife says. So it's okay. So more specifically, what I did is I started off with my methodology, which was a normal pen tester methodology when I started doing this work, a basic web application assessment. And so I then went out and manually parsed out all of the public researchers of all the bad bug bounty hunters I knew. So there's about 150 people that I knew just around the Twitter scene, not Twitter scene, sorry. Twitter scene as well as people I just knew who were good at it, right? And so I went through every single article they had ever written from the beginning of the crowd source bug bounty scene. And also all of the Google and Facebook programs that I could find, like enterprise based bug bounty stuff. And I created a presentation around what I distilled around that knowledge. So this is kind of the stuff that I'm going to bring in this presentation, philosophy shifts from doing bug bounty testing from web app testing, traditional web app testing, discovery techniques, mapping methodology, parameters often attacked, useful fuzz strings, bypass our filter of Asian techniques and some tooling that I think is cooler than other tooling. Cool. So the first section is philosophy. So the differences between kind of bug bounty hunting and being a web pen tester, you know, I'm not really to argue this debate. There's a lot of people who feel strongly about both sides and they're both right, honestly. But when you get down to the practical work, you introduce a lot of stuff here. You introduce time onto a security tester, right? They're not used to competition when they're doing this kind of stuff, unless you're playing in a CTF, which, you know, you're used to that. I played in some CTF, so I was kind of used to it. You know, you're only incentivizing it in one side for what you find and not the hours you put in. So I mean, this is a basic overview of how they differ, but the talk's more about the technical stuff. But yeah, you basically tailor your methodology based around finding stuff in the 20% as opposed to the 80% across application assessment. So we'll go into how that 80-20 rule kind of fits in the rest of the slides. So if you're doing regular web app assessment, you're following these two Bibles, right? The OWAS testers guide or the web application hackers handbook. This is usually what you're trained from and what your internal methodologies are built off of at almost any of the good consultancies. And the authors are, you know, super great testers, right? But these take you from A to Z.  So bug bounties are different. So if you want to do web hacking, you want to get started, these are what you go for. But my talk is a little bit different. So let's talk about discovery in a web application assessment for a bounty. So what you want to do in a bug bounty is basically find the road less traveled. And this is if you're aiming to get paid, I think. So you can attack the flagship application that the company has, right? But really, that's not where the vulnerabilities are going to be most of the time. That application has been tested by a pentest team. It's probably had a bug bounty on it for a long time. What you really want to find is the parts of the applications that are like subdomains or maybe obscure web servers on different ports. You want to find acquisitions that maybe the company has had recently that came in from a different development team. And they might have a whole slew of problems that came from a whole different group. You want to look at functionality changes and redesigns on sites, mobile websites, because they're set to render differently on your phone. And also new mobile app versions when you're testing. So we're going to go into some tools and stuff I use to find a new surface area for you to attack. So ReconNG is this tool that basically allows you to do a whole bunch of automated OSINT stuff. And one part of it is it has all these modules to do subdomain brute forcing and subdomain discovery. Now subdomain discovery is a big part of finding applications that have been left out there. I mean marketing spins up a site, dev spins up a test site, like you have integration stuff left up. So finding those and hacking those and getting RCE or code execution through those sites is kind of where you can get big payouts. So this script what it does is it iteratively scrapes Google for all subdomains on a given web property. So let's say acme.com. This will find, this will scrape Google for everything that is in acme.com and then iteratively remove those results until you're down to this long list of subdomains. It also scrapes Bing, Baidu, Netcraft and brute forces subdomains like your common fierce tool would. So this is on GitHub. It's a simple shells wrapper around ReconNG. So you need to have ReconNG installed. If you use Cali Linux, you can just pop the script in and go. So this is the output of something like that against a company like this. You can see there's a lot of output. Probably a lot of domains here that have gone under assessed as far as you go. So this is that idea of iterating through Google to find subdomains. So here you have site and then minus dub, dub, dub sites. And then I found on the first hit was Sandbox. So I removed that on the next run through. And this is that scraping that's happening that the tool is doing. And then you get more entries floating to the top. So you get credit apply or business or shopping or advertising. And you just keep on removing these until you have all of them. And then you brute force and then you end up with a huge list of sites to assess.  But they've just purchased a company, right? So Oculus, you know, purchased by Facebook had some bugs and they got popped as soon as they were acquired and they were not under the Facebook like six month rule or I don't know if it's six month or I can't remember how long, but yeah, they got popped immediately, right? And that was a whole different dev team but owned by Facebook and worthy for an RCE bug. They got hit with SQL injection and a custom header and that was great. So well, not great, but it was good for the bounty hunter. So, yeah. So Wikipedia for Facebook and Google does this really well. People updates these things all the time when there's an acquisition for like stock reasons, right? So keep an eye on these if your company has, you know, purchased something else and they have some new domain and it might not be in the bug bounty brief yet. You can go after this. If you're doing those types of bounties. There's also a repository of links of every kind of volume that's come out on Facebook and PayPal and Google. People like to share this information. This one is hosted on Facebook. I have no idea why. The links in there and the slides have everything hyperlink so you can go check it out. But these are all the blog articles that reference bugs here. Now, why is this important if somebody else has already found these bugs? Because bugs get represented across the domain in different places. So you can tell a lot about an organization once you read these articles and find the same bug in other locations. Like the subdomains you found, maybe rogue web servers and things like that. It also kind of tells you what they're going to do to fix them, like how they filtered out input. You get a lot of intel around the application. So really doing a lot of research on your target can't help. But it's not the fast stuff. So port scanning. I mentioned port scanning. So port scanning is not just for NetPen. So yeah, I mean how I hacked Facebook, there was an article by Ryan Dewhurst who was like, I started up port scanning and found some weird server which was a Jenkins script console with no auth. And that was it. He got in. Simple as that. Like $8,000 or something like that. Or even more, I don't remember. So is.net, the Microsoft domain that evangelizes.net had RDP open to the world with MS-12020 on it. Vulnerable. And so that was a thing. So yeah, just go ahead and use a simple end map syntax to start port scanning all of your sites and make sure you check all those services. This syntax will port scan for all ports on a domain as well as pull out any HTTP servers titles and display those in the output. It's a SIN scan and OS fingerprint and stuff like that too. So mapping. So you found all of these new servers, right? Like maybe sub-domains or maybe you found an acquisition or something like that and now you want to move into, you want to move into mapping an individual application. And taking notes is really important when you're doing this. Whether you're doing it inside of like a notepad or just using pen and paper or like Evernote. I use Evernote. All my bugs are templatized so when I find them I can just copy and paste into the disclosure email or whatever like that. So these are some mapping tips that I use right away. So Google is actually your friend, right? You can get a lot of parameter information from just Googling a domain and figure out like what happens there. I know there's some like parameter parsing scripts. I couldn't find a really good one for this presentation. But you know just will parse parameters out of the Google cache stuff. But really the next big thing is directory brute forcing, right? Finding unlinked content. Content that's not supposed to be there. So a lot of people use like Durbuster or Content Discovery and Burp for this kind of thing. And that's good. They're good lists. But those lists were created by going out and spidering the internet and finding every path after the top level domain and then prioritizing them. There's some other lists that are better for this type of work. So the raft lists are these lists that came out of a talk maybe four or five years ago. Raft was an application proxy. It was a decent one but it's since been discontinued. But it's lists for directory brute forcing have lived on. They are a spider of the internet's robot docs text files. So everything that everybody doesn't want you to see is in this directory brute forcing list. So it's super sick. I can't tell you how many bugs I found just using this list. Like config files, badly configured git stuff. It's just all over the place. SVN Digger is another list like this. They went out and spidered all the SVN projects. So if your project, if your site or your target is an open source place you can use the, you can take all the paths that have been converted for you to directory brute force to get better application coverage or find config files. Git Digger is the same thing for Git.  So there's just some really simple wins here. Wappalizer and built with are Chrome extensions that you can just click and they will give you pretty much the whole stack by looking at the headers, the comments in the pages, the way they render like analytics things that have been integrated and they'll just give you like the whole server stack. And they'll even give you version numbers if they can identify them. So Wappalizer and built with are super sick. Retire.js is one of my new favorites. It will profile all of the server side JavaScript libraries and tell you if they're out of date as well as give you all the vulnerabilities before that patch or your vulnerable version has. So you'll get a list of prioritized cross-site scripting or whatever is in jQuery at the time, right? And then once you identify all of these servers, version numbers, you just go check for CVEs and server side stuff. So that's pretty standard, that's web stuff. But these are some good tools.  WP scan for WordPress, a lot of people use this already. It will identify all plug-ins and users for WordPress install as well as look up any phones that are associated with those plug-ins that have been disclosed. And then CMS map for Drupal and what is the other CMS Drupal and Juma, there you go, thank you. Awesome. So those are the two that have really yielded any value for me across CMSs. So here you see a screenshot of WP scan. And it's, you know, found a version of a plug-in or a theme that already has like cross-site script or a file upload vulnerability in it. Sometimes there's false positives, honestly. For what this script does, it provides so much value, so it's great. So the directory brute forcing, we talked about a little bit earlier. I mean the workflow for this, a lot of people do, but I just put the slide in here because I see a lot of people do it a little bit weird. I see people brute force like off the top level path a lot and then just stop, right? And so they'll get some errors and they don't know what to do with it. So they'll go to ACME.com and get a 200. And then they'll go to Backlog and they'll get a 404 and they'll get more 404s and more 404s and you know there's nothing there. But then they'll hit like control panel and see a 401. And then they'll be like why can't do anything that I'm not authorized, right? So they don't brute force after control panel. There's so many like messed up access control on web server bugs that you can exploit. If you just brute force after that, you'll probably find something. So I just see this a lot where people stop after the top level domain when they're doing directory, you know, enumeration. So that's kind of the workflow you're doing there. Some other things that you can do is mapping and vulnerability using open source intelligence. So these are one, two, three, four, five sites, six methods that you can use to find already published bugs or almost already public bugs. I mean I guess they're considered like end day or whatever. But xss.com, reddit, xss, punk spider is actually a burp engine that just scans the internet. So if your target is a high profile site, information might already be in here for your test. And you can pull it out and use it to your advantage. Even if those bugs have already been disclosed and I've found bugs that were on here but not disclosed to the customer through the bounty. So that's actually worked before. It was like a super easy win. They help you get a feeling for what the company has faced before as far of issues, like prevalent cross-site scripting, cross-site request forgery, you know, file uploads. And then you can do regression testing on all the domains that you found earlier in the presentation. So, yeah, go out and use these resources to try to find bugs in the platform as quickly as possible because they're free and they're already out there. And the customer should know about them anyway. It's the responsible thing to do. Okay, so this is my intern Ben. He's never spoken before at DEF CON. Neither have I actually. This is my first DEF CON speaking. But he did an awesome project and he's going to talk about it for a couple seconds. I really like it, so. So, hello everyone. My name is Ben. Like Jason mentioned, I'm an internet bug crowd on Jason's team. For the past couple of months, we actually gathered a bunch of JSON files that includes all the metadata for each bug-bounding program that's out there. So, there's 250 plus bounding programs that are included in this project and they all include information like how much a minimum bound is, how much a maximum is, what mobile apps are included, what web apps are included and what's not included in the scope of the program as well. And we actually use all this data and fed it into different scripts like you can see on the screen. We fed it into knock and it just went through all the, every single one of those programs and brute-forced them for sub-domains. And this also is available on GitHub account and everyone could go and use it if they want to. So, the JSON files look similar to this. This is Yahoo's program a couple months ago. If I've updated it, we don't know. But what we have is a DNS record that shows that's the Yahoo.com itself and all sub-domains of it and Flickr and all sub-domains of Flickr are included in the scope as well as all their mobile apps that are included in the scope as well. And as you can see in the bottom, there is two sub-domains, two domains which is Yahoo.net and its sub-domains and Yahoo.net itself not being included in the scope of the program. So, what we ended up doing with this using Ruby, we wrote a script, we fed every single one of those JSON files and we crawled them.  Taking it further, we, same idea is all the JSON files and we fit that into Intrig, which Intrig is a API framework that is for intelligence gathering. And it does a bunch of tasks that you can see on the right left side of the screen. It includes doing supplement brute force, web spider, end map and you name it, we can do it with Intrig. So, also Intrig is available on GitHub as well. It's open sourced. Go ahead and fork it and commit to it if you need to. What we ended up doing for Intrig is we parsed every single JSON file with help of Ruby and I can see at the line when it says R, we are taking the task which is called DNS, DNS brute sub which is a supplement brute for sir and we need to give it an entity and then options that are all included in the manual and we are running that for a JSON file which at the bottom shows it's being assigned an ID that you can just go in your local host and check it out and see what Intrig is found. So, for example we did Intrig.io and for DNS brute force you can see all those sub domains that have been out there that Intrig found with their IP addresses as well and make sure you guys check it out. It's like I said it's online and you guys could, the possibility is that it's in finite. You can do whatever you can think of out there with those and being a bug bounty hunter I think it's a huge tool for everybody out there. Yeah, that's a sick tool and a sick framework both mapping and recon entry like I mean if you've used FOCA and Maltigo and everything like that it's like an open source version of those license tools. I love both those tools by the way just saying like using both if you can but Intrig is going to be sick you guys should check it out. Okay so on to often session I'm going to have to blow through some of this because this presentation is long and like there's a ton of stuff in here. So auth related bugs the one thing I want to say about these right these are low shallow bugs that everybody hates people to report in bug bounties the problem is if people start not paying attention to them you can't chain them to do bigger things right so we've had multiple bugs or I've had multiple bugs where where we've had a couple small issues like with password resets or like you know something like that and then we've chained them to make like a critical critical account takeover bugs so these are really important but these are the kind of bugs that a lot of people see in like the hashtag like beg bounty instead of bug bounty you know people really don't like them don't discount them just note them and if you don't want to if they're out of scope don't do anything with them but you might be able to use them later that's all I really have to say about some of these so session the kind of same things right so failure invalidate old cookies like new cookies on login or no new cookies on login and time out never any cookie length like these are all things you're going to be able to use later when you need to chain bugs but a lot of times they're out of scope so either out of scope or unappreciated or duped or something like that so but but yeah you should you should keep them in mind when you continue testing because they can be chained into bigger issues so so the big part of this one is actually tactical fuzzing so so I go through a couple different injection types or you know vulnerability types here and so we're going to talk about cross-site scripting and some research that some really cool people have done so the core idea of cross-site scripting right does the page functionality display something to the users like you know that's kind of the question I asked myself you know can I get reflection somehow with JavaScript and so you can do manual testing which is great right and you enter in your your meta characters and see you know if they return but really when I'm trying to work fast in a bug bounty I have three or four like magic strings that I use and so you probably use them before the technical definition for them is polyglot payloads these are web polyglot payloads and so the first one you'll probably recognize this one is arsenakes they used to call it the arsenic battering ram or that's what I when I worked in pentests they used to call it came out of the XSS cheat sheet you probably use this before you put it into the search bar comment field and then you pray that you get cross-site scripting right so this is the first one this is actually a multi-context filter bypass based polyglot web payload that's a mouthful I know but basically it's designed to evade filters it's about to execute in different web contexts and it's really cool so I have three of these strings that I cite here that if you're just doing bug bounty hunting you can use and just kind of move along on your on your critical functions in the site so this one is from a researcher named a charge of ed he does a cross-site scripting research I think he did his PhD in cross-site scripting which to me is like blows my mind so this is a multi-context filter bypass based polyglot as well so you can see here that he's trying to to mark up in a whole bunch of different contexts he's got like an e like an at sign here to like trick trick email like filters or you know maybe a formally takes emails or something like that so he actually ran this along like the Alexa top 100 and like 80% of them were vulnerable to just their search parameters with this string so you know more ammo for you guys doing bug bounties this one is one by Matthias Matthias Carlson and is he here right now? Is Matthias here? Hey there he is he's awesome so he did a whole presentation on this idea multi or polyglot payloads and websites so this is his multi-context polyglot payload and so this is one that I use now so thank you. Other XSS observations when I started parsing bug bounty work as well as getting bugs myself so finding input vectors is important so finding customizable three themes or profiles that use CSS but then you can trick them into using JavaScript to execute cross-site scripting a lot of names of like events or meetings in any application that deals with those types of things URI based XSS is still a big thing when people pull things from the URI and render it for some reason importing from a third party so things like Facebook integration where there may be filtering characters but your site actually displays Facebook data in line so you can set your name on Facebook to script alert and it will alert on this site. JSON post values that didn't return the correct content types a lot of people discount web services right away because they think that the content time won't execute cross-site scripting or won't execute JavaScript and so you have to really check and make sure they're returning the content type otherwise you can get reflected XSS and a lot of web services like that. File upload names when you're uploading file just try to change it to script alert or whatever like that it's going to echo that file name back usually a lot of places.  Custom error pages where they're echoing out what you can't find put XSS strings in there fake parameters where the page might parse some fake parameter data and put it into into your response and then log in and forgot password forms. Also this is a Swift parameter XSS is a huge thing as well I don't think I've ever found a Swift file that I decompiled that hasn't been vulnerable to either cross-site scripting or remote file include and actually Dennis here is like the guy I ask questions to that all the time so yeah so these things are like J player and like all of these like caught software that are Swift files that do like media or whatever like so there's a whole OWAS page on the common params that these players use and then also the injection strings but these you have to kind of do more manual analysis so to do that manual analysis I use this tool called flash bang which I think is super awesome it's by cure 53 you drop it a Swift file on the other end comes out all of the parameters that might be vulnerable to cross-site scripting it decompiles it for you and it displays them along with if they're going to execute out of the context of the Swift file I highly suggest this tool if you're going to do some Swift hacking it's way better than like a lot of the old ones cool so SQL injection the core idea does the page look like it might need to call on some stored data obviously this is Matias is SQL I polyglot where it will execute in single quote double quote and straight into query context so I've seen a lot of cross-site scripting polyglots and remember these are things that actually scanners are starting to do right they don't want to send a million fuzzing payloads to a parameter because you have like eight million parameters on a page so it just takes forever to scan things right so Matias and his presentation like has this string and I imagine a lot of fuzzers web fuzzers and scanners will start to pick up on this type of thing if they haven't already the idea of these multi-context injection strings so this is awesome as well so for SQL injection to kind of go through and fuzz things I also use this project called the sex list sex list project and it's got a whole bunch of fuzzing lists in it it was a fork of the fuzz DB and then we added to it with like username and password lists and all this crazy stuff Daniel Measler here actually helped me curate it and we decided together and it's it's invaluable right it's got like by by type of injection so if you want to just do like a login bypass in my SQL it's got all those curated all those strings curated that would you you would usually use to do SQL injection there I highly suggest using this and I just load these in the burp into intruder when I want to attack a former or something like that some parameter I think is vulnerable so other observations so blind is the predominant SQL injection you hardly ever get air based SQL injection anymore and so like in those cases you use like benchmark strings and stuff to make the page take a long time to load and that's how you identify whether you take it the whole exploit way is you know it's up to you right we have a lot of researchers I know who just want to identify and move on right I like to to run SQL map eventually because it's still king I mean there's no other tool that does it as good as SQL map and that's actually something I learned doing the research was that everybody use SQL map at some point so yeah some tips for for SQL map basically when you're doing this you can actually part parse a whole burp log file so like enable burp to do logging and then parse the whole log file and actually fuzz the whole log file with SQL map it takes forever it's not like the greatest way to do things but it's also offering a lot of coverage if you're up against some kind of like blacklist or something like that it has a SQL map has tamper scripts that you can use which basically encode all of your attacks so that you can try to evade blacklist and it there's a really good guide on there it's somewhere on the bug crowd forum on DBM DBMS specific syntax for SQL map tamper strings so if you're going up against MS SQL or my SQL or something like that there's a simple string you can pass into SQL map and start fuzzing those parameters and get past blacklist and then a really fast way to instrument SQL map is SQLI PI which is a burp extension and basically allows you to right click in any window and burp and send that request to SQL maps API running on your local box so like you can just be inside a burp right click and start fuzzing the parameter if you need to be so some common parameters and injection points like any ID value currency values item number values sorting parameters I'm not going to go through all these they're all on the slide like and eventually this is all going to be on GitHub anyway so you guys can just grab it and use it in your methodologies if you think it's useful but these are the kind of places where we saw them where I saw the most injection and where I you know my research parts out of other places showed me this is SQLI PI so right click on a request send it to SQLI PI scan and now that burp renders scanner results in the target tab but it doesn't look like this anymore but you get the idea so this is my cheat sheet of SQL injection resources when I do SQL injection broken down by my skill type and these are cheat sheets that let you know manual syntax based on my SQL a lot of these people are like pentas monkeys lists are old they're still the best like you have to use these and you have to have them handy when you're doing injection so there's some really edge case ones at the bottom like access which God who uses access that sucks ingress dv2 informix like three and active record for Ruby on rails so I keep those handy in my every note when I'm doing SQL injection testing and when I see errors or long load times or something like that I just I start you know getting in that mode so file uploads and file inclusion is the next area so local file inclusion the core ideas does it or can it interact with the server file system Liffy is my cool favorite tool for doing this obviously you can do it manually so I have all of my Liffy scripting stuff up on SecList under fuzzing and Liffy so you can see here like I've you know I've tried a little bunch of blacklist bypass or encoding to try to get common you know system files this is on the SecList project common parameters or injection points for this type of stuff is like you would think of this but it's good to have it in the list so like file location locale path display load read or retrieve these are the most common parameters that you'll find those in malicious file uploads this is an important and common attack vector been doing this type of testing not only just to upload like a swift file and get XSS off of it but you can also do pretty cool attacks one of the ones I like a lot and it's a it's a DOS basically an image that specifies itself to be super large but isn't so it you can upload it and the server will write all this or will allocate all of this space for it on disk but it's actually not that big of a file and you can DOS the application server using images crafted like that there was a whole blog on it and then you can you can actually one of the things I think is interesting I'm not going to go into too much but there's a slide about it is bypassing like security zones and storing malware on client servers so there's as well as polyglot web payloads there's also a polyglot files which can execute code in different contexts like you think of a parser reading a file it can you know it basically will look until it finds what it wants and then execute that so you can create like a jar that is actually an executable so if I make an executable that is malware but I upload it to your server because you allow me to allow a jar well is that a phone like I don't know like you are technically storing malware on your server for me right and I can send the blackouts to go retrieve it but can you do anything about that right are you going to implement a parser to look through the binary data and cut stuff out I don't think so that's kind of hard to do so interesting question there it's kind of a another road Dan Crawley did a presentation on it here at death con and it was super sweet so you should check that out oh no technical errors that that came at the perfect time actually oh wow okay that's what we're doing that we're doing shots okay so he's a first-time speaker and actually the little story I would go ahead I guess he's mentioned that you would deaf con like what deaf on 16 deaf con 16 he met someone I met Julia my wife here he met his wife yeah so you know give him a hand huh cheers cheers welcome to the club thanks man now back to the show I mean I mean if I can deal with the laptop issue are these guys doing all right should I kick them off the stage or you want to keep listening to them I can go to keep listening I guess I guess you can stay okay can you give me a second till my throat stops burning uh no okay all right let's see if this works sweet okay so file upload attacks are a thing I've never seen any better presentation to guide you along the road of file upload attacks then this guy Sarusha Dali's presentation and if I butcher names I'm sorry I love all these guys they're bug hunters just like me so about file upload vulnerabilities and this includes doing new and novel attacks as well as old attacks to get files past black lists or you know upload you know bypassing extension trickery or stuff like that so I'm trying to give you guys resources as well like the ones I would use so I mean a lot of this actually got I think got parsed into the new a was upset testing guide most of it at least so I would I would check that out too as an intro to malicious file uploads and getting shells and like web shells and stuff so oh this is what I talked about Dan Dan Crawley and I don't know that guy's real name but um yeah these are the types of binary files that can execute in different ways so you can see they have like a PDF that's a zip that's a MBR or so like you know interesting research here coming out there I would like to see you know interesting bugs come out of it so remote file includes and redirects common parameters there destination continue redirect URL URI window next common black list bypasses these are all kinds of escaping tricks that you use normally in web stuff but these are the most common ones I found these are also insect lists in the LFI and RFI fuzz lists that I use often so for RFI these are the common parameters file folder path style template yes yes yes yes so these are where I saw the most bugs or you know other researchers you know published data around their RFIs and so these are kind of the type of parameters you can do and I think eventually right the thing you do here is you write a verb extension I haven't yet right but that just like automates anytime you see these it sends it to like logger or something like that so you can just go test them later I haven't done it because I just do it with eyeballs but it's probably the better way to do it is write an extension to do this work okay so cross it request forgery how much time to have 10 minutes okay I think I can do it okay so CSERF everybody knows about CSERF right like how do you execute CSERF you find some function in the website that does something right and you what it's a security related function change password or whatever right there's a list later of the functions and then you right click and burp and create proof of concept that's like CSERF nowadays so what you really have to focus on in bug bounties is CSERF bypasses customers who have CSERF protections but haven't implemented them enough so common CSERF bypasses in my research you'll did removing the token from the request removing the parameter value from the request adding control characters to the parameter value using a second identical CSERF parameter or changing the request method so check this out this tool has gotten no love I don't know why I think it's been out for two years already it's called burpee have any of you used this tool before no good give you something to take away so what burpee does you enable logging in burp and you crawl a site completely that has cross-site request forgery protection in it right like a CSERF token and then you create this template and tell it what the token was what a good result is for getting a page what an error page looks like and this template is actually really easy to edit this is the sample one this has been out for I think two years already I don't understand why people use this super sweet right so then you write this template it's a python script and then you run his python script burpee on your burp log file and it re-request all of those across the whole domain every request that you've ever made in burp re-request with with those first three attacks for CSERF bypass then it produces an html report telling you which ones gave different error messages which ones came out the same and prioritizes CSERF CSERF bypasses for you so he made a lot of money doing this to facebook and twitter because it wasn't a direct burp extension it didn't get a lot of notice I randomly found it on github and I was like sweet this is awesome so this is a part of the html output here's the base request here's the crafted the first crafted request and then the response and then you get a report back saying if they came back the same it diffs them so I highly suggest that tool it's linked in the talk another way to do it is just to check for every request across a whole burp log file that didn't have the token in it the actual parameter so this is another script that does that it's another python script that runs on a burp log file also that went undetected kind of a little bit so super sweet I use these all the time and finds bugs with these all the time so CSERF just some common critical functions like add and upload file you know password change email change transfer money your currency delete a file edit your profile things like that so these are commonly where you see CSERF and that rewards big bugs so privilege transport and logic kind of get mashed into a section so privilege you know off and logic kind of get blurred a lot but my testing thing is just you know the you have an administrative user you need a couple of counts to do this and then you have a low privilege user and then you know the low privilege user just tries to directly call functions that are only for an admin right pretty simple but to automate that across multiple functions you might need some tooling and so this is what I use for that it's called autorize this one is available on the burp store and basically you spider a site completely you run through it all of your post requests as an admin user then you go through as a lower user and you give that information to authorize and you run the tool and it tells you which ones the lower user was able to access that the admin user was also able to access and you look through those in your burp output so common functions or views that I check for privilege escalation or anything like that and these can be actually combined with the last two sections is add a user delete a user start projects change account info view customer analytics or like there's a page that tells everything about whatever that site does you want to try that view payment processing view like receipts or any view with any PII on it you want to focus on this is what that looks like authorize browse using a high privilege user login with the low privilege user burp re-request everything and gives you prioritized output cool so insecure direct object references is one of my favorites because I want a bad-ass pair of gaming headphones a couple months back so I found a bug in a really cool company and I had to disclose it and I ended up calling them on the phone and their help desk guy was like I have no idea what you're talking about and then I actually linked in spammed everybody at of the IT group of that company and finally someone accepted and I tell them like listen I'm legit I'm not like an asshole I'm not extorting you I just want to tell you this exists because I was buying a pair of headphones already and their astros by the way they're awesome and they fix this bug so yeah and so the receipt function basically was an I door you could just iterate up and down and find other people's receipts with their credit card info on it and they sent me two free pair of headphones and I have one and one goes to Daniel for his birthday but I forgot to bring it I'm sorry but they're sweet headphones I guarantee it they're awesome anyway so I doors five minutes okay cool so increment decrement negative values attempt to perform sensitive function substituting user IDs things like this these are how you test idors these are common functions user files that deal with idors so everything from the CSERF table anything that says UID password or user hashes emails images like that are supposed to be private so you can you can go through the slides and kind of go through this and all this is going to be up on github and if you have questions or whatever so this is a simple I door I don't know why I put a simple I door in here this is deathcon you guys probably knew this you see like this numerical value and you're like oh what happens if I change it and you get someone else's receipt this is exactly what I did to another place this was a disclosed bug that's patch so don't worry about that they had a bug bounty transport you need to enable HTTP everywhere there's this awesome script that will basically take up our blog file again re-request every request in your site tree over HTTP instead of HTTPS so you can see what's going over insecure channels instead of having to like you know sort columns and burp and do all this stuff I find this really useful so it'll just try to downgrade everything and then you report like this is a SSL downgrade attack or whatever logic fog logic laws are usually pretty manual the one I see a lot is substituting hash parameters where there's like prices or something like that and they've hashed it and it's irreversible or they've done something to it and it's irreversible or I'm too dumb to reverse it right but just finding another item that's cheaper taking its hash and substituting it and usually at what it's an e-commerce payment or something like that and so doing that is usually yields the product for less money so step manipulation this is like the bread and butter example everybody gives for logic frauds like there's multiple steps like order or put things in cart order check out pay ship so you just skip everything or you like put everything in your cart and then you just ship because you have the you know the whole process and burp and so you just you know skip a process using negative quantities in or using negatives and quantity per value so I've actually had websites pay me credit because I put in a negative negative value on some price thing or your negative quantity right like order number equals one usually I want to buy one thing and I put in like negative 20 and now they've credited my account like a thousand dollars or something like that so application level DOS this is one is kind of interesting right it's not actual DOS right I'm not advocating bug bounty like using a botnet or anything like that but I've seen sites that just can't handle just like parsing a parameter with you know 40 zeros in it or something or me putting in like a math function as a parameter value and the server is like I don't know what to do let me try to process this somehow and it just falls over so those are interesting and then timing attacks I think there was a DEF CON talk about timing attacks so you can check that out mobile I'm really running in so data storage is really important check these files for data storage as well as logging this is the best tool to quickly get spun up on iOS it's called IDB it's by Daniel Mayor basically jailbreak your phone install this tool it gives you a full GUI list of the handler of all of the files all of the encryption values if it's using exploit mitigation etc etc etc it's the most functional tool I think it's partly based off a talk I gave a long time ago and he made it in Ruby and it's super sick it's the best way to get into iOS testing if you've never done it before this is a thing about logs we got to go this those other those other phones I repeat them again right like don't discard them like content spoofing referral liquid search security headers path disclosure keep them in your pocket later to escalate if you can use them this is one idea of like you know if I have five or thirty minutes or something like that what can I do so I tried to time myself with the methodology using the stuff in here so in fifteen to thirty minutes I can do most of this using burp in the automation maybe now or like it depends on how motivated I am right so these are like the steps I'd go through I'd register I'd hit the password reset I'd go to all the forms that do security functions I checked the cookie I do like a all right and perform a numeration on any like UIDs I see in the URL I directory brute force using one of the short lists in the background I'd upload a file if it had uploads and within thirty minutes you know or an hour I can usually find some pretty darn good bugs things take with you crowdsourced is different it's the same but different you find like twenty percent of the edge case stuff instead of eighty percent and a lot of stuff goes quick data analysis is cool you could probably do a fifteen to thirty minute web test and done right you could get some major volums set lists polyglots are cool and follow all of the bug bounty people on this bug bounty list I put them all into a twitter list for you and you can watch them hack things and talk about their findings also there's a lot of stuff that didn't get put in here there was a lot of data that I didn't get to parse so fifty percent of the data is still unparced someone to put it up on github as a get book I think or maybe just mark down and you guys can contribute to it if you care enough if you just want to take it and use it that's fine stuff to go in there more tooling that I found XXE that's actually meant to say SSRF a bunch of cool SSRF techniques capture bypass more detail on logic flaws and to add android mobile tools that I use often thirteen memes that okay okay attribution and thanks these are bug hunters who did researcher that are these are bug hunters who did things in this presentation all of them are super awesome I respect every single one of them or who made tools and also my team at bug crowd John Taj Ben Ben Grant Fati Patrick Katie Kim Abby Casey Chris and Sam and everybody in the bug community I love you guys I love doing this for a day job so that's it


We had a boxer break out of the peek-a-boo, punch to the face. That's Julian Quiet Storm Smith. He's legally deaf. That's why he's got the cool nickname of Quiet Storm. He was a huge underdog in this, and just came out swinging, knocked him down early, knocked him down again here in the eighth. They're just going at it. I think he knocked him down four times in the ninth round when he was feeling cocky, shimmy in the shoulders, dancing, you know, poking around, just having fun, plays a little peek-a-boo as we saw. Peek-a-boo, boom. Man, Velazquez was undefeated, and he just got beat. I mean, it went the distance, but Smith knew he was going to win. He didn't hear no bell. He just kept fighting down, knocked him down like four times. Man, that says here, Velazquez was being built up to fight a world championship in the near future if he didn't lose to Smith. But guess what? He lost to Smith.


lecture is an introduction to statistical inference in which we start looking at estimation, and hypothesis testing will be in another lecture. As you know, statistical inference involves making inferences drawing conclusions about population parameters based on sample statistics. We could be doing estimation where we're looking at the sample, getting the sample statistic like say X bar, and then using it to draw conclusions about the value of the population parameter mu. The other activity we could be doing in statistical inference is testing hypotheses where we collect a sample and we have a hypothesis about the parameter and we use the sample evidence in order to test this hypothesis. I think most of us are familiar with the notion of using one value to estimate something that we don't know. We'd call that a point estimator. There's a problem though in using something like X bar, which we know from the last lecture is a random variable with its own distribution, to estimate the population parameter mu. After all, what's the probability that we're right? What's the probability that the X bar value we happen to get from the one sample out of many that we happen to collect really is equal to the single value, the true parameter mu, the mean of the population. There's only one mean mu, but there's a lot of different possible X bar values that we could have gotten, one for every one of the different samples that we could have collected. The probability that a random variable, X bar in this case, is equal to a particular value, mu in this case, is always zero with a continuous random variable. That's why when we're doing probabilities from the normal distribution, we had to get probabilities in an interval. We had trouble constructing probability questions with a continuous random variable like a variable from the normal distribution. Well, that's exactly what's going to happen over here. We're going to use intervals here as well. We're going to say, well, I can't use X bar as a point estimate of mu because X bar is just one value that I happen to get from my one sample that I happen to collect. But I can take X bar, add a little something on the right side, add a little something on the left side, make it an interval and then say, well, this interval is my best guess. It's my best estimate for the true population mean mu. And I'll try to come up with a certain level of confidence, which is really just a probability that the interval really does contain the true population mean mu. When you think about it, you'll realize that this is actually a much more accurate estimator than the point estimator, which only seems accurate. If you, I'm just remembering many, many years ago, when the other professor Friedman and I were dating, he used to somehow get away with saying, I'll meet you at so and so a time at such and such a place, give or take 20 minutes. Well, you could never be late. If you use an interval like that, you could never be late. It can't go wrong. 3 p.m., give or take 20 minutes, you could never be wrong. If you're there at 3.02, okay.  You're late. And that's why an interval is always better than a point estimator. And in fact, it's always more accurate than a point estimator. And that's an interesting thing to consider. How wide should the interval be? Well, you're going to see that it depends on how much confidence you want in the estimate. For example, suppose you want a confidence interval estimate for the mean income of a college graduate. Well, if you want 100% confidence, there's only one way to get 100% confidence. You've got to take a census. Otherwise, you're going to have to work with less than 100%. That's another way you kind of, if you don't mind getting fired, you can tell your boss 100% confidence. Sure, minus infinity to plus infinity. That's 100% confidence. Or in the case of income, zero to infinity. That's a good way to get fired. And you sound like a wise guy, but the reality is that's the only way to get 100% confidence if you're not willing to take a census. And notice what happens. 95% confidence interval. Well, you're 95% sure that that interval, in this case right here, being 35,000 and 41,000, you're 95% sure the mu is in there. Remember, mu doesn't move. It's a fixed number. It's a true population mean. If you're willing to live with 90% confidence, notice 36 and 40. 80% confidence between 37,500 and 38,500. Okay, so basically, this is a very important concept. The wider the interval, the greater that your confidence you'll have. Again, that's why minus infinity to plus infinity, that gives you 100% confidence.  That's why we never say X bar is mu. X bar is a random variable. So I have zero confidence if I say, well, the mean income of a college graduate is $38,000. That's called a point estimate. That gives you zero confidence. We need an interval. And again, the wider the interval, the more confidence you'll have. In this slide, you're going to learn how to construct a confidence interval estimator. You're going to see it called CIE. Okay, so we're constructing a CIE for mu. Look at the formula. X bar, by the way, the sample evidence and these kind of problems is going to be the standard deviation S, X bar, the sample mean and N, your sample size, took a sample. So there's an N. Remember, N, X bar and S, those are going to be, that's going to be called the sample evidence. So we're going to use, ideally we should have sigma. We're going to use S instead of sigma. What's that Z alpha? That comes from the Z table. Okay, and that will depend on how much confidence you want. So we're going to learn how to do that. Confidence intervals, we're going to be learning in this course are called two-sided confidence. We're not going to say it always, but keep in mind it's two-sided confidence interval estimators. This is why we take the alpha. You have to split it in half. This is, so this is the way you'll make sure to get the right Z value. All right, so we're just going to call the Z value Z alpha. Really, it's been split because it's on two sides. Again, this course assumes that we're working with two-sided confidence interval estimators. Okay, now what's in the middle there? The big white space. That's one minus alpha. So let's take a problem. If you want 90% confidence, okay, that means the 10% you don't have confidence, that's going to relate to your alpha error. Okay, we put 5% of the right tail, 5% of the left tail. If you're working with 95% confidence, one minus alpha, that's the confidence, 95%, then the alpha of 5%, this again split into two tails. So you have 2.5, 0.025 in the right tail, 0.025 in the left tail. If you wanted to work with 80% confidence, okay, that means the central part would be 80%, then you have 10% of the right tail and 10% of the left tail. So one minus alpha is essentially your confidence and that alpha, which we're going to learn more about, that's just kind of an error, it's an uncertainty. That's your lack of confidence in a way. You work for a company that makes smart TVs. Your boss says, take a sample of 100 of these TVs of a machine to test it and everything, and they want to know with certainty the exact life of the smart TV that they manufacture. Remember, you took a sample of 100 TVs. So what is the exact life of a smart TV made by your company? Again, the sample evidence is always N, 100, X bar turns out to be 11.50 years, that's the sample mean, and the sample standard deviation is 2.50 years. So what is the exact life based on this sample evidence of 100, but your boss said with certainty, the boss wants certainty. So what are you going to tell your boss? Well, here's what you're going to tell your boss because your boss wants 100% confidence, that's called certainty. So the only answer you can say is, well, minus infinity to plus infinity. Actually, you could have also told your boss, you can't do it from a sample of 100, you need to take a census, and that means test every single, also get your fired. You say, okay, we're going to test every single TV we made, and the boss will say, that'll destroy each one, I can't sell them, you're fired. So these are two answers that are going to get you fired. Minus infinity to plus infinity, well, let's test every single TV. The whole goal is to use a sample. That's what inference is about. Taking sample statistics, trying to draw conclusions about the population parameters. Now we're going to share the right way to do it, but we call the better answer. Okay, the boss, you tell the boss 100% confidence, can't do that. I don't want to get fired. So the boss say, okay, we'll work with 95% confidence. Okay, now remember, you have the sample evidence and it's 100 X bars, 11.50 years, and S is 2.50 years, and you want 95% confidence. 95% confidence, again, the 95% is in the middle there, right? So what do you have in each of the tails? It's two-sided confidence interval. Take your 05, 0.05 divided by two. So you have 025, which is 2.5% on the right tail, 0.025, which is 2.5% on the left tail. And then looking at your zero to Z table, if you have 2.5% in the right tail, that means you have 47.5% from zero to Z, from zero to 1.96. Of course, you have the same thing from zero to minus 1.96. You have 0.4750. If you look it up on the table, you'll find the Z value that corresponds to 95% confidence is 1.96. That's your Z value. So now using the formula X bar, 11.5, plus and minus that Z value, 1.96 times the standard error of the mean, which is really sigma over the square root of n, but we're using S, which you can, we have a large sample, some say 30, some say 50, but 100 everyone agrees, use S instead of sigma. So you have 2.50 over the square root of 100, which is 0.25. Notice 1.96 times 0.25 is 0.49. This is years, of course, 0.49 years. That is called the margin of error. That's a very important term. Make sure you memorize that term. Margin of error. That number that comes after the plus and minus, which is basically half the interval, that's your margin of error. Okay. So you do 11.50 plus 0.49. So your upper value, the confidence interval, is 11.99 years. Your lower value is 11.01 years. Okay. Now you see the confidence interval. Any value between 11.01 and 11.99, any value in there. Remember, there's only one mu and it's fixed in place. That doesn't move. It's not around the variable. It's a true mean. You don't know what it is, but that interval is, you know, covers 99, 95% confidence. With 95% confidence, that interval got the mu in there. You don't know what it is. You never will know because you didn't take a census. But in there, between 11.01 and 11.99, you are 95% sure lies the true mean, the mu. Now we're going to summarize what you've just been taught. First, with 95% confidence that the interval from 11.01 years to 11.99 years contains the true population parameter mu. Again, the reason you don't know what it is is the only way to know the true population parameter mu is through a census. We're doing this through a sample. We have 95% confidence that that interval contains it. Or another way to put this is in 95 out of 100 samples, that's where your 95% confidence comes from, 95 out of 100 samples, the population mean would be contained in intervals constructed using this procedure, the same n and the same alpha. Please remember, the population parameter mu is fixed. It's not a random variable. x bar is the random variable. That's why somebody else could take a sample of 100 small TVs and get a different x bar. We can have thousands of people taking samples of 100 TVs getting different x bars. It's incorrect to say there's a 95% chance the population mean will fall in this interval. The population mean doesn't fall anywhere. It's fixed. The only way to get at it is through a census. We want to know it exactly, but we have a good way of getting a good idea of what it is. Another exercise. We're looking at this in a slightly different way. We have sample evidence, sample of 100 refrigerators, looking at the lifetime of a refrigerator. The x bar value, the average of the sample, was 18 years, the standard deviation from the sample, four years. What we want to know is can we construct a confidence interval estimator of mu with various different levels of confidence? The idea of this exercise is to see what happens to the interval, the CIE, when we're using different levels of confidence but keeping everything else the same. This is more or less an exercise just to see how this formula reacts when we're using different table values. You see the formula on top. We're just repeating it. The x bar is in the middle. On either side is the z value from the table multiplied by the standard error of the mean. That gives you the half width of the interval or what's called the margin of error. We want to do this five times. One with 100% confidence, one with 99, 95, 90% and 68%. Let's see what happens to this interval and why we want to care about it. Remind you that we're using s in this formula, the sample standard deviation. We're using s as a point estimator for sigma, the true population standard deviation. When we're estimating one parameter, we really don't want to use an estimate of another parameter but pretty much all the time, or almost all the time, that's what we have to work with. As long as our sample size is large enough, we're okay doing that. Remember, we're constructing a confidence interval estimator for mu with x bar in the middle. What we're doing in the formula is substituting s as a point estimator for sigma. That's because s is an unbiased estimator of sigma. The way we compute s, it's an unbiased estimator of sigma s of the standard deviation of the sample. The expected value of that is sigma squared, the variance of the population. You can see the two formulas there and how we work with them and how we substitute one for the other. This is just a brief digression just to remind you that the formula is supposed to have sigma in it but if we have it, we use it. If we don't, we will very likely be using s to replace it. We're going to talk more about this later on in other lectures. On this slide, we see all the solutions to all of the parts, a, b, c, d, and e. You already know that if you want 100% confidence, you need to take a census. If you don't take a census, all you could say is negative infinity to plus infinity, which is useless. That's one of the things we want to learn here is you not only want to be able to construct an interval estimator with a certain level of confidence, you want something that's relevant, that's important, that gives you information. Saying that with 100% confidence, I can estimate a parameter with going from negative infinity to plus infinity, it gives you zero information and it's zero important and zero relevance. But that's what you got if you're using sample data and you want 100% confidence.  You can work with it. In part b, 99% confidence, you get the value, you want the central 99% of your distribution, so you look at the z table. With alpha 0.01, half a percent in each tail, the z value is plus and minus 2.575. We put that into the formula for confidence interval. The average x bar is in the middle, x bar plus and minus, there's the z value 2.575. We multiply it by the standard error of the mean four divided by the square root of 100. You end up with 18 plus and minus 1.03. Again, 1.03 is the margin of error and your confidence interval estimate at 99% confidence goes from a lower limit of 16.97 years to an upper limit of 19.03 years. That's a little bit better than negative infinity to plus infinity. It's still a little bit wide. The interval is a little bit wide. It gives you more information, b gives you more information than a for sure, but it still would be better if it were narrower. What do we need to sacrifice in order to make the interval a little bit narrower? Well, let's go take a look at part c. 95% plugging all the values into the formula. The only thing different is the z value that we get from the table. Looking at the central 95% of the distribution, we need a z value of plus and minus 1.96 and we end up with a confidence interval of 17.22 years at the low side to an upper limit of 18.78 years. That's pretty good. That's probably as narrow as you need, whatever the purpose is for this exercise. 95% confidence gives you not only 95% confidence, which isn't bad, but it gives you a fairly good narrow-ish interval. What happens if we're willing to reduce our confidence even further? 90% confidence, we end up with an interval that goes from 17.34 to 18.66. We went down in confidence from 95 to 90. We didn't really improve the relevance of the interval too much. It doesn't really help as much, we might as well stick with 95%. If we want to go down to 68% confidence, how do we get 68%? We go to the z table, we want the central 68% of the distribution. That means that 1 minus 0.68 is 0.32. That's split into the two tails and the z value is 1. And so it's plus and minus 1, 18 plus and minus 1 times the standard error of the mean. We end up with 18 plus and minus 0.4 as the half-width of the confidence interval or the margin of error. And at 68% confidence, we have an interval that goes from 17.6 to 18.4, but 68% confidence. So this exercise is just to show you how things change when the only thing we're changing is the level of confidence and the value we get from the z table in order to imbue it with that level of confidence. We're going to talk now about how we have to balance confidence and width in a confidence interval estimate. Okay, so how do we keep the same level of confidence and somehow make the CIE narrower? Let's analyze the formula. You see the formula? X bar, that's the sample mean, that's always the center. Okay, here's your margin of error plus and minus z alpha sigma over the square root of n. And you know that the more confidence you want, you've got to use a higher value of z. We saw if a 99% confidence used 2.575, if a 95% confidence used a value plus and minus 1.96. In any case, 100% confidence needed infinity, plus and minus, plus infinity to minus infinity. Okay, so the higher the value of z, the larger the half width of the interval. Or I can say that the more confidence you want, you're going to make the whole interval wider. Okay, the larger the sample size, the smaller the half width, because we're always dividing by the square root of n. So obviously, if you take a larger sample, you won't have such a wide. There's a trade-off. Take a larger sample, you'll have more certainty, so your confidence interval will be narrower. But of course, it costs money. I mean, obviously, if you take a sample of 1,000, you'll be more certain of the results. You have a narrower interval than if you take a sample of, say, 64. But basically, this is what something you should know. If you want a narrower interval, you don't want it to be wide, which of course you don't want, take a larger sample. Now, there are some tricks that are not taught in this course about stratification, various kinds of sampling tools that can help reduce the standard deviation for you. And that, of course, would make the confidence intervals narrower. But again, we're not going to do it in this course. But just bear in mind, there are ways to do this. And you might learn about them in other courses. But the simplest concept that we've learned is that if you want to narrow the width of your confidence interval, just take a larger sample. And we'll learn about this trade-off. This problem involves the life of a tennis racket. Okay? A company wants to estimate the average life of its tennis racket. So, for every company to do this, because they want to advertise how durable their product is. Okay? So, you know what the sample evidence is. N, X bar, and S. Okay? N is 81. They took a random sample of 81 tennis rackets, put the information into a spreadsheet, like Excel maybe. The X bar was 7.50 years, and S was 1.10 years. And now you're asked to construct a 95% confidence interval estimator of Mu. Okay? It's very simple. The hardest thing here in all these problems is just figuring out what the Z to use. Remember, you're taking the, you want 95% confidence, so the 95% is in the middle. Right? And that means you're going to have, in each of the tails, you're going to have 0.25. 0.25 in each tail. Okay? You're going to have 0.25 in each of the tails. And let me give you a 470501, you know, the fat part of the curve, 470501 on one side, 470501 on the other side. And of course the Z value then is 1.96. You need plus and minus. Remember, these are two-sided confidence intervals. So, you want plus and minus 1.96. So, you do 7.50, that's your X bar, plus and minus. Now you take the 1.96, your Z value times 1.10 over the square root of 81. That's the standard error of the mean. That value, 1.96 times 1.10 over the square root of 81, which by the way is 9. That value of 0.25, 0.24. And of course that's years. That's your margin of error. Make sure you know that term, margin of error. So, you have plus and minus 0.24. So, you add 0.24 to 7.50. That's 7.74 years. Then you subtract it and you get 7.26 years. And there is your confidence interval estimator. CIE is now, it goes from 7.26 to 7.74. You're 95% sure that that interval, somewhere in that interval, is the true mu, which you cannot get from a statistic. But you could have gotten it if you were willing to test millions of rackets and take a census. This problem involves the life of digital copiers. Okay, you work for a company, they manufacture digital copiers, and they want to know the average life of their machines. They take a random sample of 121 copiers, and here's the information you're getting from the sample evidence. Remember N is 121. They find out that X bar is 6.10 years. That's the sample mean, and the sample standard deviation is 1.85 years. And now you're asked to construct a 99% CIE for the population mean mu. Okay, so 99%. Remember, these are all two-sided CIEs. You want a two-sided confidence interval estimator. Okay, if you want 99% in the middle, okay, you divide that in half. So you have 4950, that's to the right of the zero, and 4950 to the left of it. And in each tail, you're taking the 1%, that's your alpha, right? 1-99% is 01. To cut 01.5, you have 005 in the right tail, 0.005 in the left tail. And now you go to your zero-to-Z table. And guess what? You find the Z value is 2.575. Okay, so plus and minus 2.575 will give you 99% of the area under the Z distribution. Okay, so now you take that value, 2.575 times S over the square root of N, which is called the standard error of the mean. So that's 1.85, that's S over the square root of 121, which by the way is 11. Okay, and I need X bar plus and minus that margin of error. So we do 6.10 plus and minus 2.575 times 1.85 over the square root of 121, which is 11. Your margin of error is 0.43. That's your margin of error, 0.43 years. Okay, so you do 6.10 plus 0.43, and then you do 6.10 minus 0.43, and then you have your confidence interval. Simple as that. This problem of looking at the life of a baseball, okay, a company wants to estimate the average life of its baseballs. So you're working for this company, so I'm going to take a random sample of 64 baseballs, and then you put it all the results in the spreadsheet, and then you get the X bar of the 64 and the 64. Your sample evidence now is X bar is 3.35 months under normal usage.  Remember, these are all two-sided intervals. You've got to cut everything in half. So you have 0.40 and 0.40, which leaves, if it's 80% confidence, 1 minus alpha, that means 20% is the alpha, and you've got to split that in half. So you have 0.10 on the right tail, 0.10 on the left tail. Okay, now you need a Z value. Which Z value will give you 40 from 0 to Z will give you 0.400 of the area under the curve. Okay, looking at your 0 to Z table, you find out that Z value approximately is 1.28. Okay, so now you have 3.35, that's your sample mean, plus and minus. Now that value, as you know, is called the margin of error. 1.28 times the standard error of the mean, that's Z times S over the square root of N, 1.28 times 0.888. 0.888 is the S, the square root of N, your N was 64. And you do that arithmetic and you find out your margin of error is 0.14. That's the margin of error. 3.35 plus 0.14 is 3.49 months, 3.35 minus 0.14 is 3.21 months. And now with 80% confidence, you can say this interval that goes from 3.21 all the way up to 3.49 months, that interval is 80% certain that in that interval somewhere fixed in place is your mu. Here's a graphic just to illustrate the point that the random variable we're looking at and that we're working with is X bar and even the interval based on X bar, the confidence interval estimator. We take a different sample, we get a different interval, we get another sample, get another interval. In this particular example, we happen to know, but we don't know because we don't have a census, that let's say mu is 6.4, 6.40. But depending on the sample that we take and the X bar value we compute and the interval we get out of it, we're not necessarily going to know this, but we will work with a certain level of confidence and do the best we can. If we were going to let's say construct a 90% confidence interval estimator of mu, some of the times the interval will cover, will contain mu, some of the times it won't. If this is working the way it's supposed to work, out of let's say a thousand such experiments, 900 of them should contain mu, should contain 6.4. With only 10 it's kind of hard to be exact, but that's all we have here in this problem for you. But just take a look at the graphic for a minute. Mu of 6.40 doesn't move. Some of the intervals are shifted over to the right, some of them shifted over to the left, some of them fairly over the middle, fairly residing, standing with a foot on either side of 6.4. But sometimes we're off. We have to get used to the fact we're operating under uncertainty. And in fact, out of these 10 confidence interval estimators, three of them did not contain 6.4. We do the best we can. We use the z-table, the values from the z-table, to imbue our results with a particular level of confidence. But we always have to remember there's that little bit that we're allowing that could be wrong. And we have to be very, very humble. And remember, sometimes we could be wrong. We could be doing, do the experiment again, replicate it, get different results. Either results that seem better or the results that seem worse, but it doesn't necessarily mean that the first experiment was wrong. Let's see if we can summarize some of the key points in this lecture in estimating from a sample to a population. When we're working with a sample, that's the whole point of statistical inference, we can never have 100% certainty. That's a key point, kept hammering it over and over again, and it's worth saying it again. If you want 100% certainty, you have to take a census. You're not going to get that from a sample. What can you get from a sample? You can state the confidence that you have that the estimator that you created, the interval estimator, contains the true parameter. That's basically what you can do. And we saw various problems with various levels of confidence to see how that works. The traditional level of confidence, if all else, if it's not stated, is 95% confidence. But whatever problems are given you, it'll tell you, the problem will say what confidence to use, and you should be able to use the Z-table to come up with any level of confidence. And of course, the more confidence, the wider the interval, the lower the confidence, the narrower the interval. That's the trade-off. As always, do lots and lots and lots of problems. It's going to stand you in good stead. It'll help you understand the material, and it'll help you on the exams, because when you look at a problem, if you've done a lot of practice, you'll be able to quickly recognize it as far as which type of problem it is, and you'll be able to remember how to solve the problem, and you'll be able to do it faster, because we all know in an exam situation, you don't want to have to go back and forth and think about stuff and look it up in your notes or in your textbook.


Paleobotany applied to freshwater plants is a field of paleontology that reveals evolutionary trends distinct from terrestrial and marine flora, with notable groups such as cyanobacteria and persinophytes in the Precambrian, green algae in the Paleozoic, carophytes in the Permian to early Cretaceous, and freshwater angiosperms in the late Cretaceous and tertiary. This article was authored by C. Martin Closes.


Oh, I just love watching these birds. Huh. Oh, hello, everyone. I'm Alicia Anderson, and welcome to the Fort Report. Migratory birds are coming through Texas, and although they are fascinating creatures to watch, they can also cause quite a few problems. They can be pretty foul at times, not very pheasant. You should prepare for them so that you don't regret anything. Get it? So you don't like bird puns. Well, I thought it was funny. In fact, I think I'm going to tweet it. They will literally take over a tree if you don't scare them away. They can build dozens of nests in one single tree, and you can imagine when they start having their babies and they stay for all summer long, the amount of poop on your yard, it covers your mailbox, it covers your driveway, it covers everything. Now is a great time for you to prevent egress and other migratory birds from nesting in your neighborhood. So the thing to do now is to check your trees, see if you have any old nests, because they will come back to an old nest. Clear out those sticks, get ready, trim your trees, and try to prevent the birds from finding your neighborhood and your yard and attractive habitat. Once the birds build the nests and lay an egg, you can not touch the birds because they are protected by an international treaty. The number one thing that you can do to help prevent egress nesting is to trim your trees. The birds like a dense canopy with very little light showing through. So if you can lighten up your trees, you don't have to butcher them, trim your trees, don't let the trees touch. They like it when they're like this and they can walk across them. It's a job you have to do with your neighbors. Everyone has to work at the same time to do it. If you don't have an active neighborhood association, please give the city a call. For more information about egress, visit the city website or contact the community engagement office. Here's a look at other events happening in and around Fort Worth in the next few weeks. For a listing of more events and city meetings, you always can check the calendar at the city's website.