every one thousand here. Today I got a couple sandwiches here.  I'll show it to you now and then we'll get into this. So I'll show you the clip of the place and then I'm gonna do a review on these two sandwiches. Stay tuned for that. Guys so we filmed this place a few years ago. It was a house and they turned it in. I'll see if I can find a clip and they turned it into a little restaurant deal. Okay so that was so that was the that house we we went looked we didn't get to go inside but we filmed the outside of it so that's pretty cool. As you seen we had some bad weather excuse me and it's still storming so but anyway I got them I got a meatball sub and I think was an Italian sausage sub so get you excuse me pointed down here and I thought I'd try them both if you guys are ever in this area that is actually known as Wildwood here in Florida. I'm guessing it's just a family owned mom and pop place because well the younger there was a older couple in a younger girl and she was calling the older guy dad so I figured it's just a mom and pop place and she asked me if I wanted to play it I said no I should have got one and I'm telling you what it smells like fair food so this one this is the meatball let's give this a shot here now let me point you back up here sorry guys this my mind's not the right place right now but here we go very good you get tastes like a math affair and a county fair I love that food I've actually heard people say I can't stand for food you can't stand the smell most people love it that's definitely a two thumbs up and a four out of five for the meatball sub pretty reasonable because I got two foot long she got one so I actually got two orders they come with a bag of chips a small bag of chips let's go ahead and open this up I got a knife right here let's get this one open I'll show you this one too there's the there's the sausage in there let's give this a shot whoa whoa messy okay let me try it again with the bite of the meat hmm excellent these things are spectacular carries corner so if you guys are they're both four out of five two thumbs up and as far as the people awesome people they were great I said you guys are brave they were great people man just down there real friendly folks so I want to do a review I'm turning what I'm I'm gonna go four and a half out of five stars on each one of these and I'm gonna five star for the people it's wow this is really something very good so I'm gonna enjoy my lunch and then we're gonna start on that closet I'll film that for you guys but it's gonna be in a separate video so anyway that's been our day so far so kind of late in the day it's almost three it was 247 so thanks for watching guys appreciate it she barely met the man legend I'm gone for now and we'll see you guys in the next one bye bye stay safe and take care guys


Hello friends, I am Sanjay Gupta. In this video, I am going to demonstrate you how to use FL statement in macro in C programming. Before starting, you can note how you can search my YouTube channel. You can type my name in YouTube. You can watch various program related videos here.  So, first of all, I have included a header file that is stdio.h. Now, I am going to define a macro which will be using FL statement. So, name of macro will be min. It will be receiving two parameters a and b. So, this macro will be checking which number is minimum with the help of this replacement token list. So, here I am implementing if else. So, this is if condition if a less than b then printf will display %d is minimum else printf %d is minimum, b. So, this way I have implemented this replacement token list. Here, if condition is available, if it is true then printf will print value of a as min otherwise else will print value of b as minimum. Now, I have to implement main function so that I can use this macro. Here, I am going to declare two variables of integer type. This printf will be displaying the message enter two numbers on output screen. Scanner statement will be reading entered numbers and those numbers will be stored inside a and b variables. Now, I am going to use that macro. So, here I have used min macro. This min statement which is written here will be automatically replaced with the replacement token list that is available here if a less than b printf else printf. So, this complete code will be available in place of this min a, b at the time of compilation and at last I am using return 0. So, this way I have implemented fl statement and macro statement which is defined with the help of hash defined preprocessor. Now, I am going to execute this code. So, you can see it is asking for two numbers. So, I am entering first number as 100 and second number as 200. You can see the output. It is showing 100 is min. So, the program is working properly. The defined macro is working properly. If you want to define more than one macros and in both the macros you want to implement this fl statement that is also possible. So, suppose you want to define another macro as max. So, you can define that macro like this. So, here if a is greater than b then a will be printed as max otherwise b will be printed as max. I am also using slash n so that output will be displayed on new line and here max a, b. So, this way I have implemented two macros. Both the macros are using fl statement as a replacement token list and I have used both the macros in main function. Now, again I am going to execute this code. You can see it is asking for two numbers. I am entering 100 and 200 as two numbers. So, you can see the output. 100 is min and 200 is max. So, both the macros are working properly. So, this way you can implement fl statement with macro with the help of hash defined. I hope you have understood whatever I have demonstrated you in this video. If you want to watch more programming related videos you can search my name Sanjay Gupta in YouTube. You can also download my programming app TechMS which is available on Google Play. Thank you for watching this video.


Hi everyone, Sean with jaspyscasebreaks.com here doing 2020-2021 Panini Don Russ basketball two box hobby break number seven two random teams each sold out thanks to the late night rally everyone gets two teams in the NBA all cards ship looking for these nice press proofs of the net marvels and the crunch time plus raided rookies and raided rookie autos all card ship on release day so nice stuff here here is the customer list last spot mojo armar Omar is that who we back up to Eric Angel with the first spot purchased thank you guys everybody that got in and since every spot gets you two teams we need to copy this list and double it up so you can see Omar with last spot mojo once and twice and then back up to Eric with the first spot here's a list of teams from the hawks down to the wizards dice roll to randomize both lists and pair them up six and a one seven times good luck one seventh and final time from tom down to louis i'm gonna title this one here so this paper doesn't get lost there you go so from tom back down to louis you can see seven times on the randomizer seven times on the dice now seven times on the teams seventh and final time from the atlanta hawks down to the los angeles clippers so let's see how it turned out for everybody tom judge with the hawks joseph barton with the jazz jason sebert with the pacers louis ramirez with the spurs carlos ramirez joseph barton with the pelicans jason sebert with the pistons tom judge with the nicks anthony chan with the nuggets nix tober with the timber wolves and the kings joseph barton with the nets carlos ramirez with the mavericks louise ramirez with the grizzlies jason sebert with the Celtics ohmarr zahui with the cavaliers carlos ramirez with the warriors ohmarr zahui with the last spot mojo spot with the magic jason sebert with the Lakers. Eric Ainge with the Wizards. Luis Ramirez with the Rockets. Omar Zui with the last spot mojo. The second portion with the Suns and he also has the Blazers. Anthony Chan with the Hornets. Nick Stober with the 76ers. Eric Ainge with the Raptors. Nick Stober with the Thunder. Carlos Ramirez with the Bucks. Joseph Barton with the Bulls and Luis Ramirez with the Clippers. Anthony with the Hornets again. Congrats. So here is the list. Trade window is open. So you guys can see who has which teams if you'd like to try to trade. I know quite a few guys are actually in here listening and watching right now. You'll see the case that we started. This is a second break, second two box break from this case. Number six started this case. So what we're going to do is I'm going to roll a dice to figure out which row we're doing the boxes in. So this is row one, this is row two, row three, and the bottom row down here is row number four. So we're going to do the two boxes next to each other depending on which dice roll it is. If we roll a five or a six, there are one per box, Thomas. Obviously the rated rookie autos are the ones we're looking for. Biggest ones. If we roll a five or a six, we'll just re-roll the dice. So here you go. It's a four. That was easy. No re-roll needed. So we're going to take the bottom boxes down here. Take out this one first. There you go. So these are our two boxes.  The trade window if anybody wants. If there's no talks of trades, we'll go ahead and close the trade window and get ripping. I'll give it another 20 seconds. Down to 10 seconds guys. Going once, going twice, going three times, and trade window closed. Let's go ahead and print the list and rip the boxes. Good luck. So those are our two boxes here. You can see their marks with the X as well to show that these are all from that same case that we started with. Two box break number six. And here is our final list. There you go. Thank you guys all for getting in on this and good luck. So potentially last break of the night unless anybody wants to rally and get another one of these breaks done. We've got plenty more on the site. It's close to going. Depends on whether you guys want to do it tonight or get some sleep, come back, do it all over again tomorrow with Joe. First box. Good luck everybody. Jason Tatum for the Celtics. Jason Siebert and Josh Green. Rated rookie Otto 17 out of 49 for the Dallas Mavericks. Carlos Ramirez. There you go Carlos. Congrats. Wesley Matthews to 99 for the Bucks. Carlos Ramirez. Al Horford to 349 for the 76ers. Nick Stober. LeBron James. Net Marvels for the Lakers. Jason Siebert. John Morant, crunch time for the Grizzlies. Luis Ramirez. Yanisanta De Campo. Craftsman for the Bucks. Carlos Ramirez. Deniav Deja. Rated rookie for the Wizards. Eric Ainge. And Onyeka Okangwu for the Hawks. Tom Judge. Submojo. Not yet. I haven't yet. I just saw your question Tom. John Morant for the Grizzlies. Luis Ramirez. Precious Ashua. Rated rookie Blue to 49 for the Miami Heat. Carlos Ramirez. How you doing today Mojo? Ben Simmons. Net Marvels for the 76ers. Nick Stober. Zion Williamson crunch time for the Pelicans. Joseph Barton. Emmanuel Quickly for the Knicks. Tom Judge. Always good man. Always good. Udoka Azebuki to 199 for the Jazz. Joseph Barton and Kevin Love to 349 for the Cavaliers. Omar Zoe. Luka Donchich Craftsman for the Mavericks. Carlos Ramirez. And Rui Hachimura. Net Marvels for the Wizards. Eric Ainge. Killian Hayes for the Pistons. Jason Siebert. Patrick Williams for the Bulls. Joseph Barton. You got it Ozzy. I can do that. Zion Williamson for the Pelicans. Joseph Barton. James Harden to 99 for the Rockets. Luis Ramirez. Crunch time. Luka Donchich for the Mavericks. Carlos Ramirez. Luka Donchich. Net Marvels. And Pressproof. Trey Young. Net Marvels for the Hawks. Tom Judge. That Luka. Net Marvels going to the Mavericks. Carlos Ramirez. Precious Ashua for the Heat. Carlos Ramirez. Wendell Carter Jr. to 349. Four of the Bulls. Joseph Barton. Paul George to 199 for the Clippers. Luis Ramirez. James Harden. Crunch time for the Rockets. Going to Luis Ramirez. Obi Toppin for the Knicks. Tom Judge. Cole Anthony for the Magic. Going to Ozzy. Tyrese Halliburton for the Kings. Nick Stover. Isaac Cacoro for the Cavaliers. Omar for Ozzy. James Weissman for the Warriors. Carlos Ramirez. There you go. Nice first half of the box. John Wall to 349 for the Wizards. Eric Ainge. RJ Barrett. Net Marvels for the Knicks. Tom Judge. Paul George. Crunch time for the Clippers. Luis Ramirez. Precious Ashua for the Heat. Carlos Ramirez. Cole Anthony for the Magic. Going to Ozzy. Tyrese Halliburton for the Kings. Nick Stover. Grant Hill. Jersey Kings. Relic for the Detroit Pistons. Jason Siebert and Patrick Williams. 12 out of 99. Rated Rookie for the Bulls. Joseph Barton. Marquise Chris to 349 for the Warriors. And Carlos Ramirez. James Harden. Net Marvels for the Rockets. Luis Ramirez. Trey Young. Crunch time for the Hawks. Tom Judge. Isaac Cacoro for the Cavaliers. Omar Ozzy. James Weissman for the Warriors. Carlos Ramirez. Tyler Bay for the Mavericks. Carlos Ramirez. Enos Canter to 199 for the Celtics. Jason Siebert. Zach Levine. Net Marvels. For the Bulls. Joseph Barton. James Harden. Pressproof. Crunch time for the Rockets. Luis Ramirez. Peyton Pritchard for the Celtics. Jason Siebert. Tyrese Maxey for the 76ers. Nick Stover. Lamello Ball for the Hornets. Anthony Chan. Luca for the Mavericks. Carlos Ramirez. Manuel Quickley to 349 for the Knicks. Tom Judge. Kobe White to 199 for the Bulls. Joseph Barton. Joellen Bede. Crunch time. 76ers. Nick Stover. Kyrie Irving. Net Marvels for the Nets. Joseph Barton. Theo Maladon for the Thunder. Nick Stover. RJ Hampton for the Nuggets. Anthony Chan. LeBron James for the Lakers. Jason Siebert. Derek Rose to 349. For the Pistons. Jason Siebert. Kyrie Irving. Crunch time for the Nets. Joseph Barton. Lamello Ball for the Rookies. For the Hornets. Anthony Chan. Obi-Toppin. Pressproof for the Rookies. For the Knicks. Tom Judge. Devin Vossel for the Spurs. Luis Ramirez. Obi-Toppin for the Knicks. Tom Judge. Anthony Edwards for the Timberwolves. Nick Stover. There you go. That is box number one. Very nice first box. Let's see it improve on the second one. Tom Mojo which one do you think which set do you think will be the Net Marvels of last year? Do you think Net Marvels this year will be as popular or do you think the crunch time will be? I'm kind of liking the crunch time. Cool. I agree. I think that's this I think that's this this year's Net Marvels. Good looking set. I agree Mojo. I think it's incredible. I think uh I think the good the good move is to get the pressproof versions right now. Yeah for sure. Later guys. I think get those get the pressproofs now while people are chasing the Net Marvels. Scoop up as much as many of the the net the crunch time ones. Good luck guys. Box two and rated rookie Otto Trey Jones for the San Antonio Spurs. Luis Ramirez. There you go Luis. Zion Williamson to 99 for the Pelicans. Joseph Barton 71 out of 99. Kevin Durant to 349 for the Nets. Joseph Barton. Crunch time Kyrie for the Nets. Joseph Barton and Lamello Ball. The rookies for the Hornets. Anthony Chan. LeBron James. Net Marvels for the Lakers. Jason Siebert. Tyler Bay for the Mavericks. Carlos Ramirez. Peyton Pritchard for the Celtics. Jason Siebert. Tyrese Maxi for the 76ers. Nick Stover. Zion for the Pelicans. Joseph Barton. Jaren Jackson Jr. to 199 for the Grizzlies. Luis Ramirez. John Merrant. Crunch time for the Grizzlies. Luis Ramirez. Yannis Craftsman for the Bucks. Carlos Ramirez. Lamello Ball for the Hornets. Anthony Chan. Theo Maladon for the Thunder. Nick Stover. Luca Donchich for the Mavericks. Carlos Ramirez. Yannis for the Bucks again. Carlos Ramirez. Shae to 349 for the Thunder. Nick Stover. Harrison Barnes to 199 for the Kings. Nick Stover. Ben Simmons. Net Marvels. 76ers. Nick Stover. Zion Williamson. Crunch time for the Pelicans. Joseph Barton. Luca Donchich. Craftsman for the Mavericks. Carlos Ramirez. RJ Hampton for the Nuggets. Anthony Chan. Isaiah Stewart for the Pistons. Jason Siebert. Devon Vosel for the Spurs. Luis Ramirez. Obi Toppin for the Nets. I'm sorry for the Nicks. Tom Judge. LeBron for the Lakers. Jason Siebert. Skyler Mays to 349 for the Hawks and Tom Judge. Rui Hachimura and Net Marvels for the Wizards. Eric Ainge. Luca Donchich. Crunch time for the Mavericks. Carlos Ramirez. James Harden. Net Marvels. Press proof for the Rockets. Luis Ramirez. Anthony Edwards for the Timberwolves. Nick Stover. Robert Woodard to 99 for the Kings. Nick Stover. Carl Anthony Towns to 349 for the Timberwolves. Nick Stover. Luca Donchich. Net Marvels for the Mavericks. Carlos Ramirez. Danny Avdija. Washington. Eric Ainge. Onyeka. Kongwu for Atlanta. Tom Judge. Cool. Second half. Good luck guys. Steph Curry. Crunch time for the Warriors. Carlos Ramirez. Tyrese Halliburton. The Rookies for the Kings. Nick Stover. RJ Barrett. Net Marvels for the Nicks. Tom Judge. Danny Avdija for the Warriors. Eric Ainge. Sorry Wizards. Onyeka. Kongwu for the Hawks. Tom Judge. Chris Mullen. Relic for the Warriors. Carlos Ramirez. Tobias Harris to 199 for the 76ers. Nick Stover. And Nick Richards to 349 for the Hornets. Anthony Chan. Paul George. Crunch time for the Clippers. Luis Ramirez. What names? I try. Manuel quickly for the Nicks. Tom Judge. I feel like I don't have that ability when it comes to soccer. Jason Tatum for the Celtics. Jason Siebert. Bogdan Bogdanovich to 49 for the Kings. Nick Stover. James Harden. Net Marvels for the Rockets. Luis Ramirez. Trey Young. Crunch time for the Hawks. Tom Judge. Paul George. Crunch time. Press proof for the Clippers. Luis Ramirez. Killian Hayes for the Pistons. Jason Siebert. Patrick Williams for the Bulls. Joseph Barton. And last stack. John Morant for the Grizzlies. Luis Ramirez. Thon Maker to 349 for the Pistons. Jason Siebert. And Isaiah Stewart to 199 for the Pistons. Jason Siebert. Zach Levine. Net Marvels. For the Bulls. Joseph Barton. Crunch time. Joellen Bied for the 76ers. Nick Stover. Precious Ashua for the Heat. Carlos Ramirez. Jason Tatum. Celtics. Jason Siebert. Kyrie Irving. For the Nets. Joseph Barton. Cole Anthony for the Magic. Ozzie. Tyrese Halliburton for the Kings. Nick Stover. Isaac Acoro for the Cavaliers. Ozzie. James Weissman for the Warriors. Carlos Ramirez. And there you go. That ends it for you. There is our break. That was 2020-2021 Panini Don Russ basketball. Two box hobby break. Random team each. Two teams. Number seven from jaspyscasebreaks.com on release day. I'm Sean, jaspyscasebreaks.com. Thanks for hanging out. We'll see you next time.


Okay, I'm going to just kind of introduce the topic here, talk about metamorphic software, you know, what it's good for, potential good uses, potential evil uses for metamorphic software. And also just a little bit about worms and viruses and malware in general, kind of the history, just a very quick overview of that. Then Wing is going to take over and talk about her research work, which was very impressive I can say because I didn't do it. It's all about, she looked at some specific metamorphic engines that you can find online and there's hundreds of these out there. She looked at four of them that seemed to be the best according to a lot of sources. And she tried to answer two questions about these engines. First of all, you know, how good are they in the sense, how well do they morph the code? Okay, how metamorphic are the variants that they create, the virus variants? And so she did a lot of work on that. She'll present some of that and show you her results there. But there's kind of a separate, you know, but yet related question about how hard are these things to detect, okay? You know, intuitively you would expect the more metamorphic that these viruses and worms are, the harder they're going to be to detect. But I think the conclusion is a little bit surprising in that there's at least a little more to it than that, okay? And then some conclusions. Okay, metamorphic software. Okay, so our definition is that software is metamorphic. If all copies of the software do the same thing, okay? The functionality is the same, but yet the internal structure is different, okay? So somehow you've gone in and you've changed the internal structure of the code, but you've been careful. You've done it in a way that does not change the function of the code at all. Now today, this is not the way it's done, okay? You don't get metamorphic software. You know, there's a lot of reasons for that. It's potentially harder to develop, harder to maintain, and so on. But it's not impossible. I mean, actually at my failed startup company, we did some of this. So it is possible. You can do this sort of thing. Today though, the software you get is cloned, okay? You create one copy, you create one instance of this, you create one copy of the software and then give everybody the same thing, okay? So everybody's got a clone of this one piece of software. Okay, so that has, you know, implications for security, okay? A lot of implications there, okay? First of all, well, you can think of potentially good uses for metamorphism and potentially, you know, evil uses for metamorphism. So just to give you one example of each, one thing you could think about, you know, you could do this thought experiment. Think about, you write a program, it has a buffer overflow in it. You know, you didn't intentionally put it there. It's just an accident, it has this buffer overflow. Now if you, now maybe that's exploitable, so someone can write an attack and they can attack your code. Now if you clone your software, your original software, and send out the exact copy to everybody, well somebody creates an attack, that same attack can obviously work on every clone copy. They're identical, okay? But if you try to use metamorphic software, you potentially get some advantage there. And I'll talk a little more about that on the next slide. Okay, so that's a potential benefit, okay?  Potential bad use, which Wing's going to talk about, and this is kind of the main point of the talk, is using metamorphism to try and avoid signature detection for worms and viruses. Okay, so let's look at this supposedly good use of metamorphism here a little closer. Again, you have this program, you write it, it has a buffer overflow in it. If you clone it and send it to everybody, the same attack's going to work. So there's one attack, it breaks every copy. And it's sometimes called break once, break everywhere phenomenon. You create one attack, it works everywhere, universally. On the other hand, suppose you created metamorphic copies of the software, okay? Now, every copy is functionally the same, but the internal structure's a little bit different. Now, if someone gets a copy of this, every copy still has the buffer overflow in there. You didn't get rid of it just by creating some metamorphic copies, it's still there. So if I get a hold of a copy of it, I can create an attack probably is going to work against this particular copy. What's going to happen if I try the same attack on all the other metamorphic copies of the program that's out there? Well, chances are it's not going to work. If you know how buffer overflow attacks work, it's a pretty delicate affair, right? Things have to align in just sort of just the right way to make it actually work. And in fact, I had a student do a project on this as well, actually got the numbers and really implemented it and tried to see how often you could get the buffer overflow attack to work on the metamorphic copies and it almost never works, okay? One percent or something like that. So the point here is that just a little bit of metamorphism, it doesn't have to be anything really clever, okay? In this instance, for buffer overflow protection, just this one example, a little bit of metamorphism does a lot of good, okay? It's sort of comparable to genetic diversity in software, genetic diversity in biological systems, okay?  It kills 40% of the population because there's genetic diversity, okay? So it doesn't affect everybody. So you're trying to get that same sort of effect here to prevent these buffer overflow attacks. On the other hand, if you wanted to use this kind of metamorphism to make viruses and worms that are hard to detect, the point here is that you're trying to avoid having a common signature that can be discovered, okay? And if you have enough metamorphism in your virus or worm, presumably you could avoid this kind of detection. And in fact, some people talk about undetectable viruses that are metamorphic. And is that possible? Seems it's conceivable. Okay, so every copy has a different signature. And again, same point, you're really trying to get this sort of genetic diversity into your population of viruses. So the defender who's trying to discover them has a harder time trying to pick them out, okay? Because they have different signatures. But, okay, the kind of the conclusion I think you'll get out of wings part of the talk here is that this is trickier than it sounds, okay? You really need a lot of metamorphism. And you have to sort of use it in the right sort of way. So that's in contrast to sort of the, just keeping your back your mind, the buffer overflow case, you just need a little bit of metamorphism and sort of a lot of bang for the buck. But if you're trying to use metamorphism in this case, it's tricky, okay? And that's kind of the main theme of the talk. Okay, so just a few quick words here about the evolution of viruses. And then I'll turn it over to Wing and she'll get to the real substance of the talk here. Viruses and worms are not new things, okay? They've been around since the 80s, which is ancient history in the computing world. And once they got started, it pretty quickly became that the game is that the defenders are trying to discover a signature, okay? A signature that they can latch on to, some sort of common feature of the virus or worm that they can pick out and they can identify. Well, okay, the writers of these viruses and worms, they realize that too, and that's the game, so they're gonna try and hide the signature. And what can they do to hide the signature? Well, sort of three different techniques have been used extensively. Encryption, polymorphism, and metamorphism. Now, these terms are used, you know, not consistently in the literature. So here's what we mean by these, I guess. So if you encrypt a virus, okay, that's certainly a way to hide the signature. Use a different key, it's just a different random string of bits, there's no common signature. Well, the problem is, of course, that you have to decrypt the virus before it can actually do anything. So you have a little piece of decryptor code there, and that code can be detected by some signature detection means. Excuse me. Okay, so you could then encrypt the decryptor. But then you need a decryptor for the decryptor, you know, get this infinite regress, okay? So you have this problem, this basic problem that you need, some piece of code there that's not encrypted, and that can be potentially be detected. Okay, so polymorphism by our definition here, is kind of taking that to the next level.  Okay, that sort of thing's been done, there's been viruses of this sort. They can still be detected, you can do emulation techniques and such. But it does certainly up the pain, it's harder to detect these sorts of things. Finally, kind of the ultimate here would be metamorphism. Can you create metamorphic worms and viruses? So every time, say the worm mutates, it changes its internal structure in some meaningful sort of way. Well, if you could do this, potentially you would have viruses and worms that are not detectable, they don't have any common signature at all. So in a sense, what you're trying to get here is one way you can think of it is you're trying to sort of change the shape of the virus or worm each time that it mutates. And what sort of techniques would you use? Well, you can think of lots of different things. I just list a couple things here that are commonly used. Okay, so that's just kind of a hand-waving introduction. Now I want to turn it over to Wing and she's going to talk about these various virus construction kits. How metamorphic they are and how can you go about detecting the viruses that they produce? Yeah, so we see that a lot of viruses are written in assembly code. So writing sophisticated and well-functioning assembly code may not be easy. But luckily, if we go online, we can easily find a lot of virus construction kits that come even with very sophisticated, fancy, interactive, interface that can let you just click, click, click, and then you can generate a lot of viruses. Some of them come with, they claim to have very sophisticated features. Like they can produce metamorphic virus, meaning all the variants that it generates look different. Even from identical configuration or the same kind of input that you supply. So we're interested in looking at how effective these generators are in terms of doing metamorphic, producing metamorphic code or viruses. We look at the literature and find out some that are really highly discussed or have some good comments. The first one is the PSMPC. And according to a virus research book, it's working as a cold-moving engine. And the viruses it produce have decryption, routines, and structures that change in variants. Let's look at the other one. G2, we also download from the web. And then its documentation actually says that different viruses may be generated from identical configuration files. And we actually tested these configuration files and generated different viruses. This seemed to be the best one that we looked at. And just according to its documentation, it says that all viruses it created are completely different in structure and opcode. So that is impossible to catch all variants with one or more scan string. And it claims that it's really 100% variable. It has 100% variability of the entire code of the viruses. So we are really interested in seeing how effective these engines are in terms of producing different looking codes. So are they effective? How do we measure that? First of all, I want to go through the metric that we use to compare two pieces of code because we are comparing whether two virus variants look different. So this is the method we use. We take two assembly programs, extract all the opcodes like excluding blank lines, comments, labels. We only get the opcodes and also exclude no op. So we get two sequences of opcodes. One is of length n and we number them 0 to n minus 1. Second program say may have m opcodes and we label them 0 to m minus 1. And for comparison, we compare three subsequence at a time and we do all against all comparison. So for program X and program Y, if the three opcodes are the same in any order, we consider it as a match. So if there's a match, we plot the XY coordinate. X means the opcode number of program X and then the Y is the opcode number of program Y to generate a graph of matches, meaning that three opcode matches in any order. Since there may be random matches and also noise, so we keep in the graph only matches that consist of five points or more. Five is our cutting point. So we call those lines that have five points or more real match lines. So we get the second graph that consists of all the real matches. After that, we walk along the X-axis and the Y-axis for each program. We count the number of opcodes that are covered by one or more of these real match lines. So we count the number of opcodes that are identical to other opcodes in the other program and we find the percentage for each program. So the similarity score for two programs is the average of these two percentages. So if two programs are identical, we'll expect to have a score of one. And the less similar two programs are, the score would be closer to zero. And also, because we are doing sequential match, so identical segments of opcodes would form strict lines parallel to the diagonal. And if they are at the identical positions of the two programs, we will actually see these diagonal lines fall right on the diagonal. If there is some translation, we will still see these lines, but they would be off the diagonal, meaning that there is a translation. Or the opcodes appear in different locations of the two programs. All right? Okay, as we said, we tested four generators. We considered each generator a single family, meaning the viruses generated by one single generator belonging to one family. So we generated different number of viruses from these four generators. And then we measure the similarity among pairs of viruses from the same generator. This is how we measure the degree of metamorphism produced by each generator. So for comparison, we also find used 20 utility programs from SickWin. This program we take of similar size to the virus executables. And we also believe that the viruses will be doing some kind of low-level activity similar to the SickWin utilities. That's why we pick those programs for comparison. Here, we are comparing similarity within a family, meaning we compare virus variants. And we can see all the 190 pair-wise comparison among the NVIC viruses has a score very close to like 10%. They have score range from 0.1% to 21%, and the average is 10%. A great contrast is the comparison between normal programs. They can have similarity from 13% to 90%, but the average is around 35%. So in terms of metamorphism, we actually see that NVIC did a very good job because the virus variants that it produced actually are very dissimilar according to our metric. And normal files are much more similar in this case. How about the other generators? Here, we use a bubble graph to represent the outcome. We plot the minimum similarity score that we obtained from among all the comparisons. We plot it minimum along the x-axis. Maximum score is along the y-axis, and the bubble size represents the average similarity score of that particular generator. So if a generator is effective, meaning that the variants that it generates do actually look different, we would expect to see a bubble that is very small and very close to the origin because the minimum, maximum, and average should be small for effective metamorphism. So again, we can see here the NVIC generator actually outperformed the other three in terms of generating different looking viruses. And if we look at normal program, it actually has a lower, minimum, lower maximum, and lower average than the other three generators. So G2 in this sense may not be that effective because its average similarity is higher and also have a higher minimum similarity. If we look at some of the graphs that we generate while we do the pairwise comparison, we again see similar result, right? The first pair is a pair of NVIC viruses that has similarity of around the average similarity, which is 12%. So we can see that they are very short matches and they are all way off the diagonal, meaning the matches are at different locations. Compared to, say, G2, we said G2 is kind of not as effective. We can see that they have the graph that they produce. We see a long diagonal line and they are 75% similar for this pair. VCL and MPC-GEN also have very low degree of metamorphism, according to our measure, and they also produce graph that shows this. So now we see that NVIC viruses actually look very different from one another. So it seems that they may be able to avoid detection, signature-based detection because of its high degree of metamorphism. But other than that, we are interested in knowing how similar is it to the other viruses or to normal program. Are they in any way similar to other viruses or the normal programs? So we do the same kind of similarity comparison, and then we use all the same viruses that we use previously and just do a cross-family comparison and see how similar they are. Very surprisingly, NVIC viruses had 0% similarity to both G2 and MPC-GEN viruses, and to VCL-32 viruses, they're only 0 to 5.5% similar. More surprisingly is the comparison to normal file because we did 400 pairs of comparison and we only find eight pairs that have some kind of score, which are as low as 0 to 1.2%. So what does this tell us? We see that NVIC is by far the highest degree of metamorphism of the kids that we tested, and it has no similarity to other viruses or normal program. So would that make it undetectable? That's our next question. So can we detect them? We test three different scanners, and we use these three scanners, E-Trust, Aves, and AVG, to scan 37 files that are put in one folder, and these 37 files include viruses from all four families. Here's our result. E-Trust and Aves detected all the G2 and MPC-GEN viruses. AVG detect 27, which are the G2, MPC-GEN, and VCL 32 viruses, but none of the NVIC viruses are detected by these commercial scanners. So what are we going to do? It seems that they actually are quite successful to avoid detection by these scanners. We want to detect them. So we turn to Hidden Markov models. Hidden Markov models have been used in speech recognition since the 1970s, and it has been applied in a lot of applications because of its ability to do pattern recognition. It's a very powerful tool for statistical pattern analysis, and it has been used in biological sequence analysis as well. So our technique is actually inspired by protein modeling. In protein modeling, we would use one model to model a whole protein family. What is a protein family? Protein family is a protein that has similar three-dimensional shape and also have similar or common chemical and physical property. So the thing about family is that even though they behave the same way, but the underlying amino axis sequence may not be totally identical. So because we all know proteins are made up of amino axis sequences, proteins belong to the same family, may have similarity as low as 10% when we examine the amino axis sequence. But we can use a Hidden Markov model to model one family, like given a lot of training data. And after that, the model can distinguish whether any new given protein belongs to the same family. So we look at our viruses. They have the same behavior. They do the same thing, but their structure are different. So there must be some statistical similarity exists between all these assembly programs to make them behave the same way. So we believe that Hidden Markov model can capture this similarity, and we can use it the same way we do with protein modeling and try to use it to detect whether any given program belongs to a same family of the training viruses. So this is what we do. We use Hidden Markov models to represent the statistical property of a set of metamorphic viruses. So we train the model with one family virus and then use a trained model to determine whether any given program is similar to the training viruses. The good thing about Hidden Markov model is that a trained program would maximize the probability of seeing the training sequence. And it would also assign high probabilities to sequences similar to the training sequence. A model would represent the average behavior of multiple sequences, meaning that we don't need many models to represent one family. We just need one model to represent a whole family, and we can later determine whether any given program is similar to that family. So this is how we do. So we generated a lot of these NVIC viruses. We used 200 of them. For training, to train a model, we use 160 of them. And we keep 40, which is not used at all in the training, to test whether the trained model can classify these 40 programs correctly. Also, we want to know whether a program can distinguish a normal program from these NVIC viruses. So we have 40 other executables from SIGWIN. And then we also tested 25 other non-family viruses, which are just the viruses we use in the similarity test. In our tests, we generated 25 models using different number of hidden states. The number of hidden states ranged from 2 to 6. And then we choose different sets of 160 training sequences to generate different number of models and do the same kind of tests. So this is a diagrammatically view of the whole process, as I explained. So from all this, we would actually expect to see that after training, the model can assign a high probability to the 40 test file when we compare to the other normal programs of virus not in the same family. So from that, we should be able to determine the threshold. And then later on, when we are given any program, program A, we can score the program with the hidden Markov model and then compare the score to the predetermined threshold and classify whether the new program that we are given belongs to the same family. This is a typical result of all the 25 models. As we can see, the model, in fact, classify or assign high values to family viruses. And the probabilities that are assigned to normal files are much lower. So if we can reasonably select a threshold that is within, I mean, that can clearly identify them, we can actually achieve 100% detection rate to classify when we try to classify or distinguish a normal file from an NVIC virus. If we include the other virus in the test, we see that some non-family virus actually has scored very close to the family virus or the NVIC viruses. We take a look at those viruses. They are the VCL32 viruses, which show some similarity in the similarity test. So we actually can say that the hidden Markov model determines some of these other viruses for free. I think the bottom line is that we never misclassify a normal program. We would not say a normal program is an NVIC virus, but for some virus, if they have some similarity to the NVIC program, it would actually show a higher score, which is what a hidden Markov model would do. So this is a summary of our experimental result. All normal programs can be distinguished by the score alone. And VCL32 viruses had scored closer to the NVIC viruses. And if we set the threshold properly, 17 of our models had 100% detection rate and 10 models had 0% false positive rate. And as I explained, we tried different number of hidden states. So for three or more hidden states, we didn't find any significant difference in performance. So here we, with the hidden Markov model, we can actually look at the converged final matrices to get some insights into the features that a virus represent. So we train our models on the NVIC viruses. So if we are good enough, we can look at the probability distribution of the models and then determine what kind of features they have. We observe that the opcodes are actually all divided into stages, meaning that most opcodes appear in one state only. So if any one of you are good at assembly programming, you can actually look at the assembly code and see the classification and try to understand what kind of features this virus has. So we've seen that even though NVIC viruses were not detected at all by the three scanners that we tested, we can actually use hidden Markov model to do some kind of classification and they are quite successful. Besides that, we took a completely different approach because of the fact that these viruses are so different from normal program. We tried a different approach. We just use a straightforward similarity index to determine whether a program belongs to the NVIC family. This method will work this way. If you want to see if program A is a NVIC mutated virus, you can just pair it with any NVIC virus and find the similarity score. Most of the time you would expect to find zero similarity because it's actually very different and most of the time we actually define zero. If it has zero similarity, we can immediately conclude it's not NVIC virus. But if this program A has some similarity to the chosen NVIC virus, what do we do? We pick that NVIC virus and then compare it to 20 other NVIC viruses to see the average similarity of this virus to the other virus in the same family. If the similarity, all these similarities among the family is lower than, let me see, is lower than the similarity we just obtained between program A and that virus, we can say that this program is most likely a virus. So this is how we use a similarity index to classify a program and we actually test that. We compare 105 programs composing of normal regular program and also some are NVIC viruses, some are other viruses and use the method I just described just based on similarity index and compare average result. I can classify 100% of the time correct that whether the 105 program virus or not and there's no false positive. And I try this on different viruses and I find out that it doesn't depend on the specific virus that you select in the first place. So what does this tell us? First of all, from the comparison test, we know that metamorphic generators actually vary a lot. Even though we can find a lot on the internet, they may not be all effective in terms of generating metamorphic codes and they seem to be the most effective because the viruses they generate are 10% similar. Other generators are less effective, some are 60% similar or even 70% similar for the other generators. When we compare to normal files, they are only 35% similar. So we know how to rate them. But NVIC viruses can also be detected because they are too different from other virus and normal programs. Even though NVIC viruses are not detected by commercial scanners, they are detected by our hidden Markov model approach with very high accuracy and also by our straightforward similarity index approach. So we see that all metamorphic viruses, including the all four family, were detectable either because they have high similarity with thin family, meaning they have low degree of metamorphism or they are too different from normal programs. So we actually conclude that the effective use of metamorphism by any virus or worms actually require both a high degree of metamorphism and also some similarity to other programs. And this is not trivial and this is not as easy as it seems. I think that kind of explain why we don't see like many metamorphic virus outbreak because they are actually not that easy as most people would believe. We see that metamorphism for good use is easy. Just like as Dr. Sam said, like the buffer for problem, if you have just a little bit of metamorphism, it actually do a lot of good. But for evil use, say in a virus case, it's possible to get high degree of metamorphism. But if you want to do it at the same time to be similar to other programs, it may not be that easy. So I think that's the major point of our result. Yeah, so if you have any question, please go to the microphone there. I have two questions I guess. Oh, there we go. First one, your training set was way bigger than your testing set. Oftentimes people do like 50-50 split. Why'd you do that? What was your thought? This is a classical five-fold cross-validation approach. Yeah, so we actually divide the data set into five equal-sized subsets and training use four subsets and testing use one subset. This is a classical approach. Yeah, we do it the same way in protein modeling. Okay, and then as far as getting a lot of metamorphism but still having similarity to other programs, you might be able to do something like have sort of tropes or common sequences of op codes that perform some higher-level functionality and have a family of those and pick one of those and Alvar's probably gonna say that that's one of his problems that he's gonna talk about. Anyway, so did you give any thought?  Yeah, no, that's a good point. So certainly that's a possibility. This was, her research was kind of limited and so that was what she did. I actually have two more students looking at this, other aspects of this problem, although not specifically what you mentioned but that would certainly be a good project as well. I just have a question concerning the working of this actually well-working metamorphic thing in which manner does it obfuscate the control flow of the executable? Does it just replace one instruction with like a set of other instructions or does it introduce faults, conditionals which would be non-trivial to resolve? Yeah, I checked the NVIC documentation. The major techniques that it used were actually the three that we shown. They do a lot of garbage and no op instructions and then they do opcode replacement and also they would replace the whole sequence of opcode to find identical sequence that do the same function. All right. So those three are the major techniques. What would be possible to get the samples that you used for your research? Right. Yes. Cool, thanks. The question is, were all of these viruses with the toolkit identical in their functionality or is the family that you're detecting NVIC viruses in general or a specific set of variants that do a specific thing in a particular creation? So they were. When we tried, we generated them with the same identical setting because it's an interactive interface. So you just check the option you want. So we actually use the same identical option to generate the 200 different viruses. And so they are supposed to have the same behavior. My question kind of piggybacks on his. When you were looking at the similarities of normal files, did you think about maybe using similar files as your seed to train your Markov model and then see what relation regular viruses appear to see if that was maybe a valid means to detect viruses across the board rather than just one specific virus generator? Yeah, I think that's a good idea actually. We hadn't thought of that. So your idea would be to train on normal and then look for something that appears abnormal. So you just need one model then to test against, right? Seems reasonable. That'd be a good project. I've actually seen a paper. They work this way. They collect the normal files and they train different models to represent normal behavior. So yeah, with any program, they match with normal hidden Markov model and see and do identify. They're successful, but they still need a lot of models to, because the behavior varies, I think they use different models to vary different kinds of behavior. So there's no free lunch here. Yeah, I understand the need for smaller control sets, but your antivirus collection seems kind of small for your claims. You mentioned three, but you also state no commercial products detected it. You tested all commercial products. Your claim was that none do. So the claim was none of them that we tested. None of those that you tested, but it was only three. It seems like an awful small subset. Right, right. Well, those are three we had handy at the time. But they're pretty well regarded commercial testers.  I'd like to know how Kaspersky did on it. Have you tried testing it under, have you tried infecting normal files and testing it under ism model? No, we haven't tried that. Actually, that's a good point. That would certainly be more difficult, right? Cause you would have to zero in on the part of the code that you wanted to test against. So what would kind of up the work somewhat? It looks that you are doing something like fingerprinting the virus, but did you try to fingerprint the different compilers used to generate the normal stuff on the operative system? I mean, there is different C compiler used. I think you can use your technique to detect which is the compiler used for each one of the different files. So this way you're going to say if you use an evil compiler, it's a virus. If you're using something normal, it's good stuff. So as soon as far as the virus program do not use the CCC or any standard compiler, you're going to say, well, this at least is French. Yeah, I think that's a reasonable point, but all these viruses that we looked at were compiled with the same compiler, so we weren't really looking at this situation. I was actually going to say something very similar. Now, how would you describe your hidden mark of models? How did you come up with them? How many states, what size alphabets in those states? What is it that you're detecting? Maybe, in fact, you are detecting the fact that it's just not standard compiled code. Yeah, well, that's a possibility. I mean, she can go into some of the details about what the number of states were and what the states and all that were. Probably should take that offline, I think. Actually, yeah, is this going to be published? Yes. Here's a link to all the details would be in the thesis, right? But there will be a paper as well. My question is, if I made a virus and I applied the NVIC to it, but I made my virus in a non-standard kind of way, that other virus writers and other programs don't usually do, then would your model still work? We haven't tried that. I mean, that's a good question, but I would say probably. I mean, the more non-standard it is, the better, right, for our detection. So sort of the conclusion is that if you want to do this, you'd have to make it look like normal code in some sense, right? Okay, but I mean, it wouldn't look like an NVIC virus in terms of when you compare it to other NVIC viruses. Oh, I see. So yeah, we might have to train specifically on that case. Yeah, that's a good point. My question is, if you took and made a application, but you built, so we actually had real functionality, we took and built a metamorphic virus into it so that the application actually changed it. Well, you'd still have normal similarity of a normal application while still having, when some tiny corner tucked away, maybe spread out to the entire code base, a polymorphic engine. You know, that seems like it would present real problems for, well, I mean, not just for this, but there's a lot of talk about pattern matching going on at DEF CON, and that seems to be the real problem is how do you deal with something like that? Yeah, I think that's a good point too. I mean, you could certainly mask it, right, hide the part that we're grabbing a hold of in a larger mass of code. That would be difficult to deal with, yeah, I would agree. I have two questions. First question is, have you considered updating your presentation with some newer construction kits? I know some of the construction kits you mentioned go back to, it's like what, early 90s, mid 90s, I haven't even seen those for almost like 10 years now. Another thing, the second question is, in your tests when you were doing your research, actually when you infected the files, created these different generation of these construction kits. Have you actually tested each of the executables afterwards if they're still working? You know, I remember looking at them, after a couple generations, the files don't work anymore. So a lot of vendors, there's no point of detecting them anymore because they're already corrupted, damaged files. Well, to the first point, I mean, I'd be interested to know what are the generators that are out there that are considered good today. We just looked through the literature and this is what we came up with as being what looked to be the best. From the second point, I don't know. Maybe not. Maybe not. Okay, thank you. How closely do antivirus and trojan researchers work with industry? How closely do academics like yourself work with Symantec or McAfee? And how do you share information? We don't. I mean, we haven't worked with them at all. She's looking for a job if Symantec's hiring. Generally, AV folks don't share much information because they have this carol cartel where they are like, we'll take your samples, but we won't give you ours for experimentation. So they're not that friendly in that way. Okay, if there's no more questions, thank you.


Exactly. Okay, so remember, bond line structures, these aren't even more condensed than condensed structures, right, even, but they also show the bond angle information, okay, where the condensed structures don't. But there's a lot of implied stuff with these bond line structures. And we went through serotonin last time, so I think if I point to these, this molecule is called cyclohexane, you know, it's a common molecule. If I point to the different points on this molecule, I expect you to tell me what type of atom it is and what type of atoms are attached, okay? So, what type of atom is there? Carbon. Carbon. And how many carbons are attached? Two. Two. How many hydrogens? Two. Two. Two. Right? Why? Because carbon has to have four bonds, right? That's the first rule of organic chemistry for you to learn, okay? If you haven't learned it yet, carbons, if you put five bonds to carbon, automatically work. Okay? Automatically work. Okay? There are cases where it can have three bonds, but you guys aren't going to see them in introductory chemistry. You've got to get into organic chemistry before you'll see them, okay? So, what about there? What atom is that? That's not an atom at all, right? Okay? So that's a bond, okay? What about there? Ladies. What is it? Right? So make sure you guys know that.  Didn't raise your hand, didn't do anything, you know? So make sure you know it if you don't know. How many hydrogens are attached? Two. What about there? How many hydrogens? Two. There? Two. So what's the molecular formula of this molecule? Six H4. Very good. So I think Brandon got it first, right? C6H4. What about this one? Do we figure out some of the stuff about this? What about this? Is that an atom there? It's a carbon atom. How many hydrogens does it have bondage to it? Zero. Zero, right? How many carbons does it have bondage to it? Two. Two, right? How many oxygens? One. Just one, but it's a double bond to oxygen. So it's kind of a trick question because we haven't talked about double bonds and triple bonds yet, okay? But I do want you to know that if you ever see that, right? You know that there's four bonds to that carbon, okay? But if I say how many oxygens, don't say two, okay? This is just one, okay? It's two bonds to oxygen, but only bonded to one. Okay, so what about here? Carbon. Carbon? How many hydrogens? Two. None. None. This one? One. How many hydrogens? Three, right? What about here? We already did that one, right? What about there? One. How many hydrogens? One, right? What about here? How many hydrogens? How many bonds are shown to that carbon already? Two. So how many does it have bondage to it? Two, okay? So what about that carbon? Zero. Zero hydrogens, right? Why did you already see four bonds? What about that carbon? Three. Three, right? Okay, this one? Two hydrogens. Two hydrogens, right? This one again? One. One hydrogens. Which way is it pointing? It's back. It's back. What's the bond angle here? Point five. You guys remember what the bond angle is here? From Vesper theory chapter? It's a good guess, though. I mean, it's a guess. One of four points. There's three, right? And they're in a plane. You know what that's called? That molecular geometry is called? Trigonal planar. How's that? Do you know the angle? No. 127. When you cut a pie in half, or in three pieces, the most you can have is, or that angle is 120. Okay? What about this bond angle here? 120, yeah. 120 as well, I assume you're saying. Not 122, right? No, I said 120 as well. And then this one? What about, so there's a hydrogen there, right? What about the bond angle there? What about this bond angle here? 120. What about this bond angle here? 109, point five. What about this bond angle? All of those bond angles. Okay? Any questions? Any questions? Everybody's just going to look at me funky and not ask a question, huh? Just say no if there's no questions. No, okay, cool.


High Bridge Audio presents Clan Destin by James Elroy, read by William Roberts, prologue. During the dark cold winter of 1951, I worked Wilshire Patrol, played a lot of golf, and sought out the company of lonely women for one night stands. Nostalgia victimizes the unknowing by instilling in them a desire for a simplicity and innocence they can never achieve. The fifties weren't a more innocent time. The dark salience that governed life today were there then, only they were harder to find. That was why I was a cop, and why I chased women. Golf was no more than an island of purity, something I did exceedingly well. I could drive a golf ball three hundred yards. Golf was breathtaking cleanliness and simplicity. My patrol partner was Wacky Walker. He was five years my senior, with the same amount of time in the department. We first bumped into each other in the muster room of Wilshire Station, each of us lugging a golf bag. We both broke into huge grins and knew each other instantly, and completely. With Wacky it was poetry, wonder, and golf. With me it was women, wonder, and golf. Wonder meant the same thing to both of us, the job, the streets, the people, and the mutable ethos of we who had to deal daily with drunks, hopheads, gunsles, weeny waggers, hookers, reefer smokers, burglars, and the unnamed lonely detritus of the human race. We became the closest of friends, and later partners on Daywatch. The Daywatch commander, Lieutenant William Beckworth, was a golf fanatic and hopeless ball-beater. When he heard I shot scratch he had me put on Daywatch in exchange for lessons, it was a fair trade. But Beckworth was unteachable. I could wrap the lieutenant around my little finger. I even had him caddying for me on Saturday mornings when I hustled games at country clubs and municipal courses, so it was easy to get Wacky bumped off Nightwatch and assigned to Days with the two of us as partners, which took us deeper. Herbert Lawton Walker was thirty-two years old, death obsessed, and an alcoholic. He was a genuine hero, a World War II recipient of the Congressional Medal of Honor, which he was awarded for wiping out two machine gun nests full of japs on Saipan. He could have gotten any job he wanted. Insurance companies were deluging him with offers when he went touring for war bonds, but he opted for the Los Angeles Police Department, a blue suit, a gun, and the wonder. Of course, as a juice head, his perception of the wonder was somewhat predicated on the amount of booze he consumed. I was his watchdog, denying the sauce to him in the mornings and regulating his intake until our tour ended and we returned to the station. In the early evenings, before I went out looking for women, Wacky and I would belt a few at his apartment and discuss the wonder, or talk about the war I had avoided and he made his name in. Wacky was convinced that killing the fifteen japs on Saipan had made him a wonder addict and that the key to wonder was in death. I disagreed. We argued. I told him life was good. We agreed. We are the sworn protectors of life, I said. But the key is in death, Freddie, he said. Can't you see that? If you ever have to kill, you'll know. We always came to that stalemate. When that happened, Wacky would lead me to the door, shake my hand warmly, and retire back to his living room to drink and write poetry. Leaving me, Frederick Upton Underhill, twenty-six-year-old outsized crew cut cop on his doorstep, contemplating nightfall and neon and what I could do about it in what I would later know to be the last season of my youth. That season was to become a rite of passage composed of many false starts and erroneous conclusions. I was to blunder love and call it many different things. I was to savor the amenities of life on the make and feel last surges of callow power. I was… All complete. Ready to continue?


Hi everyone, thank you for joining our session today. I will be presenting identifying the gaps in the coverage of web domains and Wikipedia and Wikidata for credibility assessment purposes. My name is Melinda Lu and this is work that I did last summer with my peers at the Wellesley credit lab supervised by my advisor and Mustafa. One of the goals of our research at the Wellesley credit lab is to assess the state of current online information environment and the role that actors like Google and Wikipedia play in shaping this environment through their content and organization of information. The research I'm presenting today continues to explore the interplay between these two platforms. So the presentation is composed of four parts. First, I will briefly introduce schools about this result feature. And then I will discuss our data collection process that starts with auditing search results on Google, then extracting links to Wikipedia pages and domain properties from Wikidata. I will then share the results of our analysis and finally discuss what our research means for future work. As you can see on this example, all results in the Google search result pages or SERPs are now accompanied by a vertical ellipses menu item. If we click on the vertical ellipses, Google reviews a public window about this result. If we examine the content in this window, we notice that the most prominent element is the source referring to the web domain of this result. And typically there are two types of information, either a short description of the site usually lifted from Wikipedia or the time span for which the domain has been indexed in Google search. So why did Google implement this feature? According to the future announcement on February 1, 2021, the about this result pop-up window was designed to help users make more informed decisions about what sites to visit without having to do another Google search on the sites pack plan. But there are also other uses to this feature. With the Wikipedia page linked in each about this result window, we can also access the corresponding Wikidata entries, which contain structured data that help researchers automate the process of identifying whether a site possesses a certain credibility criteria. For example, the award that the organization has received. And furthermore, the characteristics of a domain can be useful in auditing the nature of Google search results. For example, we can tackle questions such as how many domains belong to commercial entities versus nonprofit organizations, and overall what is the level of diversity of domains that appear on Google search results. And with the various meaningful applications we can gain from the content in about this result, we motivate our two research questions. First, we want to examine how often does the Google about this result link to a corresponding Wikipedia entry. And we also want to explore whether the associated Wikidata entry contains structured data that can be mapped to W3C credibility signals. And for some background information, W3C has a committee working on credibility signals, and they have proposed a large number of them. Although the list is quite long, we selected a few examples, most relevant to our research, as you can see on this slide. Now let's turn our attention to the data collection process. When we audit Google search, we try to focus on topics whose results have real-world impacts, such as political elections or gender identity. Then for each topic, we gather possible queries to search by interviewing people or other means. Some examples of the queries that we use are shown. In order to collect the search result pages, we use Solanium and Chrome driver to automate the processes, opening up the Chrome browser, setting the geolocation, searching the queries, and then finally saving the search pages as HTML files. Since the about this result window only appears after clicking on the vertical ellipses and is not contained in the static HTML code, we also use Solanium to automate this process. To be collected about this result content, we extracted the embedded Wikipedia link whenever it was present. From each Wikipedia page, we then extracted the link to the corresponding Wikidata entry by clicking on the Wikidata item using Solanium. And from there, we extracted item ID or QID for the entry. And to provide some background information on Wikidata entries, here's what a typical page looks like. In this example, the entry is on Wellesley College, which has a unique QID Q490205.  In our example, we can see that Wellesley College is an instance of university, private, not-for-profit institution, and women's college, and it's also part of the Seven Sisters. Once we had the Wikidata entry IDs, we then extracted all property and property values using Wikidata's own Sparkle Query service, which takes in a QID and returns a complete list of properties and values for that entry. Now, with all the Wikipedia and Wikidata information on the domains appearing on Google Search, we can now answer our original research questions. Recall that the first question we asked was, how often does the Google about this result link to a corresponding Wikipedia entry? And turns out, Wikipedia's coverage of what domains appears to be low. In particular, for queries related to the 2022 US elections, only 39% of search results had Wikipedia links. And for queries related to gender expression and relationships, only 37% had Wikipedia links. Our second research question was, does a domain's associated Wikidata entry contain structured information that could be mapped to WPC credibility signals? To answer this question, we examined the common properties that appeared on the Wikidata entries we collected. As you can see from the first table, instance of official website and inception are often present on Wikidata entries with our current rates over 80%. They're all useful background information on the domain's organization potentially relating to WPC credibility criteria, such as publication type. Some other useful information, like founded by owned by an awards received that more directly correspond to WPC credibility signals are also present. However, they're more sparsely populated among Wikidata entries, as you can see in the second table on the right. If we look at the top values for the property instance of, we also see that a variety of domain organization types exist. However, some categories are actually more useful than the others. For example, being an instance of website business organization doesn't really tell us much about the nature of the domain. But being an instance of a public educational institution, open access publisher or magazine may potentially indicate something about the domain's credibility. In general, we conclude that for the audited topics, the coverage of domains in Wikipedia is less than 40%, which means that more research should be done to identify what is missing and why. In addition, we discovered that Wikidata has a number of properties. However, it's difficult to map them directly to WPC credibility signals. Future collaboration between the platforms should be needed. And last but not least, for existing properties such as instance of, the values are not always useful. Therefore, identifying more useful property values to include on Wikidata should be considered. Thank you for your attention. I will be happy to answer any questions you might have.


Back then when I was running an event, part of my business, I was so attached to it. I was getting flamed online. A lot of people were saying bad things about it. And for me, I took it so personally that I went into like a semi-depression for six months. Because I took it so personally, I'm like, I spent two years building this project. And how can you throw words out there so lightly without understanding the amount of work that I put in? And then I got super, super sad. And throughout that time, I did a lot of soul searching. And eventually I was able to get myself out of the loop. And the biggest lesson that I was able to learn from that is that I don't attach my self-identity with the business. I can pour my heart, soul and tears into this business, but this business does not represent me. It is only a vehicle for me to be able to bring my family on travels. Just because I sell it doesn't change who I am.


Hello and welcome to the session. In this session we are going to discuss the following question which says that evaluate cos square of tan inverse of x plus x square of cot inverse of y. Let's start the solution. We are given the expression cos square of tan inverse of x plus x square of cot inverse of y which can be written as cos of tan inverse of x the whole square plus sec of cot inverse of y the whole square. First let us solve cos of tan inverse of x let's tan inverse of x be theta which implies that x is equal to tan of theta and we can write tan theta is equal to x in a triangle ADP if theta is the angle between the lines AC and CD then tan of angle theta is given by perpendicular upon base that is AD upon BC which is equal to x by 1. Now if perpendicular AB is given by x and base BC is given by 1 then we can find the value of the hypotenuse AC by using Pythagoras theorem. By Pythagoras theorem we have hypotenuse AC is given by square root of perpendicular AB square plus base BC square which is equal to square root of x square plus 1 square which is given by square root of x square plus 1. So hypotenuse AC is equal to square root of x square plus 1. We know that cos of theta is given by base upon hypotenuse that is BC upon AC which is equal to 1 upon square root of x square plus 1 which implies that theta is given by cos inverse of 1 upon square root of x square plus 1. Now we have tan inverse of x is equal to theta and theta is equal to cos inverse of 1 upon square root of x square plus 1 which implies that tan inverse of x is equal to cos inverse of 1 upon square root of x square plus 1. Therefore the expression cos of can be written as cos of cos inverse of 1 upon square root of x square plus 1 which can be written as 1 upon square root of x square plus 1. Therefore cos of tan inverse of x is equal to 1 upon square root of x square plus 1. Now let us self set cos inverse of y let cos inverse of y be phi which implies that y is equal to cos of phi as we can write cos of phi is equal to y. In a triangle PQR if phi is the angle between the lines PR and RQ then quarter phi is given by base upon perpendicular that is RQ upon PQ which is equal to y by 1. If we are given perpendicular PQS1 and base RQSy then we can find the value of the hypotenuse PR by using Pythagoras theorem. By Pythagoras theorem we have hypotenuse RP is given by square root of perpendicular PQ square plus base RQ square which is equal to square root of 1 square plus y square which is further equal to square root of 1 plus y square. Therefore the value of the hypotenuse RP is equal to square root of 1 plus y square. Now we know that second side is given by hypotenuse type base that is RP by RQ which is equal to square root of 1 plus y square by y which implies that phi is given by second whole source square root of 1 plus y square upon y. Here we have quarter inverse of y is equal to phi and phi is equal to second whole source square root of 1 plus y square by y which implies that quarter inverse of y is equal to second whole source square root of 1 plus y square by y. Therefore the expression second whole source square root of 1 plus y square by y which can be written as square root of 1 plus y square by y. Therefore third whole source quarter inverse of y is equal to square root of 1 plus y square upon y. Now we have quarter source tan inverse of x is equal to 1 upon square root of x square plus 1 and second whole source quarter inverse of y is equal to square root of 1 plus y square whole upon y. Now substituting these two values in the given expression we get quarter source tan inverse of x the whole square plus third whole source quarter inverse of y the whole square is equal to 1 upon square root of x square plus 1 the whole square plus square root of 1 plus y square by y the whole square which is equal to 1 upon x square plus 1 plus 1 plus y square upon y square. On taking the LCM we get y square plus 1 plus y square into x square plus 1 whole upon y square into x square plus 1. So we get quarter tan inverse of x the whole square plus third whole source quarter inverse of y the whole square is equal to y square plus 1 plus y square into x square plus 1 whole upon y square into x square plus 1. Now we can also write quarter square root tan inverse of x plus 6 square root quarter inverse of y is equal to y square plus 1 plus y square into x square plus 1 whole upon y square into x square plus 1 which will be required answer this completes our session hope you enjoyed this session.


Our first winner of the $100,000 sponsorship is announced in this episode, was it you? And we sent you $1.9 million freedom family for all your hard work in May and June. And we will go over your 545 comments that you left in yesterday's episode where we said, yes, the $100,000 sponsorship is back. This is round two where we set aside $100,000 in cash that we will give to you, the deserving members of the freedom family who show us how you help the family grow. To learn all the rules of how you can win, click that I to see yesterday's video where I explain everything. In the spirit of collaboration, what are we watching? Hello, I'm Sataro 2013, a freedom partner and today you are watching The George Show. George Bus. Thank you, Sataro 2013 for that George Bus. Collaboration is the fastest way to grow on YouTube. Click that I to learn more about collaboration. And if you want to submit a video introducing The George Show, click the submit your best video link down there in the description below. And to see Sataro's channel behind me, his link is also down there in the description below. The $1.9 million we sent to you yesterday was for your May and June earnings. Why, two months? Because some of you opted in for early payment, 30 day early payment. We actually sent almost $800,000 in early payments, your June earnings. Click that I to learn how to enable 30 day early payments in case you want to starting next month. We also sent over $500,000 by wire instead of PayPal. You can all now choose wire as a payment direct deposit into your bank account in case you don't want to receive money by PayPal in your shiny freedom dashboard. I am so excited. The $100,000 sponsorship is back for round two. Keep telling me down there in the comments below how you think we can make it even better. Right now we are offering $100 cash prizes for people who help freedom grow. What other kind of prizes should we offer? What other kind of events should we do? Share your thoughts. Now on to spotlighting some of your deserving comments. Starting with Ashley Silver dust. Who wrote, exciting the comment scrolling background was pretty epic to be honest. So many comments. I think we should do this every episode. What do you think? Tell me down there in the comments below. Notice how Ashley is using time stamps so we know exactly what part of the video her comment relates to. I recommend you do the same. Master Audit also uses time stamps in his comments and yesterday he went wild. Well done. And Master Audit called out my mistake at 1 minute and 56 seconds where I said freedom fricks and I meant to say frosted fricks. Sorry about that. Cosplay nominated someone to win the $100,000 sponsorship frosted fricks since he does a lot to help on both MGN and his own channel with channel reviews. He is awesome. Do you want to nominate anyone to win the cash prize? Tell me down there in the comments below and why. That gives me an idea. If you are one of the winners of the $100,000 sponsorship you get to pick another winner. If you make a video explaining why you chose that person and send that video to me as well as put it on your own channel so we can all see who won and why. Do you think that's a good idea? Tell me down there in the comments below. You can submit those videos using the same submit your videos link down there in the description below and I want that to be a public video on your channel because we want this to be a public announcement. Let's celebrate the people who help the freedom family grow. So who is today's winner? I chose the person who I've seen give amazing help for years for freedom. He's never complained. He's always been active on discord in the forums in these comments and he is this person right here. Zoustfrags, you are the first person to submit a welcome to the george show video. Thank you. You are always answering people's questions like estetian freak and you wrote you don't need to promote other channels just leaving a tip or two for channels is good as well. When estetian said kind of unfair small channels like me cannot help other channels grow. So Zoustfrags is not a large channel. He has 479 subscribers. Here is Zoustfrags channel behind me. I encourage you to check him out by clicking his link down there in the description below and you can see on his channel he even uploaded a video talking about the sponsorship already. I did not ask him to do that. He did it on his own initiative. For these reasons I just sent you $100 cash Zoustfrags to the PayPal email in your shiny freedom dashboard. Tell us when you get it and how it makes you feel to be the first winner of our hundred thousand dollar sponsorship. I would love to see a video that I can feature on the george show. Submit it using the submit your best video link down there in the description below. Congratulations. You deserve it. Now you can nominate someone else to win. Put that person in your video and he will also receive a hundred dollar cash prize. Let's read some more comments. Noble Addiction wrote I try my best to help out the freedom family as well as non-freedom members by making short helpful step-by-step tutorials for things like YouTube, Xbox, PlayStation, Nintendo and other major social media services. That's awesome Noble Addiction but send us links to those videos in a playlist down there in the comments below. Let's go see if we can find them on your YouTube channel. Scrolling down we see how to make your own Xbox profile pic as your default video. Why don't you have a channel trailer that introduces your channel to tell people what to expect and encourage them to subscribe. That should be your default playing video. Continuing on I see you have a tips series. An awesome first playlist that tells me exactly what your channel is about helping people with good tips. New Nintendo Switch news channels. How to change your custom gamer tag. This is all pretty helpful for people who need this kind of information. To check out Noble Addiction's channel click the link down there in the description below. And these glasses keep fogging up. Sorry about that guys. Frenzy Gaming was one of the winners of the previous hundred thousand dollar sponsorship round one. He wrote Hi George I was a winner on day 22 of the original hundred thousand dollar sponsorship and I couldn't thank you enough for that. I'm not here to try to win again. Why not? This is round two. You can win again. Everyone has a chance. I just want to say that it helps a lot to join a community of streamers and YouTubers that are all about supporting each other and helping each other grow. That is the spirit of freedom. That's what we do. We all help each other grow together as a family. The freedom family. Frenzy Gaming joined a community to help streamers grow. He is now the YouTube manager of a small streaming community. He doesn't want to give the name because he doesn't want to do sneaky self promotion. Please tell us. Send a link. We'd love to see this community because he helped once or he witnessed one streamer grow from 400 followers to over 750 and he just reached his 10,000 view mark tonight. Congratulations. For those who don't know YouTube requires 10,000 lifetime views on your channel before you can join an MCN or monetize your videos with ads. Click that I to learn more about this 10,000 threshold with all that said and done. I leave you with these final words. Check your LinkedIn messages, George.  I've just had too much to do but I'll get to them, I promise. Kaiser Hertz nominated Helix Brightstar to win because he was the only freedom community staff member that he actually connected with that helped him with issues on and off YouTube. He treats everyone fairly and without bias regardless of opinion. That's a wonderful compliment for Helix Brightstar. Thank you for sharing Kaiser Hertzen. And if I don't pick Helix Brightstar as a winner I'm apparently gonna get pinched so hard. How many of you agree Helix deserves to be one of the winners of the $100,000 sponsorship? Tell me down there in the comments below your experience with Helix Brightstar. I would love to know. The sensible guy wrote thank you George for featuring my comment at 419. I hope you like the idea and I look forward to seeing this idea become a reality fingers crossed. The comment he is referring to is to bring sponsorships back and to create a fame bit platform within the Freedom Dashboard where you can sign up for different sponsorship deals by creating your profile and companies come to ask you to do a sponsorship for their product. Thank you for telling us this is what you want. How many of you also want this type of branded entertainment feature in the Freedom Dashboard? Tell me down there in the comments below. Eklonic wrote if I could win that would help me a lot. I have a dream to reach 100,000 subscribers and get my silver play button. But with my hardware it's impossible and I can't save any money for my hardware because I live with my grandma and I don't have parents. Our family is big. I have three sisters and one brother so I can't afford better hardware. I know many of you are in the same position where you don't have the right tools to grow. But remember YouTube is not about high production value. It's about personality, entertainment, information. You can just make logs even with your cell phone. Some people become very popular just from that. So don't let a lack of hardware discourage you continue making videos to find the right audience that loves you for who you are not for your high-end hardware. Hope this helps. And who am I? I am Papa George, George's son Pie and many other names you the beautiful Freedom Family has given me. I'm still a little sick. I apologize for that. I am George CEO and founder of Freedom for Freedom and the Freedom Family. And all the amazing winners we are going to crown with the $100,000 sponsorship pool of money that we will distribute to you in interesting and creative ways not just cash prizes. Keep sending me your suggestions on how we can do that in the comments below. I am George and you've been watching. Click that I to partner with Freedom and join the Freedom Family so we can all grow together. You get many perks like position music. You also get Epidemic Sound, a lot of other access to royalty free videos, sponsorships and many things to help you grow. Just click the links down there in the description below to get involved in our community, our forums, our Discord chat servers, meet our graphics team, meet our community team, all of that on Discord and the forums. What are you waiting for? Get started. And we will grow together as a family because this is the Freedom Family. You are part of it. We are all part of it and we're all growing together. To get more George, click that big F that will subscribe you to Freedom Central, home of the George Show. And PewDiePie gave one of you, Freedom Family, a big shout out. Click that video to see the shout out and to see our new 3D sets for you. And click that video to see what YouTube recommends you watch next.