Hi everybody. Good morning, good afternoon, good evening, wherever in the world you are and welcome to OpenJS World. My name is Justin Dolly and I'm the Chief Security Officer at Sauce Labs and I'll be joined momentarily by Christian Broman. We're here, it's a pleasure to be here and exciting to be sharing with you some of the innovations that we're working on to be able to give you and our customers more insights and a deeper understanding of your software.  Well, I'm standing in front of you because I'm responsible for the security of Sauce Labs, which includes not only our company and our users and our networks, but also the platform and services that our customers use 24 hours a day. I've been leading security at public and private companies for more than 20 years, but I have something to admit to you and that is I am a tester. I've always regarded myself as a tester. If you think about it, the act of testing is a pretty big important part of any security program because it provides you with results and those results, if they are treated in the right ways, can provide deep insights to be able to allow you to make risk decisions. These signals are critically important and to date I think have been relatively speaking underserved. We aim to change that situation by providing you with security signals alongside all of the rest of the signals that we provide as you go through your current testing routines, so the quality of your software can be as good as it can possibly be. After all, security issues are just simply an outcome due to a lack of quality. It just so happens that these days the security outcome is one of the graver outcomes given our current technology landscape. And now I'd like to pass it over to Christian Broman for more information on why we do these things and to give you a little bit more detail. Christian. Thanks, Justin, and hello everyone. As a DevOps engineer, we are living by three very important mantras which are increased automation within the software development lifecycle, and therefore allow faster delivery of features and software, which is essentially shipped with high quality and therefore contains as less parts as possible. And we see that more and more responsibilities are moved towards the developer. We see that in testing we see that in DevOps. Instead of having one team that builds a feature, one that tests it and one that deploys them, the developers have moved towards a unique delivery team that develops, tests and deploys at once, and therefore takes ownership of the complete pipeline. The advantage, of course, less back and forth between teams, which means less cross-department communication, which results in faster time to delivery. It not only saves time, but also a lot of resources. Now, we see the same desire now happen in the security space, rather than having a dedicated security team auditing software, faithful at the end of the software development lifecycle, we want the majority of that work done in an automated fashion early in the development. It applies to the same shift left principle as we see in other disciplines. Now, as a developer, the question is when and how do I have to test the security of my application? As with all types of tests within security, there are various approaches for different areas within the software development lifecycle, starting from threat modeling, which is a process to identify potential threats such as structural vulnerabilities or the absence of appropriate safeguards way before the development even starts. Then we come into the SAS area, which stands for Static Application Security Testing, which is an approach similar to what we know as code linting, where you scan your code to try to discover problems, in this case, security problems. Like all linting tools, this offers fast feedback and allows to detect a lot of issues. But the value of those signals is often very low and you get a lot of false positives. Lastly, there's security testing in the infrastructure of code and as well as security monitoring and attack detection, which come with their own complexity and challenges. So, from all these stages, which one would you think is part of the developer development lifecycle, but has not really much explored when it comes to tooling? That's right. It's the dust approach. There are plenty of well-working tools in the static security analysis space from starting looking at SAS provider like SNCC or integrated tools in GitHub, for instance, the dependency management tool they have there, or SAML, which is a code analysis platform that was acquired by GitHub in 2019. This, however, is just one way of testing security. Security in general, there's no civil bullet to it. There will be never one single tool that allows you, that can guarantee you the security of your whole application. This is why there are so many stages. Dust in this area is the security end-to-end approach that, as opposed to SAS, provides much more valuable signals for us as developers to act on. Now, why is this interesting for us? We at SAS have been thinking about this over the last couple of months, and we thought that we see in this space that security testing is not really accessible to everyone. At SAS, we are running over 3 million tests every day, which is a very large number. And this gives us a huge advantage compared to any other competitors in the market because we know your application from the in and out. Based on your functional tests, we know where you log in and which credential you may use. We know the process of your checkout flow, which steps and pages are involved, and most importantly, what data is being transmitted from one form to another. All these information help us to properly analyze the application on the test to provide you insightful and meaningful signals. That said, this solution cannot only be applied on SAS Labs. It can be deployed on all running tests across all vendors or even locally. Because one of the big problems we have here in the security space in general in the security tooling space is that people have to implement scanners to understand your application. Those scanners need to be able to go through a login to access a secure area. Running functional tests already solves that problem for you by accessing that through functional commands through WebDriver or newer tools like Cypress. So with your functional tests, you can already prepare everything that you need to analyze the security of your application thoroughly. The question now becomes, what kind of tool would be ideal to build on top of an existing testing infrastructure? Sure, we could implement a new preparatory solution in our company. However, that would mean that we would need to reinvent the wheel. We at Source Steps found that a perfect fit that is also open source and can be used on top of your functional test. This tool is called ORA SAP. ORA SAP is not only open source and part of the OAS Foundation. It is trusted tools by security experts with a long development history. I think its first release was somewhere around 2010. And it has been always developed in the open since then. And there are also some very interesting similarity to WebDriver, which is a protocol that is driving functional tests from the beginning with. Which is, it's built with an API first in mind. This means that you can, that all interactions with the application, with the tool is abstracted into API endpoints. That means that you can deploy that tool similar to deploy a driver anywhere in the cloud and operate to it through API plans. The way SAP works is that you can use it for, for instance, calling your page to analyze what the browser is doing, to analyze what the browser is requesting and receiving during the exploration. There are two types of scanners. One is a headless approach where SAP is doing requests on your behalf and scans the HTML response and parses links and see what's, what's page just next to call. And the second one is an HX scanner, which is designed to operate on top of a browser to allow you to inspect also a single page application, which are not rendered when, not necessarily rendered when you request them from the browser. Another way to explore an application is also by using SAP as a proxy, which is a really fantastic feature here. By automating it through WebDriver using Selenium WebDriver or other tools, you can use them to write automated tests, automated functional tests to explore your application in an automated fashion. So instead of, you know, your browser making requests to the internet, it tunnels all the requests through the SAP proxy and can, and those kind of information can be later used for scanning purposes. Now, if we move SAP into the cloud and connect our functional tests with them, this can be, this can become a very powerful setup. Your functional tests can pull your application 100 times more efficient than every scanning tool would do. So that scanning vulnerabilities with SAP in the cloud can be 10 times faster because you can run them in parallel rather than executing that or scanning manually. So that eventually testing functionality and security becomes almost one single workflow. So let me show you how this workflow might look like in your company. Let me introduce you to Christine Watson. She's leading the QA team in her company. She and her team have been running a lot of functional tests in their Selenium grid that they have deployed in their CI system. And they are now getting the requirement to ensure that they also add security tests to make sure that security bugs are caught early in development. Her tests are written in Node.js. And now she wonders how, what she needs to do to fulfill these new requirements. She finds out about the capabilities that OASlap provides them. So she starts deploying OASlap in the cloud or locally and starts tunnel all the browser traffic through that application. To do that, you need to start SAP as a demon to wherever your machine runs your test. For instance, your CI server. Then you apply the appropriate proxy capabilities in your web driver setup so that the browser is started in a way that it automatically proxies the request through OASlap. The nice thing about this is that you can enable this for all browser tests. Applying proxy settings is part of the web data protocol and therefore works across all the browsers. So besides these small settings, Christine can keep running the test the same way as she did before. Now while her tests are running now and Christine is getting a coffee, which gives me some time to explain the options that you have after your functional tests are done. So many of you will have different kind of security requirements. I had conversation with folks that said, I would never fail my pipeline because of security reasons. Others said that there would be actually indeed interested doing that because they are running and testing a critical security related component in their infrastructure. What you see here that it comes ultimately down to what your team's requirements are and every team's requirements can be different. Adding security tests to your pipeline will have some implications to it. For instance, scanning for vulnerabilities will definitely add some additional time and therefore might be slowed down your pipeline overall. And with that, slow down the time that you can test and deploy new features to production. However, when you run everything in the cloud, you can start analyzing and scanning for vulnerabilities while your functional tests are still running. So our goal should be to always opt in for security tests because we can run them in an efficient manner in parallel while some other checks are still being executed. And security in itself is also very difficult to describe. Security does mean different things to different people, right? Certain vulnerabilities that are critical for one person might not be for the other. And there will be, again, no service that will give you a feature or the assurance that the whole application that you provide or that you want to deploy is secure. The fact here is that by detecting and mitigating common vulnerabilities early in the software development cycle, you give your security engineer more times to focus on the difficult aspects of your overall architecture. In case of Christine's teams, their policy is to, you know, to check after every, before every deployment, how many security bugs exist in their recent tech version of their system. So if you store these up sessions that you haven't been opened during your functional tests and you put them into a cloud bucket, you can always download them at all times and access them through ZAP proxy to use them for scanning security vulnerabilities. So every functional test therefore can now store not only general logs like Selenium logs or WebDriver logs. It will now also store ZAP sessions that include a lot of data on requests and responses that the browser has been made, was making during the functional test. And Christine can trigger these scanning, these vulnerability scans completely automatically by using, by downloading the ZAP session from the bucket and using the ZAP client to trigger these security scans. ZAP as a tool comes with a lot of utilities equipped to give you a nice overview about all the vulnerabilities that are exist in your application. So if you manage this, of course, within a cloud platform, you can make this much more simpler by just having a click on a button or providing a CLI client that does everything for you. Now for the last build, Christine found 10 new vulnerabilities. Some of them are related to things like missing security headers or a secret key in one of the JavaScript files or something really simple and common. These things are really easy to fix. Christine knows her application and she knows how to fix, how to apply patches to fix those issues. However, with, she needs help with one critical bug that she has seen, that she hasn't seen before and nor anyone in the team. With the help of a geo integration that she is using, she is sending that security vulnerability to one of her security engineers working at her company. That security engineer can now download the same Zap session and can use the Zap GUI to inspect what happened during the functional test. So he downloads the session, drags and drops it into the OSAP UI and introspect all the requests and responses, finds the vulnerability and message Christine's back with all the information she needs to mitigate that problem in the future. So with that, Christine and her deaf team are happy because they have shipped the most secure application and the security engineer is happy because he made the internet a little bit safer. Now I said in the beginning that there's no silver bullet for security. Well, the closest thing that gets to it is awareness within developer and QA teams. So to sum up this little workflow example, with OSAP and functional testing using web driver and Selenium, you will be able to analyze the vulnerabilities of your application by using your functional test. With OSAP under the hood, you can create some deep analysis on the most common vulnerabilities that exist. Things like you might have heard about the OSAP 10, which is an updated list about the most common threats of today's modern application that you find on the internet. In addition to that, you can store that data and keep an historic backlog of the information or security bugs that have occurred when you have been developing on that application and learn from the mistakes that you have done in the past to mitigate this happening in the future. Now, while this talk really doesn't provide you any step-by-step guide to install or deploy this in your application, I hope it inspires you to that with just open source and open standards, you can deploy such a system in your organization. We feel like with this approach, you can make security really accessible to everyone and test it early in the development lifecycle. And again, every organization does this differently. We at Source, we have a lot of ideas how testing security and testing the security of your application can look like. And we are interested to talk. So reach out to us. And with that, I want to give it back to Justin for the closing remarks. Thank you so much, Christian. It's very, very informative and very interesting. I think Christian's point is a great one. I can tell you having been in security for such a long time that the silver bullet that Christian's referring to about just the simple awareness and a little bit of education inside of the areas of our companies where the development occurs and where the ideas are spawning, from where the ideas are spawning, is really the silver bullet, is that we're actually aware of some of the choices that we're making and some of the security implications to those choices. I think with everything else, one of the promises of open source in particular and the open source standards is that we can all adopt them and adopt them quickly and they're flexible enough to be able to support our businesses. And any other initiatives we have non-commercial, maybe non-commercial in nature, but this is truly one of those opportunities where a rising tide lifts all boats. That, you know, by leveraging these standards and these initiatives that we can increase the general security across the board of applications and websites and platforms and so on that are attached to the internet, protecting all users of those platforms and those sites. So it truly is incredibly valuable and I would encourage you to investigate these things and see if they fit for you and your organization. And with that, I'd like to thank you all very, very much for your attention. Thanks to Christian for the expertise and the details and we hope you continue to enjoy the rest of the content at this wonderful conference at OpenJS World. Thank you. Thank you all. See you.


Welcome back, in the last segment we discussed a scheme for dispatching taxis, that is matching taxis to customers. Now in that scheme, that scheme was, first of all that scheme was perfectly fine in the sense that it did the job, but it seems that the scheme can be improved. So let us see why. So here what we did was that after a customer came in, we assigned the waiting drivers but then after that we sort of moved all waiting drivers one step forward. This does not seem, this seems like a little bit of an unnecessary thing. So you will see that we can essentially eliminate this movement. So there might be maybe say 400 drivers waiting, so then we are doing these 400 moves, so why should we do that? So you will see now that we can do without actually moving so many drivers forward. So here is the new idea. So the previous program seems to be copying data a lot. So we are going to avoid copying. So for this we are going to look at what might happen without computers. So this is actually something that you should think about anyway. So if you are trying to solve a problem and practically anything that you do in this course, you are going to do something which you probably would do manually, you are not going to do something which is terribly different from what you do manually. So therefore figuring out look what would I do manually. So in this case would I do the equivalent of that copying manually that is something that you should ask. So what would you do manually? So maybe you might write the names on a blackboard. So what will that look like? So this is my blackboard and then I start writing the names of the arriving drivers from the top. So I wrote that and maybe at some point a customer comes. So what will I do? I will take the first driver over here and give it to the customer, give that driver to the customer and maybe I will erase this name on the blackboard. So I cannot erase when I write it in pen and paper but if it is actually a blackboard I can wipe it off. So I am not going to erase this and push everyone up. I am just going to erase it and say that look now really the front of the cube is over here. If one more customer comes I will erase a little bit more and I will keep going in this manner. But suppose a new driver comes so then I will put down the driver information over here. I will put down the driver information over here, here, here, here all the way till this point. What happens if I come down all the way and a new driver comes? Well in that case I may start from the top. The new driver information would be put at the top. So now this is an interesting way in which the information is given over here. The information starts from over here, goes down and then continues up from the top. So you can think of this as some kind of a circular cube. So it sort of starts over here and comes back over here. So think of that blackboard as sort of a rolling as a circular board. So because it is a circular board we need to worry a little about where is the first person. So we know that the first person is over here. If that person, if this driver is given to a customer then the first person becomes next. But we are really not shifting up the driver names. So that is exactly what we are going to do in our solution. So arriving driver ID is written top to bottom. When board bottom reached we begin from top if the drivers have left. Of course if the drivers have not left then we cannot do anything. We have to tell the new driver that look my cube is full I do not have space to write down your number and so please wait, you please try a little bit later. But usually this will not happen because we will allocate a reasonably large amount of space anyway. So we are going to think of this driver ID as a circular array. What that means is that the next position after driver ID n minus 1. So this is index 0, this is index n minus 1. So after this position the next position is this. So after this position the next position is this. After this the next position is this and so on. But after this normally you might be tempted to say that there is no next position. But no in this case we are going to say that the next position is this. So it is kind of a circular array. So that is basically the idea that we are going to implement. And again the idea is simple, it is not completely simple, it is more complicated than the previous idea and therefore we should make decisions sensibly at the beginning and then stick to those decisions. So that is what we are going to do next. So we are going to say whatever invariants are. So as before we are going to have n waiting denote the number of waiting drivers. So that variable we are still going to have. We have to keep track of how many drivers are waiting and we are also going to have this invariant that n waiting is going to lie between 0 and n. But now we are going to have a new variable called front. The front is the position of the earliest assigned driver, earliest unassigned driver. So if you look at this picture, so let us say these 2 drivers which we had assigned, so position 1, 0 and position 1 these drivers have gone away because they have been given customers. So then this position 2 is the first position where there is an unassigned driver. So really front will be equal to 2 at this time in our entire execution. So front is the index of the earliest unassigned driver. So it is 0 initially but as drivers leave this front increases. But we want front also to be between 0 and n. Well, this time we want front not to equal n because front is not the number, front is an index. So front cannot be, it cannot be n, so it can be at most n minus 1. So front is an index, so it can start at, it can be 0 all the way till n minus 1. And the first valid driver id is at driver id front. So that is what we said. Now this raises an interesting question, where is the next driver id? Should we say that it is at front plus 1? The next driver id, shall we say that it is the element driver id of front plus 1? Well, there is a bit of a problem. If front is n minus 1 as it can well be, because after all we have said that front can be n minus 1. So if it is indeed n minus 1 then front plus 1 of n which is what we are asking over here is n and that is not a valid index. So if front is n minus 1 what would we want this to be, we would want this to be 0. As we said if this is front the next one from here is 0. So therefore we are going to say that the next waiting driver is at front plus 1 mod n. So this plus 1 mod n is like going forward on a circular array. So if we keep doing that, so maybe I should explain this. If front is n minus 1 then front plus 1 is n but n mod n is 0 which is exactly what we wanted. So in this by the same logic the last waiting driver id is not at front plus n waiting minus 1 which is what would be the case if this were not a circular array. But because this is a circular array we have to take a mod n. So these are the things in green are the propositions that we have to care about these are the invariance. So we have to make sure that these statements are always going to hold. So here is a pictorial view of the same thing. What we have said over here is that this is the initial position. So this part is the entire array and this entire array is right now unused. So what I have shown over here are indices 0 all the way till n minus 1. What happens after some drivers have come in and some customers have also come in? Well if so many drivers came in and if so many customers left then this initial position becomes unused. This last position parts are unused and this middle part is occupied by waiting drivers by the id's of the waiting drivers. But now if we continue in this manner what might happen is that more and more drivers will come over here. They may also occupy the top position and maybe more and more drivers will leave. So at some point in execution the situation might resemble this that there are some occupied positions by drivers some occupied position by drivers and the middle portion is unused but that is okay. This is the front so this is the first driver position or the position of the earliest driver waiting in the system and this is the next one this is the next one and so on this is the one after that and this is the last driver that came in and after that there is this empty space. This is just to clarify what these variables represent and just give you a picture of all of that. So how do we process a driver arrival? So first of all if n waiting equals n then that means that our entire array is full of drivers. So we do not have space to put in this new driver so in which case we are going to say well our queue is full please try later. But if it is not n if we do have space so then we get the driver details. So the driver might type in some ID or maybe some phone number whatever convention we have and we are going to place it in the position where we said the next driver ID should be put in and that is front plus n waiting but mod m and whatever we read in is going to be put in in that position and at this point we will increment n waiting because we got a new driver and so there is one additional driver waiting that is it. So front plus n waiting mod n is the index of the empty position after the n end of the queue so this is just a reminder as to why we use this over here we had discussed this earlier that this is the position where the next driver should go. How do we process a customer well if this is kind of the complement of it if complement of driver processing now this time we are going to check if there are any waiting drivers at all if there are no waiting drivers then the customer cannot be served. So we have to send the message tell the customer look try later well of course this program this will be read the output of the program will be read by the controller who is typing the C's and D's and the details of the drivers so the controller will read this message try later and presumably the controller will tell the customer otherwise what happens well otherwise we are going to print a message saying look here is the driver for you and which means which driver ID will we take well according to our invariance the driver ID in the index front is the earliest waiting driver so that driver ID is going to be sent or assigned to that customer and we have to say that this position is no longer containing useful data because we assign that the driver so we have to increment front but remember now this is a circular queue so we increment it but if that increment has gone to n we want it to become 0 and so therefore we take mod n and we decrement n-waiting that is it we are not going to shift up the waiting the waiting drivers data in our queue instead we are going to shift down what is the index of the first and assigned driver that is it alright so this is another solution so a different idea and it is a better idea because it copies it doesn't do really any copying but we have had to have an extra extra variable called front so we had to manipulate one extra variable but we saved a lot of copying so now as an exercise I would like you to go through the programs and check that the invariance are indeed true after each customer or driver arrival and of course the code is there in the book and I would like you to run the code and test for yourself whether the code works correctly so that concludes this segment in the next segment I am going to talk about something called the disk intersection problem but before that we will take a short break


George Bruno with the 21 report we are at the 21 summit actually the end the post-game show of the 21 summit We're in Orlando, Florida The year is 2020 and I'm having a conversation With the founder and president of the 21 summit and the Manusphere Anthony dream Johnson welcome to the show. Yes, sir. Good to bear All right. We have four days behind us a lot of work went into this Yeah, Socrates said you dared to pull this off. Yep, I can say that you did you did it Thank you hold it off. I feel like it too. I feel to my bones, man Usually I'm exhausted after an event and I feel physically a little you know tired from doing, you know Sleeping so little sleep and work my ass off like everybody else But I feel like a million bucks or a billion bucks man I feel awesome and I feel you have my bones and feel my spirit and I feel just super awesome about it I know how much we nailed it everyone I just I mentioned this yesterday to one of my GM James that everyone performed at a super high level in terms of even the attendees I guess but I'm talking about speakers staff volunteers, especially the guys that we knew coming back with the alumni You know staff and volunteers killed it killed it killed it killed it and that happened is kind of organically Maybe it was a good momentum. Maybe it was a good year for it. The year being so sucky Actually, I think maybe is what helped inspire that people wanted a good experience They wanted to have a good time. They wanted to go to Florida, you know The governor's been good here a lot of the Kobe stuff had they want to go to 21 summit They want to learn they want to be better men. They want to have something positive and such a negative, you know toxic kind of year so 21 summit I think provided that and It's been an awesome year for me. I mean the year has sucked in a lot of ways in terms You know the government stuff and the Kobe stuff and all that but Yeah, it's just super awesome. Like I am amazed that how well this went You know We've only done before 21 summit one convention at a time either the Patriarch or the under 21 or of course the main event 21 convention We did to we did a brand new event We did the second Patriarch and we did the main event for the 19th time or whatever in 19th time since 2007. So this was just This is unreal. I mean this was truly epic and everyone felt that I feel that you feel that we all feel that the new speakers even feel That the new guys I mean coach Greg and like Bulldog for example, and they must be just like holy shit Yeah, they think this is the normal for them now Yeah, and this is this was we've done amazing events for you know forever and especially since 2017 when you started the 10-year anniversary But yeah, holy shit, man. This was like epic truly epic. There was a lot of moving parts here.  You had returning speakers and New attendees and new speakers wait a new Gender of attendees a new gender. I mean literally yeah literally Because there's only two genders is that correct? Yeah, I'm a big fan of the two-gender system It's worked very well for a couple hundred thousand years Yeah, and yeah, there are only two genders anything else is pseudoscientific garbage. We're not gonna have another Convention with a third gender not planning on it. I'll consider it You know okay if one pops out of the womb here for our species. Yeah, but until then well, I think you know I've said before Sort of seriously sort of jokingly jokingly that women Women see three genders. They see men. They see women and they see betas by the mouth Jesse Peterson style but now there's just two genders to gender system It's a great system and we should keep that intact indefinitely. Yeah The 22 convention make women great again started out super controversial. Mm-hmm. I didn't Let's pause on that because remember in late 2018. We actually announced that event You did the website went up and it was you did whoever was on at you and AJ I think it was you and AJ on it or something. It was you and AJ on the headliners And it didn't really do much but when we fully announced it. Yeah, I went nuts. It was amazing or amazing as we like to say yeah It was a risk wasn't it? Oh Yeah, it was a huge risk. I mean we got serious heat some of her speakers had really serious legitimate legal issues fate coming into the event just by Associating with the event as a speaker not saying anything and not and not otherwise endorsing it You know the content is the contents marketing, you know, they're not responsible for that They didn't do that you didn't go on the website and type the marketing and stuff But no, there was serious issues death threats threats of violence bombs mass shootings we actually had to change our Logistics for the event a little bit in terms of usually a people We don't tell anyone where the venue is at for the main 21 convention ever Just to keep it private safety security big fans privacy But for this event or typically in the past we email the attendees, you know The venue once they buy a ticket buy ticket of course tell you where the venue is at the hotel room all that This event for both the men the fathers and the women all three events They didn't even get the venue location until 72 hours before the event started. Yes, because I was just so You know concerned based on the feedback that was so intense and hostile that they were gonna try to protest Especially threatened to we found reddit posts Facebook groups I heard Planned Parenthood in Orlando was coming after us trying to organize to cancel the event cancel culture And I've been preparing for that for years because I knew this event would most likely to get super controversial and I won we won we win and these losers lost because we had the event and now we're out of here with the footage It's like a like a like a bank. Hi, so they run get the footage and run Yeah, but anyway, yeah, they did actually in a call in the hotel and trying to get us canceled even at 72 hour mark 40 about 48 hours out they're blowing the phones up and Yeah, they stay followed through on the word the venue and I'm really proud of them, too I'm really happy I chose this place and that we're here and we got it done. Yeah at any point. Were you worried about anything? Hmm Not really like we had a really good safety Coordinator and security team things like that. We heard law enforcement of as usual to protect the events The part of Orlando that we do the events in there's not really that a protest anyway You stay away from the kind of the protest areas and Orlando is pretty safe anyway with that stuff Not to say but pretty mild in Florida than the place really had protests this year for example We're like major ones were like Tampa and Miami, which were not where Orlando So I felt pretty good. I myself a carry fire. I'm full-time at the events I'm not security person. I'm just an individual a man and adult and a Floridian But I take safety seriously for myself and I don't know what anyone else to get hurt So I take personal responsibility for that too isn't to an extent as an event or manager and organizer No, I felt pretty good everything was super smooth and there's always a risk though I mean there's always a small chance at minimum that people want to protest cause problems You know through a Molotov cocktails peaceful burning looting as we call it. Yeah, but now we had none of that the feminists Either they got distracted by Trump or covert or whatever.  Yeah, they still hate us But they lost and we won so much winning you're gonna get tired of it. I know it What was the highlight of the 21 summit people have to realize 21 summit was three different conventions under one roof at a convention center this place hosts conferences and conventions all the time That's what they do. We had the 21 convention we had the patriarch convention and then we had the 22 convention make women great again There was a lot going on here. Yeah, what was the highlight for you? Man, I know it's hard to come up with one thing. We're one day after two here in the post game show I still get a process catch up and sleep. Yeah, we all are at this point Yeah, you know during the event sometimes as the manager anywhere as the organizer the event architect I've had like I've gone on zero hours of sleep, but this year I was doing about two to three at night Sometimes four so you get sleep deprived. It's so tough to process all the events that happened Man, I'd say actually one of the highlights was the interview Just did with the guzzies I call them like the Brady's the Brady bunch the guzzie bunch Yeah, that was a really impressive interview Tanner and his wife You as a you know interviewer did fantastic and they were just you know Bracaeli in particular She's one of the first woman. We've ever interviewed right after Jennifer Molesky. That was impressive to see that kind of content Speaking particularly about patriarchy and being humble on these things right alongside her husband right there He was also talking about his experience so candidly about their marriage and all that they were on the same page Weren't they yeah, and here's what that represents that represents masculinity and femininity getting along and interacting in a really positive Productive way because she's very feminine. Yeah, and he's very masculine. Yes So it wasn't just them as individual human beings getting along, which is excellent and good as a married couple But really represents like what needs to happen in culture. We've been a culture where gender relations are collapsing The genders are kind of at a nonviolent war. There's a lot of combat intention The 2010s as we saw with me too and the rise of super radical feminism was an example that but they have a very healthy relationship And that's very healthy gender relations We just saw and to see that at the event to see her promoting patriarchy and saying she cherishes it That was seriously impressive that blew me away. Yeah, but also say with the conferences I mean, yeah Running three three conventions at the same time that was a challenge and I knew that would be a challenge We've wanted them one at a time before that now last year We did three at three throughout the year, but they were separated not compressed into one So as I tell them the speakers I was joking. I was like last year. I did three conventions in the same year Now I'm gonna do three at the same time for my next magic trick Done boom. Do you see a fourth convention being added or is? Yeah, for sure like we could do the under 21 convention again We've done three of those in the past for young men. That'd be a fourth one We could do I don't know if I want to do for at the same time I just did three at the same time and it was like I knew I could do that I knew in a confidence way. I had belief in myself I knew I had the technical ability to do it and the intensity and the energy required But it's different to think and know these things and believe those things and then actually do it physically like walk your talk So I pulled that off and that was I guess I'm proud of that too I'm not just I guess I'm proud of that But that's a highlight of it is being able to do that and feeling that in the moment during the event But also you mentioned like some highlights I'd say seeing you know a woman speak at 22 Jennifer Malski I was a lot of fun. We got woman's playing and going They promised a man's planning and the game a side of woman's playing at the end there. That was fun How'd that go? I didn't get a chance to see it. Well, good. Yeah, she was very animated like we were talking about I think it's gonna translate well on a camera Okay, and I thought you did as a speaker one of her first times didn't public speaking she did really well Yeah, I was impressed. I would say too that my own speeches. I was really happy how they came out Yeah, I like focused on you guys primarily terms of marketing stuff But you know being up there and having the lights on you and the cameras and all the eyeballs and stuff Really happy thought they came out and it's difficult to pull off speeches in the middle of running three conferences Yes, or even one conference. So that was badass Seeing the patriarchs. I'll say this to seeing Tanner and Zach operate almost completely independently the patriarch edition That was a real like proud moment for me to see that happen.  I mean it was it was unreal But even seeing you know walking on the hallway and just seeing the men at 21 convention and feeling and seeing the woman at 22 And the father's down here. I mean it was a whirlwind of this energy and action and intensity and dialogue and intellectual stuff content and emotions like Yeah, I was unreal man super summit 21 super duper triple summit. It really was now there were guys sitting in the patriarch Convention whose wives Were in the 22 convention and that's interesting how a husband a wife can attend an event like this There was one step further than that There was a patriarch attendee think it was only patriarch attendee and then he had his wife at 22 convention And then they had their eight-week-old daughter With the mom and 22 who's pretty well behaved and pretty pretty quiet and all that Yeah, so she was a technically the youngest attendee ever Over any of our conferences eight weeks old and I was like gosh He's the greatest woman ever and right off starter boom make her great again.  They were mothers in there. There's women who never had children There's some feminists in there a feminist one or two at least. Yeah And a couple grandmothers. Yeah, that's a pretty good That's a very diverse group of women. Mm-hmm different races, too. Oh, yeah different religions all kinds of stuff Yeah, all the stuff the the the feminists and the lame-stream media and then these radical lefty wackos and stuff Like they talk about diversity, but they have none of it. They hate diversity of ideas and they hate diversity of opinions Yeah, we have diversity across every spectrum. You can imagine there's did there's you know There's I'm an atheist speaking alongside Christian pastors and Christian reverence and they even they don't get along on everything They get along, but they don't agree and everything. Yeah, it's some important ways Yeah, and then I'm an atheist and then there's other guys like me and then you know all kinds of Yeah, we just have a lot of that stuff in a very healthy and positive way that the feminists and stuff They they pretend to have that but they don't we have in spades By the truckload and it's cool and it works and it's put together in that way where we share common values and missions and purposes, you know, like family and fatherhood masculinity and femininity So it's pretty pretty amazing to see that. Yeah The 22 convention was not filled with victims was it? No, I think it was very much the opposite woman who wanted to Uh make themselves great again, right? They wanted to support what we were doing even above and beyond themselves And I heard even uh, I don't know if it's Pat Steadman or Socrates or somebody But they said they were talking to a few of the female attendees in the hallway You know the kind of the hallway conversation experience one of the most important parts of attending any 21 event or 22 event But they said it was a therapeutic to just to be there Uh just to hear the speakers talk to them get the the mansplain and as we call it But it varies genuinely sincerely that like this is awesome because they don't get to hear from men You know feminism, I think is really strangled masculine voices in culture Grandfathers and uncles and fathers. They're not really able to talk Especially once the children into adulthood young to women anymore. That is neutered It's neutered not only in real life and day to day life But in movies and tv and art and entertainment and news and stuff It's rare that you see that like a masculine voice on tv Like where do you see that Tucker Carlson? Maybe or some or trump and super bobastic kind of way But it's really rare and it has to be at a high level And so they got a lot of that they got unlimited mansplaining basically For me and all the speakers there. So that's cool. You know, I'll say one more thing too That I'll say that this is this is a moment that I I want to get on camera And 22 can mention them one of the water breaks, you know after a speech between the speeches There was like a feminist girl and she said she was a feminist the two girls behind her And they started arguing didn't get to heat or anything But they started arguing and this girl was again the feminist blah blah blah and this girl was like no feminism is super toxic And they were kind of not I'm maybe over playing the the voice You know inflection on it But it was really kind of like that and I could see them there wasn't Anger or madness, but What a couple of inches from that and it was there it was combative Yeah, nothing too crazy, but it was combative and I was I was like that's awesome Where do you see that like no women's arguing with each other about it? And I was so proud to see that and I was like that is a spark that could light a fire And we're getting that kind of stuff on camera here at 21 not the attendees per se, but the that's kind of conversations See it was a magical event. I would say all of it. Yeah, I heard more than a few women Say after chatting with them out in the lounge area and in the hallways The common theme was I feel like I was brainwashed. Yeah, that's what they said Those were their exact words. I feel like I was brainwashed. Yep I agree. And I know Pat Steppman talked a lot about that in one or both of his speeches Uh commissioning and propaganda and stuff has been super effective and feminism I've out crossed many spectrums of life out in the west, but feminism is a big one I called it my speech the religion and I had previously too Basically, it's the religion of modern women if you look at religion in terms of a set of beliefs that guide your life In terms of important decisions and day-to-day beliefs and interactions Family relationships marriage things like that It really dictates how they feel and believe in what they're going to do with big decisions in life and small ones too So, yeah, it's it's brainwashing for sure and it's bad and it's toxic and it needs to end The idea with feminism today that feminists will openly propagate Is that you know feminism needs to go on indefinitely into the future like when is it going to end? They have no answer just forever forever and ever and ever and ever You're gonna smash the patriarchy forever even though it's already in pieces and burnt to the ground It's delusional crap at this point and it needs to end like you know quality and before the law is a good thing You know quality before god or nature or whatever and that's a good thing beyond that This is super toxic Orwellian crap and it's got to end and we are actually doing something about that This is the real in my opinion the first cultural pushback against feminism That's not political. It has very little political involvement at all It just looks like it was just funny and pisses off a lot of people which is fantastic gets attention you know triggers them Yeah, we got to push back against it in a useful funny provocative way That you know walks a lot of good lines and balances even killed and in spite of the ambabastic marketing I think we've actually hit the nail on the head and the woman who attended saw that first hand Skeptical or you know encouraging as they were walking in whatever the case was. Yeah Yep, I loved you too, man. I know that heard them.  She was almost twirling her hair. You could see her just kind of like George Bruno But she was like, yeah, you can tell he is a hairstylist And my safety coordinator my guy teedog.  Yeah, it's two completely different trainings Okay, okay, but for the woman, that's just how she referred to you and she yeah, that's just her experience I guess insulting a silver. Yeah, so she feels it and that's why I'm risky too. Yeah, women love them men revere them And I knew you would have that effect on them. Yeah, Texas got them too. They really love Texas. I I had heard that Yeah, yeah, which is kind of surprising, but I'm glad it I'm glad it happened. Yeah, not too surprising but interesting Do you think this was Do you think the 21 summit And the way it was executed Is successful enough to want to duplicate it again A year from now. Do you want to do it again immediately? Yeah, 100% Yeah, tickets. I think one or two sold last night already. Yeah We're ready to roll We're gonna do, you know next year 2021 the year of 21 21ception 21 conventions and 21 21 2021 all that stuff 21 summit 21 convention 20 of the major paycheck edition unlimited 21s We will have 22 convention again the second one and the plan is to keep repeating it Uh, more or less, you know, indefinitely until I just don't feel like doing it or there's a reason to stop doing it So yeah for next year for sure the second 22 convention in 2021 at 21 summit Probably going to be uh, it's going to be in october I don't know the exact weekend yet, but october 2021 And I think it'd be bigger and better the fact that we had we pulled it off You know, we had to change the schedule due to covet, you know lockdowns and crap And then people couldn't even travel in due to you know travel restrictions and you know concerns about the virus and stuff The fact that we actually pulled it off and then on top of two other events is just amazing Uh, you know, I think the media the mean a good morning britain, for example, they were kept contacting me before the event Uh, you know, I almost went on the show again on the start of the event at thursday morning Scheduling issues, so we ended up doing it But basically they were kept asking me repeatedly even like two weeks out in a week out I'm like, why are they asking me the same question again? Like is 22 convention happening? I'm like, yeah, it's happening Like tickets are on sale the venues paid for and booked like we're doing it and they just couldn't believe it Maybe because they're in britain and britain has really the uk has heavy lockdown restrictions So that's their reality, but in florida and here and i'm like, yeah, we're doing it And sure as shit. We're done. Boom. It's just in the hard drive ready to rock. Yeah That is Well You beat the odds number one because there were some people that were saying that You'll never be able to pull it off. Yeah Because of the restrictions across the country. A lot of people are saying that. Yeah, yeah Inside and outside the man's fair. Yeah the controversial nature of The 22 convention which the execution was flawless And also I just think the concept of the summit Of having everything together Under the same roof is very unique challenging. Yeah. Yeah How far did people travel from? We had a few international that was you know limited We lost a lot of speakers due to that due to the last minute due to cobit stuff But yeah, they traveled mostly from the united states around a few from florida but not many Most of they were flying in So that's something yeah, but we beat the odds man I've called myself previously since about 2018 an unstoppable american entrepreneur But I guess really i'm the unstoppable american dream and I proved that To the to the nines here to the tea you did. Thank you. You did absolutely It's kind of surreal to actually do it, you know in the middle of the pandemic and all this crap But I knew that I could and I knew if I'm a nose of the grindstone and surrounded myself with the right people Good people good men a good woman too.  Yeah, this is really really awesome I can't wait to get the videos edited online and just start just blasting the internet and you know angry and more feminist Your volunteer staff were incredible. Can you comment on them? Yeah, they're critical to making the event happen We have staff and speakers and you know paid guys too crew, but the volunteers are necessary for a lot of What do you call it? Fine tuning and the the odds and ends the eyes and you know dotting the eyes and crossing the t's And without them the event would not run to smoothly. So i'm really glad that they came out We have them every year since about 2012. We started adding volunteers And I love them man that this year that they they did really good too.  It's not like this pre-recorded thing. It's happening like live on the fly It's like a theater. It's a production times three now or four counting the 20 port See, I just worked out really well the energy Maybe people were pent up this year because they were so locked down and cooped up the whole year With a lot of stuff like that a lot of stuff is canceled. The bars are closed around the united states and stuff Just one just went awesome for whatever reason Maybe we have a man of sphere guardian angels, you know i'm looking over us very well The private man and stuff on stuff in arneo. Yeah Yeah Let's talk a little bit about the new speakers. Yeah Comment on some of the new speakers Coach greg was one of the most impressive. I agree. Someone commented to me that he was probably the Like of the best new speaker or the the most surprising to like the dark horse. I guess you call that Yeah, coach greg man. He just I had that feeling going into I even was uh, you know teasing that he makes a good candidate Socrates better watch out for vice president of the man of sphere That's due to his beliefs and his attitude towards the man of sphere and about the connections and the unification of it Unifying it behind like common values and common messages and themes That are universal among, you know, all the different groups and stuff like that See a coach greg was uh, easily one of the best new speakers and probably the best new one And he had the you know, it was his first time here only only a few speakers spoke at all three events Some did one a lot did two and he did all three. Yes other speakers did three two like yourself And I was like, I don't know if you could actually I knew you could push come to shove do that I don't know if that was a good idea though because you have to do like over It's like two dozen interviews like this, which is a lot of work But you did all three and you did great. We did it.  He spoke at all three Socrates has been here for a long time. He knows the drill and he did all three To put that in coach greg was tough and that dude handled it like a boss He did he had some uh, there was even a woman who was combative You know arguing with him outside the hallways and stuff for his speeches And even it's a lot of the q&a in 22 was very energetic. Let's say it was and everyone handled it really well Coach greg though, man was just on it. I saw him take on the hallway some seriously intense like I just kind of verbal combat and this guy was absolute rock solid frame the dudes got People know him as migtau and you know, I guess stand away from women or whatever the peace sleeve stuff But that dude's got game like I can guarantee he's got game To describe him as migtau is actually putting him in a box. He is so much bigger I think he's self describes is I think he does anyway if he's not I mean, I'll take that back, but I think he describes himself that way And certainly a lot of his followers I think would take would consider themselves.  I've known him since last year I was good. He came out good thing But there was one new speaker added pretty late in the game and that was jay vincent And that was really awesome. He spoke at 21 and 22 And for him as a fitness speaker to do 22 is actually, you know, kind of risky and pretty interesting And he took to it like a duct of water is great He did but the reason that he's interesting to me is he was the youngest speaker here I believe he's 30 years old. He's two years younger than me So I got older. I used to be the youngest guy at the event now. I'm like not even the youngest speaker Yeah And we have attendees, you know, eight weeks old or 18 years old too Some of the fathers brought their sons a one father brought two sons. I was like, damn, he bringing a whole whole litter is great Um, but jay vincent did great because this can well it was important to me because the 21 convention event in particular The main event it is a long history and what's called high intensity training or the school of arthur jones for exercise Essentially that spirit of arthur jones He died in 2007, but he was a inventor and uh entrepreneur and millionaire and stuff And we have a net speaker speak on that issue for high intensity training for a couple years now We've had a lot speak on it the biggest names in the world for it. Dr. Ellington Darden, Dr. Doug McGuff, Bill Day Simone Jim Flanagan huge guys in that community that anyway, this is the community It's like the atmosphere and in a sense for exercise. It's its own, you know, subculture And so for him to pick up that torch and he actually started his business After speaking or not speaking of it watching 21 convention speeches from those speakers in the early 2010s Dr. McGuff's speech in particular from 2010 That was awesome to see that continuation of a generation of ideas And he killed it too. The speeches were badass. They were good.  Yes, of course. I was really impressed with him Yeah, truly was great addition to what we do and so totally familiar with the manusphere too Yeah, so he's a good fit unlike some of the other guys in high intensity training Which would be great for exercise, but they don't not super familiar It's interesting to them to meet the speakers and learn about what we're doing But jared, he knew all that and everything I needed him to know about high intensity training and exercise and fitness So it was a really good match and he's young too. So it was a really It was a unique speaker dad to the event. Yeah Are you going to have a red man group Filled with speakers to do kind of a A post game debriefing kind of thing like I did last year Yeah, we'll do it probably this saturday or next saturday for sure. Okay easily. Yeah. Yeah. I look forward to it. Yeah, definitely Well, this was a great talk. Thank you for a great conference a great summit. You pulled it off It went off well And the fact that you want to do it again and you have no regrets about it My only concern is that it's a year out that my only I need I need to you know, I could use a vacation But yes, um, it's sad. It's sad that we don't get to do it for a year Even though I'd like the time and I will use the time to build it. Yes, and it's sad that we don't get to meet each other again Probably for a year. Yeah, maybe some of us obviously. I see Tony all the time and your dad But in Socrates, but that's you know, a very small selection of dozens of speakers So the meeting at the meeting face to face is just so amazing, man. Highly underrated I can't believe how fast the last year went by. It seems like we weren't just doing this Seriously last October. I can't believe a year went by. I know and the next year is going to go by justice fast So you've got work to do. Yes, sir. Yep Me and my guy my editor. That's right Thank you, sir. Thank you, George the conversation with anthony dream johnson Founder and president of the 21 summit and of the manisphere


In developing countries, financial services will be provided to over 2 billion people who currently have no access to bank accounts. ITU standardization is working to build a human-centric ICT environment, where physical space and cyberspace are managed in unison. We see a future built on international standards created in a spirit of collaboration. Standards that are created by ITU.


using these search engines and we will use our software called search query for that and you can see that most of these most popular freely available ones are implemented in search query but that is for the practical and when you go through the tutorials you will play with these engines but so it's very easy to use them because we built nice software around them. And to look at the results we have a tool called peptide shaker. Again, we will use that extensively in the practicals. Just wanted to preview this for you where you can see the match between the theoretical and the real spectrum. You can see the B and Y ions plotted on the sequence and you can see them annotated on the spectrum and so there's lots of information here to play with including multiple possible matches to the same spectrum and how they differ comparisons between different search engines if we run multiple you can map stuff onto 3D structures and you can see the scoring engine and the false discovery rate stuff close up in detail but that is something we will talk about next where this is calculated but all of this will be visualized and played with you in the spectrum and then in the practical, sorry and then you will be able to find out about all of these things. Okay, there's other stuff in there as well we won't have time to go through all the details. Now there's special kind of search engine called a sequential search algorithm and these guys actually try to do what I started to do very at the very beginning of this whole identification thing and we're going to try and jump peaks like some kind of crazy Super Mario and we're going to try and find distances between peaks that correspond with amino acids but we're not going to do it throughout the whole spectrum we're just going to do it for a bunch of peaks and we're going to get a few possible amino acids and then we're going to append that with the mass that is not explained here which is in this case 1,080 Dalton's almost and then the mass that remains here which is this peak subtracted from the precursor mass so this is what is left here and then we're going to say we call this a sequence tag which is there's 1,080 Dalton's of something we don't know what that something is but it weighs this much then there is SD IRL remember IRL we can't tell apart and then there's 303 Dalton's left and then we're at the end and then we're going to ask the database have you seen any peptide with the total mass of this peptide which is a good filter and then we're going to say it has to contain SDI or it has to contain SDI and that peptide if you find one or more of these whatever comes before SDL has to have this mass and whatever comes behind that in the peptide has to have this mass and if there is a match we say that's the one we've identified and this is a search engine that's built on a completely different principle it starts from the spectrum extracts the sequence a priori so without looking at the database and then matches this to the database and this approach was originated by Matthias Mann who is a very famous scientist in the field still in 1994 exactly the same year as Sequest now interestingly enough this approach very quickly started losing interest because the database has got too big and you've got too many ambiguous results because you don't use the full power of the spectrum you're only using a few peaks which is actually not the best way about this however in Sequest you can actually use all of the peaks in theory because they could be matched to a theoretical peak so obviously it's better at resolving the ambiguity than this particular approach so this approach was kind of lost if you like and the Sequest approach won out that's how everybody built their algorithms afterwards nevertheless there's something really fun you can do with this Dave Tapp in 2003 and then we find it in 2008 made a tool called Guten tag and direct tag which are actually so this is the 2003 one this is the 2008 one which actually builds on the Mann approach and does something special I'll show you what they do so they generate these short bits of sequence the sequence tags from four peaks and then with this list of sequence tags they're going to look up in the database and notice that you can have multiple sequence tags from a single spectrum and they may or may not overlap you look up all the possible peptides in the database that could match any of the tags which is what you see here and this is where the original algorithm would stop but then Dave actually had the clever idea to put a second stage sequence scoring in so he's now going to score each of these going to create a theoretical spectrum of course and then score that theoretical spectrum against the real one so he's using this as a filter then making spectra and then scoring the spectra with a sequence like scoring and then get a result the cool thing is that because you use an additional filter you can now also allow say an amino acid mutation to take place and see how that scores against the spectrum and then potentially get an identification of a mutated sequence so this is where they kind of position this in direct tag and he said we can allow strange modifications strange mutations which would otherwise be very difficult to find so it's a very interesting take on this but unfortunately direct tag and good and tag didn't really take off very much so they're still not very popular but they're built into search GUI so if you ever want to play with them in the practical you will learn how to do that finally you can do the novel sequencing and this is what I really tried to do at the very beginning which is by jumping from peak to peak from start to finish try to read the whole spectrum of the peptide but that is very very difficult because as I mentioned for that to work you need to have very many preferably all of the peaks in the spectrum and that unfortunately doesn't happen very often so you need really beautiful spectra nevertheless there are many algorithms available to do this probably the best one out there right now was published in 2015 is called Rapid Novore that is also implemented in search GUI but it's only free for academics it is not free for commercial use if you want to play with that it's also implemented in search GUI I'm not going to spend too much more time on de novo because the only time you really want to use it is when you deal with peptides for which you don't have a database and most people try to use proteomics only on samples for organisms where they have a good database okay so what we have done is we've looked at amino acids, peptides and proteins we've looked at mass spectrometers how these mass spectra are made we talked a little bit about issues with quantification now we're going, then we talked about identifying spectra so how do we figure out what we saw in the mass spec now we're going to see in the whole list of identifications that come from an experiment there are going to be some mistakes the question is how many can we control them and can we even pinpoint roughly where these mistakes might be which is essentially the false discovery rate calculation if you take all of the experimental spectra in a set and this can easily be hundreds of thousands to millions of spectra each of these will be assigned with the best possible hit this is the top hit that we extracted say from Xtundam remember the hit with AT as a score that was very special so for that particular experimental spectrum that peptide would be the hit but of course we have many more of these experimental spectra so we have many more peptides each spectrum has one peptide or zero peptides but then these spectra are not counted so here you see the scores of all those top identified hits and you can see that some have a really poor score but not that many quite a few have this mediocre score and some have really good scores now we know that some of these are going to be wrong in fact here we just get rid of all the filters that we did previously even in Xtundam all the E values are gone all the E value filtering sorry is gone all the E values are represented so we start here with very low E values if you like and low E values means high reliability and here are the high E values with the low reliability the question is we have to somehow put an E value cutoff to make sure that we have all the good stuff to the right of the cutoff and all the bad stuff to the left but how do we do that because we don't know where good or bad is now if you've learned anything from my explanations of proteomics and how we do identification by now it is that we when we need to find out where is the good stuff and where is the bad stuff that we need to take somehow an empirical null distribution so how do we create an empirical null distribution here well ideally and this is theoretically we would have perfect knowledge of where all the bad guys are and all of their scores which are mostly here and then very few high scoring ones and then we know where all the good guys are some of which look like bad hits but most of which are special and don't look like bad hits mind you this is what such a curve will look like if you have a good scoring function we have decent scoring function so we would expect this to look like that and here we could set a threshold where the number of bad hits is small compared to the number of good hits and how small well we can determine that value we could say the ratio of bad versus good has to be less than five percent for instance okay or less than one percent and then we can choose where to put that line so that we achieve that desired condition that's the theory okay now in in practice however you may want to have a little bit more fine-grained detail so as a morbid example let's say in the whole class of statistical genomics we say there is a psychopath okay one psychopath so that gives us a percentage number of psychopaths in the class so if you are 35 students and there's one psychopath there the ratio is one on 35 but that doesn't help you figure out who to avoid in the group right so if you want to know who to avoid in the group and I would say well one thing the psychopath is not online the psychopath is participating in class today right which is a bit silly because we're doing this all online but you get to drift right now if I say that it automatically makes it easier for you to limit the scope of that or the presence of that particular person and so that's what we're aiming at here we're aiming at trying to limit the scope of where the bad guys could be the way we're going to do that is we're going to have we're going to calculate a local false discovery rate and the way we can do that is by saying what is the ratio of good versus bad at every given step in this histogram for instance here we have practically no good hits and practically no bad hits but in fact there are zero good hits and say five or six bad hits so the ratio of bad versus good is 100% so everything is bad and that continues for a while up until here where we start to have maybe one or two good hits and the rest bad hits so here that ratio is 99.9% then that ratio continues to fall as more and more good stuff starts to creep in until we hit this point here which is half half and if we go down here half of these hits are good and half of these hits are bad so at this score point we know that 50% of the data is reliable and 50% is not that's very accurate information right that's something you can work with and then as you continue you get fewer and fewer bad hits until all your hits are good and so you get 0% bad hits and everything is good okay so if we have something like this we call that a local false discovery rate that's because it's empirically estimated if you had models for this you could call this a posterior error probability so this is just as a a means to remind you that this is kind of the same thing but this local false discovery rate which for every score gives us the chance of finding a false positive a true positive hit sorry a false positive hit in this case so a local false discovery rate at each core gives us the probability of finding a false positive and so here for instance is 50-50 but here it's 0% and here it's 100% because everything is bad now we know that this is super reliable this is very very reliable this is kind of reliable this is 50-50 which is actually quite bad already and this is 100% bad now we can again put our threshold but we can do it based on the some of the local false discovery rate so in fact we can do an integration exercise so we make it calculus okay because we fitted a model to this now this is all in theory it's all nice in theory how do we do it in practice well in practice what we need to do just like we always do is we have to make a lot of crap and if we make a lot of crap we can see how crap behaves and then we extrapolate that the made-up crap is behaving like the real crap and therefore if we look at what the made-up crap does we kind of know what the real crap is doing how do you make up crap well you make sequences like my name is a sequence you make sequences that don't make sense for instance the easiest way that everybody uses nowadays is you take the sequence of the protein and you flip it around and now this flipped around sequence is being put in as a fake protein and it's labeled as such and we call these fake proteins decoys and of course we make one of these proteins one of these fake proteins for every real protein a cool effect of doing this is that every amino acid prevalence remains the same so there's three ends in this protein and there's three ends in this protein there's two a's in this protein and there's two a's in this protein and ends tend to co-occur next to other ends in many cases like these two and that's the same here and for instance my a's tend to be close to a's and that's the case here okay so we maintain a lot of these features of the sequence here so all the patterns remain this is very important because you don't want to mess with the patterns because you might have consequences to messing with these patterns you can also shuffle the letters so think of these as scrabble blocks and then you put them in a shaker and then you roll them out in a new order but then you actually destroy a lot of these patterns for instance the a-r-t-a-r-t pattern is gone here yeah so this is not often used also this is not reproducible if i shuffle and then you shuffle we will get different patterns and this is actually computationally much easier so most people use this you can do other stuff but let's forget about that it's not worth it people have tried this is good enough it's as good as all the other stuff if not better so reversing is what everybody does now with these reversed search spaces we can plug them in together with the real search space with the actual database as a concatenated hit is a concatenated database and then we can see how often does a spectrum match a decoy hit and at what score do they match so now we've got all the target hits and we know that these target hits in the real database contain crap we just don't know where it is and we've got hits against the decoy data set and the decoy data set every hit is crap that's the assumption because we made the decoys as crap so let's look at where the decoys are oh look at this the decoys start going down dramatically here whereas the real data actually has a tail here so what can you do well you divide the estimated number of crap hits which are the decoy hits that's our estimate our empirical know divided by the target hits and that gives you your false discovery rate and as you can see that local false discovery rate starts falling down here where you have 50% of your hits are expected to be random hits because the decoys are 50% of the total and then here you can see that they go to zero whereas here it's 100% because you have as many decoy hits as you have target hits okay so this is what we're going to do we're going to append a decoy database to our searches and as a result we can calculate the red curve and we can calculate at each point of this curve with the blue curve we can calculate what the local FDR is and we can then set the cutoff so that the sum of local FDRs above that cutoff is no larger than our particular threshold for instance one percent or five percent or two and a half percent or whatever you choose okay so that's the way we do that in proteomics we'll also look at that in the practical and of course once you set a threshold you guys know that there's true positives and true negatives and you want to maximize these numbers that there's going to be false negatives which we really hate because that's good data that we're missing and we have false positive which we hate even more because that's bad stuff that we think is correct and therefore is going to mess up our downstream analysis so usually what people do is they try to limit this because they're really annoying downstream mind you if downstream you can do a very fast cheap and easy essay to filter out between these two then maybe that black line can shift all the way here because you don't care that you have to still filter stuff out if you can do it cheaply quickly and reliably however if downstream you're going to have to invest an enormous amount of time and effort to validate each of these hits maybe you want to move that black line over here because you will not have time to do more than this anyway and that means that every ounce of effort you invest is likely to be good effort so it really depends on what you want to do with the data where you put the threshold okay right and now the very last bit because this has gone long this has gone on for long enough I think you will agree with me protein inference so I'll very briefly explain to you what is bad, ugly and not so good about this so fundamentally we have four peptides identified in my hypothetical example and we have three proteins in our database so all nice and simple real life is much more complicated peptide A can occur in protein X but we also find it in protein Y so this is a peptide that matches to more than one protein B is only found in protein Z C is found in X and Z and D is only found in Z okay so again very simple very simplistic example the question now becomes given the identification of A, B, C and D which proteins were in the sample so there's many ways to go about this and fundamentally you have to make a choice based on philosophy so as you may know there's something called Occam's razor which is usually quoted as the simplest solution is the best that's something we use a lot in science where we say we take a simple system and we assume no crazy stuff happens and in this particular case Occam's razor would say we need to take only those proteins that will allow us to explain all the peptides and so what it turns out that we can explain all these peptides by having protein X and protein Z in the sample because protein X explains peptide A, protein Z explains peptide B and then protein Z or protein X can explain peptide C and protein Z can explain peptide D so we're done we only need these two proteins and we can say this is the minimal set of proteins that explain our peptide observations and we call that Occam's razor or the Occam set however there's a problem with this because those of you who are perceptive and still awake have noticed that you could also take protein Y and protein Z and you could explain A, B, C and D so why do we not take Y and Z for our identification? Well as some of you will then figure out this guy has two peptides and this guy has only one so it's capitalism this protein has more peptides and therefore must be better right? and that's how most algorithms will do this they will say yes but this guy is richer in peptides and therefore we like him better that's a bit sad because you know this peptide which is something that protein X has that Y doesn't have is actually a shared peptide so is it really from protein X? we don't even know okay but nevertheless that's how these algorithms work and you can see that there's sometimes equivalent sets and it's not always that clear cut it's also algorithmically quite complex to calculate this now Ockham is a particular type of philosophy it's a philosophical point of view and for every philosophical point of view you have the exact opposite and the exact opposite of this is called anti-Ockham and that states hey wait a minute why are you throwing protein Y under the bus here? you are now declaring a protein absent from a sample even though you have some evidence that it may be present so you are not God you're not supposed to do this kind of stuff you have to include every protein for which you have tentative evidence and that's not an unreasonable claim to make and so that we call the maximal explanatory set it says you must include every protein which has at least one matching peptide be that a shared peptide or not and that is called anti-Ockham also notice that algorithmically this is ridiculously easy because what do you do? you just go protein X does it have a peptide in the identified? yep in the list protein Y has a peptide you're in the list so this is extremely easy now there's another thing that you can do and you can say yes but we don't have enough information in fact protein X turns out to be purely hypothetical and nobody's ever seen it it's a pure prediction from the genome we don't even have an mRNA never seen one protein Y on the other hand is beta-actin which is literally everywhere so now what we're going to do is we're going to say you know what? we don't believe in hypothetical proteins when we have an explanation that makes a whole lot more sense beta-actin is everywhere everybody finds it all the time so we're going to claim that protein X is absent whereas protein Y is present because we use additional evidence that we take from knowledge that we've acquired earlier and now we call this our set and we call it the minimal set because we exclude protein so we make it a minimal set which retains maximal annotation and in a way you could call this the true Occam's razor and why do I call this true Occam's razor? well because what you've effectively done is you've eliminated hypothetical entities in favor of real known solid entities and this is actually when you look up Occam's razor on Wikipedia it does not say that the original statement is the simplest solution is the best it specifically states the solution with the fewest hypothetical entities and in a way that makes sense here so this is something that I personally think is a much better way of doing this unfortunately I have no real evidence to back that up I have some but I don't have time to go into that it's also not that important what you need to learn is that the list of proteins that you end up with is different in every single scenario you see they're all different and that difference does not come from the data that difference comes from your philosophical point of view and that's a problem because it means that you cannot reproduce this unless you know what philosophy is behind this and so protein inference therefore is a really big problem so why do we make such a big fuss about that? well if you want to quantify a protein well first you need to know is the protein really there so in this case you will not quantify protein y even though it may very well be there in this case you will quantify all proteins but maybe protein x is not even there and then the other thing is peptide a are you going to let it contribute to protein x are you going to let it contribute to protein y or to both right so it becomes very complicated when you go into quantification and as a result some quantification algorithms will only take into account peptides which are uniquely matching to a protein like B and D because they say these guys we don't know what to do with them they confuse us and we will leave them alone now how big is that problem of finding multiple proteins per peptide it depends on whether you care about isoforms all of these databases include isoforms all of these databases do not include isoforms so this is all explained in a review in 2013 but to make a long story short in this particular database which is SwissProt which has one gene, one protein so no isoforms for human data 92% of all the peptides in that database match to only one protein so if you pick a random peptide you've got a 92% probability that it does not match to more than one protein so in fact situation B and D is 97% of that database sorry 92% of that database so 92% of peptides are like this and then 8% of the peptides are like that so the vast majority of peptides do not give you any problem however if you care about isoforms then now you can see that we have only 36% down to 26 so on average let's say 32% 33 if you want to make it easy so one in three peptides matches only one protein the rest will match multiple proteins but they will most likely be isoforms so proteins that are spliced differently from the same gene so if you care about the difference between different splice isoforms life could get very difficult in quantification if you do not care about that life is going to be reasonably good okay so it really depends so peptide shaker will show you this and a blue circle is a peptide and a red circle is a protein this is a typical swiss plot situation all of these peptides beautifully mapped to only that one protein this is beautiful perfect and simple this is okay because most of the peptides match to only one protein and some of the peptides could also match to other proteins but because the evidence here is so overwhelming we tend to think this guy is here and we tend to not think these guys are okay this is really nasty because this one peptide could match to all of these proteins and it really doesn't help us anyway okay this is also nasty because these peptides are each of them matching to very many proteins so which protein have you found it's practically impossible to tell so this is really bad also this looks like a bunch of balloons which is nice and this one looks simple but it's nearly a fully connected graph which means that again these three peptides will not allow us to differentiate between these proteins so if you come across something like this quantification is going to be extremely extremely difficult of course the same thing goes here the same thing goes here here and here however it's easy worst-case scenario you drop these three and these two from the quantification you still have plenty of data left for this protein okay so that's just how to show you how peptide shaker will show this to you so that you can estimate how bad is the problem in fact we back in the day we built a tool in 2010 to look at protein inference issues and what is relevant here is that this is the protein sequence you see the peptides identified and when a peptide is colored in blue it means it only matches to this one protein when it's colored in yellow it matches to more than one protein okay and so what is the issue now if you look at the ratio this is the ratio between two samples say patient control for that for each of these peptides remember we always compare a peptide with itself so the ratio between patient and normal for this peptide this peptide and this peptide are three lines here and you can see them they're very closely matched which means that that is probably correct and we know that that is probably correct for the protein because of course each of these peptides is only found in that protein now if you look at the yellow guys we can do the same ratio for the yellow guys patient versus control and you see that all these yellow guys are over here this by the way is the global distribution of all the peptides of all the proteins and you can see that most of them are not changed zero and this is a log scale log ratio so you can see these are actually going down a bit and these are going up a bit so they're at completely opposite sides of the center point and that's probably because what we're measuring here is the superimposed signal of two or more proteins you're seeing the signal from the original protein but you're also seeing and possibly more strongly the signal from the other protein which is also present and which may actually be up-regulated whereas this one is probably down-regulated and so but not significantly in this particular case so here you can see that these shared peptides completely alter your perception of the regulation of the protein compared to the unique peptides and you can see here in green you can see the median and the average for all the peptides and it's heavily, heavily skewed by these yellow peptides so in this particular case you definitely would do best to ignore all of the yellow peptides because they do not give you an accurate representation of the quantification of that protein between when you compare healthy and disease right so that's all I have for you thank God because this has been going on for long enough thank you very much for watching this class I hope to see you in the practical in real life or online and if you have any questions about this lecture make sure to ask me those questions then or otherwise you can always forward them either through Professor Clement or by sending me an email thank you very much for your attention and with that we're done


Yo, what up welcome back to another episode that I'm making just for myself Dear future Jason, please watch this really quick informative video if you ever smack your head so hard that you completely forget how to shoot 35 millimeter film Hopefully in that case you don't forget that this video exists So you think you have the balls to start shooting 35 millimeter film well before you embark on your quest Here's a quick breakdown on how to get started Let's begin with the most obvious thing you'll need a camera and a lens for beginners We really only need to worry about two types of cameras point shoots and SLRs Point shoots are little compact cameras that make all your dreams and photos come to life through the power of automated Computer technology all you have to do is load the film and the rest is taken care of try to enjoy these times when computers are our slaves and not vice versa SLR cameras are a little bit more complicated SLR stands for Lazzania recipe Wait, no it stands for single lens reflex popular 35 millimeter SLR cameras for beginners are the Pentax K 1000 the Olympus OM 1 the Minolta x 700 and the love of my life the Canon AE 1 if you're on the hunt for a camera your parents or your Grandparents might have one line around that you can use you know after they mark up the price and sell to you Alternatively you can cruise eBay I'd recommend looking for a listing that has been tested and confirmed working once you get that camera Make sure to look up what type of battery it takes and pop that bad boy in no not into your mouth into the battery compartment of The camera most SLR cameras come with a standard 50 millimeter lens Which is a great lens to start with because supposedly 50 millimeters is nearly equivalent to the field of view our eyeballs have That's just what I've heard though I've reached out to the mythbusters for confirmation 50 millimeters is what's called a focal length the bigger the number the more Zoomed in the lenses conversely the lower the number the more wide the field of view is okay You have your camera in your lens, but what about film? Let's see a good film to start out with is my personal favorite Portra 400 because it has good range and produces great results especially with skin tones Portra 400 isn't cheap though So some less expensive alternatives are Kodak color plus and Fuji c200 if you're going through an artsy phase Ilford HP 5 is a very good black and white film. Okay, let's load our film to pop the back We'll want to lift this module or something similar on your own camera until the back hatch pops open Take your film and insert the cartridge and now push that top module down They should hold the cartridge in place grab the lead of your film and pull it out to the other side of your camera And insert it into the gap of the spool no need to be nervous You're just loading film not performing open-heart surgery in the bathroom of a Las Vegas Quiznos with your thumb on the film And the spool to keep it in place you can advance to the next frame by pushing this dongle here Haha dong get used to this you'll need to do this after every shot you take in every move you make Hopefully your film is caught and by advancing you pull out more film from the cartridge You'll want to make sure that the sprocket holes on your film line up with the sprockets on the camera fire the Shutter and advance one more time everything looks good. So let's close the back Now we need to get to frame one advance your film until your camera shot counter reads one If your film module over here isn't spinning when you advance your film is not caught on the spool at this point You definitely should not open the back of the camera until we rewind the film. Oh boy Here it is the moment you've been waiting for time to fire off some bangers Well, hold up their skivvy not so fast before we can snap off some fire picks for our tender or grinder profile We'll need to measure how much light is in the scene This is our vocab word of the day light metering a lot of 35 millimeter cameras have light meters built into their Viewfinder, but first you'll need to set something called ISO ISO or ASA is how sensitive your film is to light Portra 400 and Ilford HP 5 are 400 ISO color plus and Fuji C 200 are both 200 ISO Which means they are less sensitive to light like how I'm becoming less and less sensitive to emotions now your camera is basically a Child and children are idiots So it has no idea what ISO your film is thus you will likely need to set it manually with a Canon AE one All you need to do is lift the outer cap of the shutter speed dial and set it now If you look through the viewfinder and half press the shutter button you should get a reading by gosh We're on to something now it's time for a crash course on the most important part Exposure so now we know what ISO is but there's also the f-stop and the shutter speed these three settings work in conjunction to bring The forces of evil to justice and exposure shot properly for the sake of this video You only need to worry about f-stop and shutter speed once your ISO is set f-stop can be found on your lens Typically the numbers go from around 1.4 to 22 the higher the f-stop number the wider the range of your depth of field IE more stuff is in focus, but there's less light coming in and we like light shutter speed typically goes from two seconds down to a Very fast one one thousandth of a second the faster your shutter speed the less motion blur you get in your shot But the less light your film will get again we like light for handheld shots I recommend never going slower than one thirtieth of a second now every camera is different I recommend reading the manual if you can find an online version of it My AE one here is set up in a way where we select what shutter speed We want to use and then it tells us what f-stop to use based on the ISO This is called shutter priority for hopefully obvious reasons the light meter in my viewfinder of my AE one is telling me to shoot at f11 If I'm at one two fiftieth of a second if I obey its commands and switch to these settings I should get a properly exposed photograph easy peasy lemon squeezy if that all seems too complicated or overwhelming worry not this process Will get easier with practice or frankly It'll keep getting harder and harder to the point you quit change your name and move to another state Hell I was halfway to Tijuana before I started this channel So you've gone out into the world and fired off your roll of shots and you've come back a new person ready to lay your eyes on the masterpieces or dumpster babies you've created Well first you're gonna need to rewind your film do not open your back before or during this process Otherwise all your film will burst into flames Press the release button on the bottom of your camera flip up the rewind crank and start rewinding the direction your arrow tells you You should feel a click and a change in tension when the film is rewound all the way If you're unsure keep rewinding for a bit now you can pop the back of your camera and all you should see is a lone film cartridge Pop that bad boy out and head down to your local lab if you live in the middle of f***ing nowhere You can send your film to one of these labs by mailer film labs will develop for you and even scan upon request as you grow and blossom into a Beautiful 35 millimeter photographer You may start wanting to learn how to develop and scan on your own as it's more cost-effective a week or so will pass before you get Your scans back via email link or compact disc But when you do finally receive them hopefully they are as gorgeous as you imagined and if they're not and you hate them Then congratulations, you've passed your final test and you can now call yourself a film photographer If your shots have light leaks then you might need to get a new light seal kit for your camera If your shots are too dark or too bright your light meter might be old and not functioning properly Maybe try using a light meter app on your phone Now all that's left to do is repeat these steps with a different film until you find the one that you like Or until you have no money left in your bank account and are forced to make s***y YouTube videos about how to shoot film