Hi everyone and welcome to the proteomics hands-on training. In this training we will learn how to use Maxquand and MSTATS for the analysis of a label-free tissue cohort data set. The data comes from skin tissue samples of 19 patients and they have different types of tumors. One group consists of metastasizing cutainous squamous cell carcinoma and the other one consists of RDB cutainous squamous cell carcinoma and we shorten the cutainous squamous cell carcinoma as CSEC here. So it's a type of skin cancer and our objective for this training is to learn how to use Maxquand and MSTATS for the analysis of such a real label-free tissue cohort data set. So we will start by uploading the data. Part of the data is deposited on Zenodo. We can copy everything and transfer it here to the upload button, choose paste and fetch and then paste the links to these three files. We press start and close. If you are not familiar with Galaxy yet, please look at the Galaxy beginners lectures and hands-on training. We then continue with uploading the raw files. Here on this head you have a direct link to the Galaxy training material site. That was where it was before. And so we can also copy the links from here. So this is raw data deposited in the pride repository and this is a repository that hosts publicly shared proteomics raw data. And we have 19 raw data files and they take really long to load and they will also take a long time in the Maxquand run. And that's why we will already load also the results of the Maxquand run, which you can find at the end of the hands-on Maxquand analysis box. So we will already get the Maxquand results and we will also upload them. Because of this Zenodo link, they now have a weird name. So I will rename every of these files. So to do so, I click on the pencil and then remove the beginning of this file name. So we have here a protein database. This is an annotation file that we need for MS Dots. And then we have a comparison matrix that we need for MS Dots. The raw files will take quite a while to load. So we can already look at these files here. So this is a human protein database with 20,000 entries. You remember from the theory lecture that this first line is the header line and it contains here the union prod ID and more IDs and the name of the protein. And then this is followed by the actual amino acid sequence. And here then starts the second protein. The other two files, so this annotation file is important because for the statistical analysis, we want to compare our two conditions. So we have the metastasizing tumor type and the RDB tumor type. And we would like to compare the files of both groups to find differentially abundant proteins between the two groups. So it's really crucial how this annotation file is set up and many of the MS Dots errors happen here. The raw file actually has to match the name in the evidence raw file column. We can later check that out. So here it's very important that this name fits to what is written in the evidence file, the output of MaxQuant. Otherwise, this metadata cannot be attached to the raw files names. Then we have the condition. So this is a binary comparison here. We have only two groups and replicates indicates that from each patient, we have one sample processed. So we have 19 files and we also have 19 different patients. And because it's label free, the 19 samples were measured in 19 different runs. And when we want to perform MS Dots after MaxQuant analysis, it's important that we also add an isotope label type column to the annotation file. But this is in all cases only L, which stands for light because we don't have any heavy spike in peptides. The third file is also needed for the MS Dots analysis. It's called a comparison matrix or table. And in this table, we specify which comparison we want to perform on our groups. In our case, there's only one possibility because we only have two groups. So we compare the metastasizing condition versus the RDB condition. It's important to write again the names here exactly as they were written in the annotation file so that they can be matched. The name can be anything. It doesn't matter. So I just named it that we want to compare the metastasize versus the RDB cancer type. And the condition that comes first has a one. And the condition that is compared against comes in with a minus one. And if there would be more condition, we could also leave them at zero so that we first compare only those two conditions. And then we could add another row with a new comparison. And then we could leave these conditions zero and, for example, compare two other conditions. But here we only have two. So this table is quite short. And you can see that the raw data is still downloading. And this is not typical for Galaxy Trainings. So often we use Galaxy Trainings on cropped and very small but descriptive data files. But in this training, we actually use a real cohort and we didn't crop the files to get a proper statistical analysis. And therefore we don't have time in the training to wait for the MaxQuantRound to be finished. And therefore, on Synodo, you can already get the results of the MaxQuantRound. So this is the evidence output. I rename it again. Then we have the protein groups, save and last, the quality control report. So we can already try. We need them in our collection later on. I know let's first rename. So the renaming might not work well if the data is still uploading. But we might be lucky here. But it can happen that if it's still yellow that it will not keep the new name. So we have only six of the RDB condition, but that's actually quite a lot. So RDB stands for Recessive Diastrophic Epidemolysis Bulosa. And this is a rare disease in which patients have a defect in the Collagen1 gene. And they cannot produce functional Collagen7 actually. And Collagen7 anchors the epidermis to the dermis. And in case this protein is not functional, the problem is that the two skin layers are causing friction, and this leads to blisters and inflammation. And one of the long-term outcome of the disease is the development of such a squamous cell carcinoma, which is then relatively aggressive. And that we have six patients here is already quite a high number because it's such a rare disease. And we were lucky because the proteomics experiment was performed on formalin-fixed paraffin embedded tissues. And they can be stored at room temperature for many years. So for this experiment we could use, yeah, we could make use of also tissues that were collected long time ago. And the metastasizing C-SCC is also not so common because most cutaneous squamous cell carcinomas are quite benign, or at least they don't really often metastasize. But in this case we were lucky to get several metastasizing C-SCCs. But these are just a sporadic C-SCC, so there's no genetic defect. It's most often generated by UV light, normal sunlight exposure over a long time, many, many years. And the aim here is to find differentially abundant proteins between these two relatively aggressive C-SCC types, but they have quite a different origin. So I can try to... So we want to have them in a collection, then it's easier to handle in the history. So I click on operations on multiple data sets and then select all, but not these three files and not these three files. So I build a data set list. This is raw files. I hide the original elements and create a list. And then if I click here again, then the boxes go away. And now it looks way nicer. So here's my collection with the 19 raw files. And if I click on them, here they are. And now they are green. So on the European Galaxy server, the upload immediately turns out the right format. So the format has to be thermal.raw. In case this went wrong on your Galaxy instance, you can manually change it by clicking the pencil again and then click on data type and then look for thermal raw and then press change data type. You would need to do this for every of the 19 files then. We can also give the history a name. So this is the MaxQuant MS stats label free training. And now we are ready to look into the MaxQuant tool. So MaxQuant is pretty powerful. It has, it cannot only identify peptides and proteins. It can also quantify them and it allows many different quantification methods. So label free, but also many types of labels are supported. In our case, we can leave pretty much the default parameters. So here we have to choose our input type. It's possible to also load MCML and MZXML. So the open standard formats, but MaxQuant requires them to be in quite a specific type. So it might be a bit tricky to get the right subtype of these XML files. So we will use the raw data in the thermal raw file format. And here it's already pre-selected our FASTA file with the protein sequences. It's already chosen as FASTA file input. We leave the pass rule, which determines how we split up the header of the FASTA file, which parts we keep. We don't have any fractions or PTMs. So we don't need a template. We are quite happy with all these parameters here. So we do an FDR of 1% on peptide spectrum matches and protein level. The protein quantification is performed again in MS stats. So MS stats will only use the peptide level identifications of MaxQuant and then performs a protein quantification or summarization by itself. So here the parameters for the protein level quantification don't matter that much. And here we have now the actual input that we need to choose, because it's possible in a parameter group to specify different parameters for different files. But in our case, we want to have the same parameters for all files. And in order to recognize the collection, we need to click here to choose the dataset collection. And now it says it's not available. That might happen if there is one dataset which was not correctly recognized, but it looks good. So I just reload MaxQuant. So we anyway left everything the same. And it still says we don't have a collection. This one is wrongly recognized. This is interesting. So normally it either works for everything or for nothing. So we change the data type here. And this was right, right, right. This was wrong. This is wrong. This is right, right. Okay, the others seem to be fine. So for whatever reason, there were three datasets that were not recognized correctly. So now I can go back to MaxQuant and see if we have it now. Yeah. And now here it is, the collection number 45. We also leave all these parameters as they are. If you want to learn more on the MaxQuant parameter and their meaning, please check the beginners MaxQuant training. There's a whole section explaining the most important parameters of MaxQuant. So we had that chested with trypsin. So we have selected this. And here we could choose label free quantification. Um, but actually this adds only a normalization step on protein, um, level. And because we anyway work with the peptide level data and MS stats, it does not matter. So we can also leave quantitation method as known. Included into the MaxQuant wrapper is a PDXQC functionality. And this generates an automated QC report with many different and very helpful plots that describe the dataset. And also the, the, um, indirectly the machine, um, the instrument performance. And now in the last step, we need to select an output. There are several options. So from max, uh, from MS stats, we need protein groups and evidence file. And we could hit start now, but the analysis takes several hours. So it's not really worth to click the button because it will only finish, um, many hours later. And if we would like to continue with the training, we already have the PDXQC report downloaded from Synodo, the protein groups and the evidence file. So we can look into them and just assume that this will also be the output of this MaxQuant run. So I open the PDF with this scratch book. So this is an overview about, um, different parameters and how they perform in different files. Um, don't worry too much about red lines here. Um, it's often the cutoffs, um, do not always fit to all types of, um, mass specs and all types of experiments. So there's many different parameters written down that MaxQuant has used. We get several overview figures and this is an important figure. So MaxQuant depends automatically, um, potential contaminant sequences onto the FASTA file. And in this cohort, quite many of these potential contaminant proteins were found and they have, uh, intensity up to 60% of the samples can be contributed to potential contaminants. But it's important to think about what this means and which types of contaminants are in there. And many of the contaminants, um, or many of the potential contaminants, of course, human proteins, um, that the experimenter could, um, yeah, contaminate the sample with such as, um, a skin piece or a hair. And so all the skin proteins are inside the contaminant list, but here we have analyzed human skin and therefore we expect that we have the typical skin proteins in our sample. So in the later analysis, we will assume that we have, um, that the human skin proteins are actually from our sample and not contaminants. So there's a lot of more plots, which you can explore. I will go back. So I close this, um, let's also have a look at the protein output. If we click on the file, we can see that we have 2,635 lines. One line is the header line. So there's, um, 2,634 protein entries in the file. Here's the beginning of the file. We can see that some IDs are already labeled as contaminant. That's why we have the con in front of the Uniprot ID. And now there's a lot of information about peptides and unique peptides, their mass and so on. What we would need to know is, um, yeah, so here are the intensities of the proteins, but we will not really care about them. So we need to have this column here. Every entry that has a plus is considered a potential contaminant. And what MS starts will later do is to remove all the proteins that have here a plus that are potential contaminants. So what we will do next is we will remove the contaminants that are not human, because they actually are contaminants. They are not expected in our sample. And then we will replace the pluses that are remaining here, um, with nothing. And this removes the plus signs from this column. And then MS starts will not recognize that we will still have a few, um, protein IDs that come from, um, contaminant potential contaminants. So this was column 118. And there's also one column in the evidence file. So here we have, um, the feature information. So we have peptide sequences and they have different charges and come from different of the samples. And what I said before was that in the annotation file for MS starts, we need to put the right name. And this here is actually the names that we need to put in. And so the raw file column of the evidence, um, is how the run, the files should be named in the annotation file of MS starts. And then we also look for the contaminant column. So here it is, it's 54, at least here at the top, there are no contaminants. Um, so we need to remember that column 54, we need to, um, replace the pluses with nothing in order to remove them. So we will start, um, by keeping only, um, the human proteins because all non-human proteins are probably contaminants. We will do this with the select tool. So from the select tool, there's many different varieties. It's quite tricky to find the right one. Um, sometimes it's also easier to just look into the different, um, categories because the search is not always working so well. Um, so it would probably be filter and sort or text manipulation. No, it's not here. So the easiest option is, um, if you're on a Galaxy server that supports that feature, you'll click again on the head and go to the training material. And then you can directly click on the tool and it will be selected for you. So I can already copy also the pattern that we use. I click the select tool and here it is. So it's called select lines that match an expression. And first we, um, so we keep only lines of the data set that either contain the word human because these are the human proteins or majority and majority is just there in order that we can keep the header line because in the header line, there's no human word written. And here in the second, in the second column, the header is called majority. So if we filter the protein groups, which is number 24 with the majority option, we keep lines that are either matched to human or to majority. And we do a similar thing with the, um, evidence file. So we also want to remove non-human proteins from the evidence file that are probably contaminants. We do it again with the select tool. Um, I can click again on the training. I can copy the pattern that we're looking for and press the select tool. And this time we use the evidence file and the evidence file contains the word sequence and one of the headlines. Here it is. So here in the first column, it's called sequence. So we can keep the header by also selecting the sequence. And now we need to get, so now our data set has lost several lines of proteins. And these were non-human proteins, um, that came from the contaminant database and they are removed now, but still we have contaminants here in our protein list, but they are only potential contaminants. And here we can already see this is probably a keratin and we would like to keep it. But because this has a plus in the potential contaminant column, we need to remove the plus and this we can do with the replace tool, the replace text in a specific column tool. And we start with the select on data 24, that was the protein group file. And if you remember, we have seen the plus sign potential contaminants in column 118. We are looking for the pattern plus and we replace it with nothing. And this is how we remove it. And we do the same again. We can use the rerun button. So this is the rerun button. With this button, you can first of all see the parameters that you have chosen in the run before, but it also allows you to use the same parameters and the same tool for another run. So I only need to change here the input. We want to have 47 to select on the evidence file and the column already appeared. So 118 does not exist in the evidence file and this was here the contaminants were in column 54. And again, we find the plus and we just remove it by replacing it with nothing. And now we are ready for the MS-STATS analysis. So if you're not sure, if you can remember with just the protein group and with just the evidence, you can also rename it and say this is the manipulated protein group. And this will be the manipulated evidence group, no group, just the manipulated evidence file. And with these files, we are now ready to go already to the statistical analysis. So we look for the MS-STATS tour. There are two different tools. So the MS-STATS tool is for label-free data and MS-STATS TMT is for TMT or iTrack labeled experiments. So here we choose the MS-STATS tool for label-free data. And now we need to choose where our data comes from. And this time the data comes from MaxQuant. We could load directly the evidence file from MaxQuant here. But then we would still have the non-human contaminants inside and would automatically remove our human potential contaminants, which we don't consider as contaminants. And therefore we use the filtered or manipulated evidence file and we use the manipulated protein groups file so that we make sure that we keep the human skin proteins. And now here we select the annotation file and we can here choose that we use the leading razor protein column. So that might be a bit less IDs. Then there are some transformation options. How we can transform the data which is now formatted into an MS-STATS compatible format and here we just select remove the proteins which have only one peptide in charge. The data process options are all good as they are. And here we just add the sample quantification matrix table and we can also add the run level data. Here are some processing options hidden. And for the condition plot we do a zero degree label angle and we only choose one. Yeah, we only generate a QC plot for all proteins. Otherwise we would generate a plot for each protein and we have more than 1000 proteins. So that's too much information which we don't need now. And last we would like to do a group comparison. Yes. And if we select this we need a comparison matrix and here it is. We load the comparison matrix. We leave the outputs. We don't even need a comparison plot which is only helpful if one has more than one comparison. And again we can adjust the plotting options for this volcano plot here. We set a full change of 1.5 as a cutoff and we don't want to display the protein names in the volcano plot. Let's check that we have everything. Yes, this looks good. So we can start Maxquan, MSDOT, sorry. This is MSDOT now. This takes a few minutes so it might be a good point to make a short break. Okay, so let's have a look again at the MSDOT's parameter. So that was all to do the statistical analysis. So the first part is only a conversion of the Maxquan output files into an MSDOT's compatible format. And here it's really important that the annotation file contains the right annotations and the right file names because that's how the metadata in the annotation file is actually mapped to the run names in the Maxquan output. The second part is the data processing. You might have realized that we didn't change any parameter here. So the processing consists of a log 2 transformation, media normalization and feature selection, missing value imputation. And then the feature intensities are summarized with the TMP method into protein intensities. So these outputs all come from the feature and processing level. And only afterwards we decided to compare the two conditions and they were defined in our comparison matrix. So only the comparison result table and the volcano plot are actually showing the actual statistical modeling results. So let's have a look at the MSDOT's output. So first we have a log file. This is just a text file that captures information about data analysis steps, warnings and so on. If you click on one of these, we also see the number of proteins and the number of peptides per proteins that were in this data set. So the process data summarizes or is a list of features. So we have the peptide and the charge state. And for the features we have a lot of data, including the meta data from the annotation file. So the group they belong to and the subject they belong to the run they belong to. And then for each feature we have an intensity and we have an abundance value. And the abundance is the log transformed and normalized intensity value. And in the next step, these features were then summarized with the abundances to generate an overall abundance for each protein in each run. And that's actually the data that we can find in the run level information. Here we have now a summary of protein intensities per run. So here are the intensities and we will now calculate the distribution of the numbers of features per protein and run and see how many features were on average used to quantify a protein. And we will also look into this column with the run identifiers and count how often each unique run entry appears here in order to find out how many features were present in which run. So to do this we use the data mesh tool. So here it is. So this is a really helpful tool to perform summarizations on text files. We want to do it on the run level data and we need to know the column we are interested in. We can still check this here in the small window. So the run name is in column eight. We actually have a header line here. We want to print the header line and we would like to sort the inputs. And now we count with this we just count the number of lines that are now getting summarized. And the other thing we would like to know is the amount of features per protein. So we perform a simple summary statistics again on the run level data. And this time it's column four on the number of features per protein and run. So we change this to C four and run this step. In the meantime, we can explore more of the MS stats output files. So the next file is a QC plot. Only wants to open like this. So this plot shows the protein abundances of all proteins for each sample. And we see that the medians here are pretty on one line. What is a good sign? And we also have a relatively normally distributed data. And this plot you could also generate for each protein. But then the PDF file would have many, many plots. And that's why we have chosen that we would only like to get the summary for all proteins. Then there's the sample quantification matrix. This gives us the quantities for each condition. So for each, for each sample and for each protein. So we have 19 columns here, one column per sample. And we get an intensity or abundance value for each protein. And then the same is here per condition. For each protein, we get a summary of the intensities for each condition. And then the comparison result is the actual statistical table. It contains the protein name, the label. In our case, we only have the comparison metastasized versus RGB. The log two fault change. And here the most important column is the adjusted p-value column. There's also an issue column. And here is an example of a protein that was not at all measured in one condition. And therefore the issue says that the protein is missing in one condition. And later we will filter this table for the adjusted p-value and the log two fault change in order to find the significantly differentially abundant proteins. And the last output file we have here is the log two fault change. And the file we have here is the volcano plot. So here we have plotted a p-value in a way that smaller p-values actually give higher values in this plot. So it's easier to interpret. And on the x-axis, we have the log two fault change. And in the MS-STATS tool, we said that we would like to plot the volcano plot with a fault change cutoff of 1.5. And if we take the log two of it, it's 0.58 and minus 0.58, that's the dashed lines here. And everything that is now in the upper right part and in red are proteins that are up-regulated in metastasized condition. And here in this part are the proteins that are down-regulated. What means that they are up-regulated in the RDB condition. So in the meantime, we have a result for our summary statistics. So this is the summary of the numbers of features per protein and run. So we can see that on Median, there are three features summarized into a protein or on average around five. The data mesh tool gives us an overview now about how many lines the run level data had for each file. And because one line was one feature, this corresponds to the number of features. So we have here the numbers for each file. But we could also visualize it when we click here on this visualize button. And then choose the bar diagram. We can give it a name. So it's a number of features per sample. And here we can choose the data that we want. And we only need to change this here. And now we can interactively browse to see which file is which bar. We can save this and you will find a plot if you go on to use visualizations. That's where your plots are stored. Okay. So now let's continue with this actually statistical result here. So we would like to filter only for proteins that have an adjusted P value below 0.05 and that have a log two fault change above 0.58 or below minus 0.58. And furthermore, we would like to make the protein idea a bit easier to read. So we only keep the actual uniprot accession number and remove this part before and this part afterwards. So we start with this. We do this with a replace tool replace text in a specific column. We would like to use the comparison result and column one. So we want to get rid of this part. But because this pipe sign has also another meaning in this regular expression here, we need to use a backslash to actually find our pipe. And because you want to remove it, we do not replace it with anything. And now we do the same with the right part. So again, we want to have a pipe and then everything that comes afterwards. And again, we want to remove it so we don't need to put anything in here. So next we can already start the filter tool. So when we click here, it's a safe way to find it. We want to run the filter tool on our replaced file here. And we start by filtering for the adjusted p-value below 0.05. So it's column eight below 0.05 and we skip one header line. And now we can directly click the rerun button. We want to use the same tool again, but this time on number 61, which contains only the low p-values. And now we filter for the log 24 change, which is column three. And we only keep the proteins that have a log 24 change above 0.58. And we repeat the procedure. So we do it still on this p-value filter data set, but this time we switch it and look for the down-regulated proteins. And we can now count the number of lines. So we started with 1,297. And we have 123 that have a p-value below 0.05. And we have now eight lines that have a negative four change. So these are the proteins that are higher in the RDB condition. And once this is finished, we know about the proteins that are higher in the metastasized condition. And we will continue with both data sets, and therefore we give them a tag. So when we put a hashtag in front of the tag, we can actually keep the tag. So every file that uses this file is then having this tag here. And here I think we can only do it once it's finished. All right, here it is. Maybe it survives. And you can also use a tag without the hashtag, but in case you use a hashtag, you will find the same tag propagated during your history. And it makes it easier to track the many files that we will generate in the next step. That job is finished. And now we can visualize the results. And we see that the adjusted p-value is below 0.05 for all proteins. But many of the proteins have adjusted p-value of zero because they are missing in one condition. So in one condition, there was no feature identified and quantified for this protein. And also these proteins might be of interest. And so the first follow-up that we do is on these proteins that are missing in one condition. And to obtain them, we will now filter this adjusted p-value and keep only the proteins that have a zero here. So this is still column 8. We can use the filter tool again. We would like to use the metastasized up-regulated proteins first. And we say that column 8 should be exactly zero. We still have a header line. And now we would only like to keep the protein IDs. So we use the cut tool on the last file that we filtered. And we keep only column 1. So the other values are now not important anymore. We just look at the proteins that all have in common that they were not detected in the rdb condition, but in the metastasized condition. And we repeat the same for the rdb data set. So from file 63, which has the rdb tag, we only keep the zero adjusted p-value proteins. And then the next step, I use again the rerun button because it's way faster. And we filter this, we cut the filter data set and keep only the id column. And here we can already see the id column. So it still has a header line. And because we would like to combine now both IDs into one file of one file, we need to remove this header line. And this can be done with the remove beginning tool. So from this rdb id list, we remove the first line. And then we can combine them. And this can be done with the concatenate tool. We concatenate file 65, the metastasized IDs with file 68, which is the rdb IDs that have no header line. So we can directly attach them here. So that was the rdb. There was only the rdb file. There was only one protein that was detected in the rdb condition, but not in the metastasized condition. And now we have removed the beginning. So it's only one line left. And this line is now attached to file 65. So that we should obtain 81 lines now. And these 81 lines contain the protein IDs that we're missing in one condition. And what is interesting now is to see in how many samples each protein was actually found. And to do this, we go back to the sample quantification table from MS stats. There was the sample quantification matrix. And here we have for each protein and each sample, we have an intensity value. And we would like to get these values now for our proteins that turned out to be missing in one condition. And because we have only kept everything between these two pipes, we now also need to extract only these IDs from this matrix in order to make it possible to automatically join the IDs of both files to obtain the quantifications. So we will do the replace step. Where is it? We have done it before. So we will rerun this step just this time on the run level data. And everything else, we can even keep it how it is. So we want to remove this part before the ID and everything afterwards. And we just do it not on the run level on the sample quantification matrix. And after doing this, there's a tool called join, which automatically joins two files according to mutual information in one column. So we can already select a tool. And in this tool, we could even decide if you would like to keep the IDs that are in both tools, if you would like to keep the IDs that are only the first, but not in the second and so on. So there are many conditions which make this tool very powerful. So here we would like to use the sample quantification table. That's the one where we just replaced the protein IDs in order to have the same format as we currently have it in our IDs. And the IDs are in column one. And we would like to join this with our IDs that we have from the metastasized and the RDB file where the IDs are also in column one, which we can verify by looking at the data set once it's finished. And in both files, we should have still a header line. So we say header line, yes. And next or last, what we do is we use a heatmap tool. And we will put in our data into the heatmap. The data will have a header and a row name. So the IDs are in the, after joining the IDs are still in the first column. And that's what we also have here automatically. And we would like to change the output file a bit, so make it a bit bigger. So 15 width and 10 height should be fine. So we need to wait for the concatenate data to finish and then the join tool and the heatmap will automatically start. Okay, so the jobs are finished. We can look at the visualization heatmap. There was only one protein that was present in RDB samples in four out of six and a none of the metastasized samples. And there were many proteins that were not at all detected in RDB, but in several metastasized samples. But there's only one protein that is actually present in each metastasized sample and in none of the RDB samples. And this unibroad ID corresponds to collagen seven, which is expected to be absent in the RDB patients that have the genetic disease. And this shows that looking into proteins that are missing in one condition is actually also an important step. One might want to apply further filter criteria because there's for example protein that is here a few proteins that are only present in four out of the metastasized samples. So they might not be of interest, but others that are more frequently found could actually be interesting proteins. And that was the part on the missing proteins. Now we will switch to the proteins that were not missing. We will continue with the significant proteins that were not missing in one condition. And to do so, we go back to the first filtering step here. So we already had a filtering step to get only the proteins with p-values. Below 0.05. And then we filtered for the full change cutoff to find up and down regulated proteins. And we have seen that these proteins have an adjusted p-value of zero when the protein was missing in one condition. But now we are interested in the proteins that have a p-value above zero and below 0.05. So we will continue with the file 62 and then afterwards with 63 and filter it for a p-value. So we will filter 62 the metastasized upregulated file in column eight. And we would like to have a p-value above zero. And because we have already filtered for below 0.05, this should now give us the proteins that are significant and not missing in one condition. And we repeat exactly the same for the rdb upregulated proteins. And then we only keep the id column. So we use the cut tool. We are now here. We follow up here. We find differentially abundant proteins. We use the cut tool and we keep column one. And because it's exactly the same procedure, we can actually use the multiple data set here and select both files that we have just filtered for the p-value between zero and 0.05. And from this we cut column one to obtain the protein IDs. And here we can already see how many proteins we have. So seven lines means we have six proteins that are upregulated in rdb. And we have 36 that are upregulated in metastasized csc. And now we would also like to visualize their quantities. So we will again match the IDs with the sample quantification matrix. But this time we do it separately for rdb metastasized. So we don't concatenate the IDs and do only one heat map. But we repeat the procedure twice. So here. So we would like to have the sample quantification matrix. But we had to replace it before. So it was file 70 where we had the correct IDs. And then we had the quantities for each sample. And that's column one. And we should be able to do the same trick again because we do it for rdb and metastasized files. We use both files here and we will obtain once the joint data for metastasized and quantification sample and then once for rdb and the quantification sample. And the files all have a header line. And we need to select a column. It's column one for both files. And this time we use a different heat map tool just because it looks nicer. So it's a heat map tool. And we can do it again in parallel for both files. Now not if we want to give it a name. Then we do it separately. So if you want to use a plot title this would be the upregulated proteins in metastasized cscc. You can have a look here. Column one and then the quantities. We don't want to use clustering. And we would like to scale the data. But now we can use the rerun button. And we just need to change the input file to the rdb file. And we need to change the plot title to rdb. Okay. But now we are still at the level of the protein IDs and in order to have more meaningful protein IDs we would like to know the protein names. And this there are many different steps how we could do it. So there's a uniprot tool in Galaxy. The uniprot ID mapping and retrieval. And here we can put in our metastasized joint tool use the first column which is the ID column. And then retrieve as a faster file the uniprot entries. And we repeat the same for the rdb file column one. And because we now have a faster file it is quite tricky to read it because there's all the amino acid sequences. So we use as a last step the faster to tabular converter. We can convert again both in parallel. And we split up the header into two parts. And that gives us now a table format with the with the protein names of the up and down regulated proteins. In the meantime we can inspect the heat map. One heat map is at least finished. So this is the heat map for the proteins up regulated. And metastasized csdc and we clearly see here that they have higher intensities than in the rdb file. And we will see the opposite here once it is finished. And here's already one result. So this is now the proteins up regulated in metastasized csdc. And we have now split the faster file into this ID again. But then here in the second column we can read the actual protein name. And in the third column comes the sequence with the amino acids. And here's for example vitronectin that was also found in the original study. But in the original study the max quant parameters were different and a different statistical approach was used. And we see several histones here and also RNA related proteins. Okay, so we will just need to wait to see the results. So if you want to analyze your own data with msdats, the really crucial part is to set up the annotation file correctly. And because this is the data that msdats uses to decide on how to fit the linear models to the data. And then as you have seen now there's many text manipulation tools in Galaxy. So everything that you can do in Excel is also doable in Galaxy. The most complicated part is probably to know the exact name of the tool. But we have already used a lot of tools here. So you know already some of them. Now the tools are finished. We can look at the heat map here. And we see these are the proteins upregulated in rdb condition and they have higher intensities than in the metastasized condition. And now we also have the tabular file with the protein names. And we find two collagen here in the list of proteins that are higher abundant in the rdb condition than in the metastasized condition. And that could be explained by the fact that when collagen 7 is missing, there's a compensation effect and other collagen become upregulated. And actually for the collagen 14 protein there were also immunofluorescence staining stun and the original publication which confirmed that the abundance of collagen 14 is way higher in rdb cscc than in metastasized cscc. So that was the training. I hope you have learned something and you will be able to repeat the training on the data sets and maybe in the future also on your own data. Thank you very much for joining this video. Hands on demonstration.


Good morning everybody. Good morning. Do you see Joe's talk last? You can see like all the weird stuff that happens in the background just to get this thing produced. It's pretty crazy. Okay, I want to introduce our next speaker, Bruce Schneier. Many of you I bet subscribed to Cryptogram or Rita's website or you see him quoted like maybe four or five times a day in various publications.  I'm barely functional in one. You're flying around speaking everywhere, writing everywhere, writing. He reviews restaurants and writes food review books. He has like a whole secret side to him. He's a foodie in case you didn't know that. So he's got this whole foodie thing. I don't have enough time just to watch reruns of Lost. Okay, so we're going to kick it off. Bruce is going to try a new format. It's just going to be questions and answers. It'll last as long as you can keep coming up with interesting questions. Nothing stupid, please. He's a bright guy and we don't want to slow down his brain. We've got to keep it running in top shape. We're all counting on him. I want to introduce Bruce and then we've got two microphones up here. Don't use wireless microphones at DEF CON in case you... We're wondering why. So if you can't yell, just walk up and use one of these microphones so then everybody else can hear the question. Otherwise he'll just repeat the question. Oh, and don't forget to turn off your phone because it's rude.  So Jeff, the trick is don't watch television. And I got secret lies I haven't even used yet. So hi. What? Hi. Who are you? Don't tell me. There are a lot of you and only one of me. That's actually probably best, otherwise it would be a lousy format. So hi. Are there any questions? How did I come to Vegas? Yes. Actually, he's asking about TSA. I don't know if people know. Actually, who here reads my blog? So maybe about half of you. Maybe a third. Who here gets cryptogram? The rest of you should. Cryptogram is really where I publish all of my essays and writings and such. It comes once a month in your email. If you want to sort of know what I'm doing and writing, that's the place to get it. And over the past week, I've been publishing an interview with Kip Hawley. Kip Hawley is the head of the TSA. His title is TSA Administrator, which I kind of like. I kind of like that government understatement of title, like the Secretary of Defense. You think he takes the notes at the meeting. So he's a TSA Administrator. And in the months of May and June, we did an interview over email. And a lot of topics. And I got to ask him some really good questions. He gave some answers that were at the same time surprisingly forthcoming and impressively evasive. And I think the interview is worth reading. The last installment published today, and then there's a link on my blog to the entire thing. And that'll be in cryptogram next month. So yes, I fly. I still fly. Actually, I flew without ID to get here. There's a paper in Minneapolis is doing an article on me and security and airline security. And they wanted to sort of get as sort of a human interest thing, me flying without ID. And if people don't know this, I mean the TSA spends a lot of time convincing you that you need an ID to fly, which is of course nonsense because lots of people lose IDs. And for the TSA to say, stay, well, you've got your wallet stolen here in Vegas and you have to stay here for another 25 days before the state of whatever mails you replace with an ID. It's just nonsense. It'll never happen. So you can, and I did this, and it's written up on the web in lots of places, walk up to the airline counter, say, hi, I'm Bruce Schneier. I lost my ID. You actually say you're somebody else if you know they're flying, by the way, because no ID, right? They will print you the boarding pass. They'll write no ID on it.  And then they put you through special screening, which in my case meant that you waited 10 minutes while I found somebody to do it. Then they put you through security. They wandered me and they ran that explosive trace detector swab over my bags. So that was it. So if you're on the no fly list, right, if you're so dangerous you can't fly, just lose your ID and fly on somebody else's name. If you don't have a ticket and know somebody else's itinerary, just get to the airport early and then fly to my friend's earlier flight so you're gone by the time they get there. You know, I'm just not impressed here.  And he answers them more or less. So thank you for that question. Yes, so you're up front so you get to, I get to hear you. Those in the back, you can either, you know, sort of walk forward or shout or write out a piece of paper and pass it forward or, you know, make up some other way. If you've got a megaphone, yes. This is a question I'm more than about the TSA if they realize how pissed we are at them. You know, it's interesting. And I try to write about this. You know, security is in a sense a negotiation. You know, what are you going to get versus what are you going to give up? And the important thing about negotiating for security is to understand where to negotiate.  When you walk through airport security, there's no negotiating there. The TSA employees are really just cogs in a machine. I mean, you can think of them as parts of the security system. They are not decision makers. They are only autonomous in the narrowly defined ways they're allowed to be.  I shouldn't have to do this.  And I think we see that in the frustration. That when you see them being arbitrary or nasty or spiteful, I think a lot of it is their own frustration of being stuck in a system that isn't working, that isn't satisfying anybody's needs. And they can't do anything about it. They can't change it. They can't complain.  And also, it's a stressful job. It's very similar.  One of the most, one of the toughest jobs here in Vegas is being a dealer. Being a dealer is incredibly boring. You have no idea how mind-bogglingly, numbingly boring it is to watch people gamble. And you have to pay attention, right, because they might cheat. And there's lots of casino cheats. I mean, I've given talks in Hacking Las Vegas. And that's why when you see them rotate out so fast, even at a craft's table where there are five people, they'll switch positions very frequently. And that's to keep them moving, to give them something different to do, to keep them alert. Because otherwise, they basically fall asleep in place doing their job. And they do it as a robot, and they don't notice anything. And being a TSA screener is kind of like that. You know, look at that, and you'll see if you go through the checkpoints. Look how quickly they rotate through the jobs of, you know, walking the person through the metal detector and checking the lights up on top. Or screening the bags. Or doing some of the special processing, the additional screening. Or, you know, the guy who sits in front of the checkpoints with a box of baggies and says, do you need a baggie for your liquids? I mean, this is a high-played government employee giving away ziplock bags.  And you just look at these jobs. These jobs are mind-numbingly boring. And yet, if they're not alert, what happens is what we see happens. And you see surveys of, you know, 70% of knives go through and 50% of guns go through. I saw a survey with 90% go through. And you wonder why. This is why. So, you know, as much as I don't like it, as much as I think we deserve better, I can really understand some of the shitty treatment we get. Because, man, is that a hard job. Let's have a non-TSA. I don't have a crypto question. Will VPN ever go away? No, because we all want to access remotely. I mean, security's not driven by security. It's driven by what we want to do. And one of the things we learn is that the stuff we build, we're stuck with forever. So, no, I mean, these protocols aren't going anywhere. I mean, you'll see better ones. Because the nice thing about a VPN protocol is you don't have to worry about too much about compatibility. If your company decides to use proprietary VPN protocol X, you can just do it. Right? Everyone gets the client and you enforce it to work. So, you know, that's a little better than 802.11, right? Where if I have now proprietary wireless on my computer, suddenly I can't log in anywhere. And being able to log in in airports and Starbucks and various companies is part of the protocol. But, no, there's such a need, especially now that more and more companies are going virtual. Someone asked me about hash functions. Anyone know what's going on about hash functions? All right. Well, that's not the question, but that's another question. A question is about SHA-1. SHA-1 is the NSA... This is actually kind of fun because it's NSA stuff. And breaking NSA equipment's kind of like breaking alien technology, so you'll kind of feel kind of cool about it. Right?  It arrives on your planet sort of out of nowhere. And the manual's written in Klingon. You can't read it. And, you know, you sort of try to reverse engineer what these people could be thinking. Because you have no idea, because they're basically alien mentality. And, you know, you try to figure out what the NSA knows. So, SHA was an NSA-developed algorithm over a decade ago. SHA-1 was a replacement. And again, alien technology.  And we all felt pretty good when we found the vulnerability, right? Because once we knew to look, we started paying attention. Okay, what changed in Shah and SHA-1? SHA-1 is a good hash function. It's been around for a long time. Its main limitation is its length. It's 160 bits, which means its security is 80 bits. Don't ask me, just look that up. But you only get half the bits in security, basically. And it survived pretty well. Over the past few years, there's a phenomenal researcher out of China who has been basically beating at SHA-1 slowly. And she's finally got to the point where she has better than brute force attacks. You know, not yet feasible.    And that's the reason, this is the question I actually thought you'd ask. We're going to have a new hash function. I don't know if people remember AES. AES is the Advanced Encryption Standard. Back in the GADs, late 90s, I forget the year. NIST, which was formerly National Bureau of Standards, people who actually gave us DES back in the 70s, said we're going to have a competition for a new encryption algorithm. And it was great fun. I mean, it's basically the cryptographer's demolition derby. We all wrote our algorithms, put them in the center of the ring, beat on each other, and the last one left standing one. It didn't, you know, it worked out that way mostly. At the end, there were five good algorithms, and one of them, mine wasn't the one that won. The one that won, I'm happy to use, and I think it's damn good. And that's Ridingdahl, which is now called AES, Advanced Encryption Standard. So a couple of years ago, NIST decided they're going to do the same thing with the hash functions. We're going to have the AHS, the Advanced Hashing Standard. And it's the same deal. We all get in our corners and develop our little hash algorithms. We submit them to NIST. They all become public. We spend a couple of years beating up on each other, and then NIST picks one. So that's going on now. Submissions are due next year, I think, in the fall. And it's, we don't really know who's working on a hash standard. If it's the encryption algorithm, we got a submission out of Japan. We got one out of, a few of the security companies submitted one, IBM submitted one. It went out of Australia. It went out of Korea. I think there were 15 total. Now, when we sort of look around, he was working on a hash algorithm. We might have 20 or 30. He might have an enormous number. The two fish team, which is my group, has reconstructed. Reconstituted, sorry.  We don't have a name yet, so email me suggestions. And this is likely to be, I mean, aside from this being a really good way to develop a security standard, right, not by committee. You develop stuff by committee, you tend to get the lowest common denominator. But as separate groups develop something, and then you choose based on peer review and what people want. That's a way better approach, designing a standard. For encryption, when we did the AES, the encryption algorithm, it was a phenomenal learning experience as cryptographers. And it was interesting, I didn't expect it. But we as a community learned an enormous amount about how block ciphers work by going through this design process. I guess this is another NSA secret, you know, we now know, that actually having these sorts, I mean, because we all did a lot of analysis and very little design. And the design was sort of ad hoc and you'd come up with something and you publish it and that'd be done. But this sort of multi-year, multi-stage, multi-review design process I think taught us all an enormous amount about block cipher design and about crypto design in general. So I would expect some really high quality hashing submissions.  It is frightfully slow. I mean, it's just plods. You look at some of the operations and they are slow, slow, slow. So, you know, when I go into designing these algorithms, what I look at is, you know, the basic unit of, the basic unit I'm paying for my security is a clock cycle. What can I do in this clock cycle that will make it more secure? And we look at, you know, we're looking at the top Intel chips. The top Intel chips have two processors. What can we do to keep both of them busy constantly that will make this more secure? We have a free operation here. What can we do here to make it more secure? I try to build the maximum security in the fewest clock cycles. This is a development style. It just didn't see 10 years ago. I mean, you were looking more at primitive operations in a more graphical sense and you wouldn't really differentiate between operations that took one clock cycle or even operations that are free. I mean, there are, there's stuff you can do for free in some of these processes. They don't take anything. You know, when for AES, Mars, which was the IBM submission, had a multiply, 32-bit multiply, which was a great thing. I mean, it had a lot of good security properties, but it was like a dozen clock cycles. And the question to ask is not, you know, is it good to multiply, but is multiplying better than 12 clock cycles worse of this, of, you know, of rotating or XORs or these other things? Right, it was 12 clock cycles and it's stalled some of the pipeline. So there's going to be really exciting stuff in cryptography coming up. I mean, you're not going to see the results of this for maybe four or five years. And that's good, right? I mean, and the reason to do it now is that there's no emergency, right? SHA-1 is still fine to get back to your original question. There's no security problems with using SHA-1 for any applications. But, you know, in the future, we don't know. And the way to do the transition is slowly, deliberately, and properly. And not, oh my god, there's a fire, we must exit. That was a joke, don't leave. Good.  Right, so that's hash functions. There's a hand, the menu. Okay, good, now I'm getting some hands. Let's go. Can you speak loud enough?  Come to the light. Don't come too much closer. I assume people didn't hear that. The question was about privacy.  What's going to happen in the future? This is something that's really important. And actually, this might be the most important thing I'm going to say here. Because here is a point where we really get to make decisions about what society we live in and how the future looks and how the future works. And people were at the conference and I have pioneered what I talked about this. Basically, data is the pollution problem of the information age. Right, all processes produce it. Every computerized process produces data. Naturally, it's a natural byproduct of basically everything we do now. And it stays around. And how we use it, how we store it, how we dispose of it is fundamental to how we live in the future. And these are the questions that 100 years from now, we as a society will be judged by. Just like we look back 100 years ago at the way, you know, the industrial age dealt with pollution. And we right now are finally struggling with some really hard issues about pollution. Data's going to be just like that, and it is. So the fundamental problem is that everything we do produces data. When you hear the big brother talk, this is where Orwell got completely wrong. It's not about deliberate surveillance. You don't need to do that. No one needs to follow you. No one needs to watch you. The things you do naturally produce data about who you are, where you are, what you're doing, who you're doing it with. And it's cell phones everything we do on the internet. And it's every financial transaction we make. And it's all the cameras. It's absolutely everything. And this has been true for a few decades. But what's changing is data storage drops to free. Data processing drops to free. So everything now is saved. Amazon can save not only the list of the books you buy, but the books you look at and you don't buy. In some ways, we like that. I love that when Amazon puts up suggestions of books I might want based on my purchasing habits. I see books I do want. I didn't know they existed before. That's a great service. What we don't like is Amazon selling that data to Choice Point or some big data broker. It gets resold to telemarketers and then who knows where it is. We tend to like the primary uses of data, not the secondary uses. We like an itemized phone bill, but mailing a copy to the FBI squicks us the wrong way. It's that sort of thing. And we as a society really get to make a choice here. We get to decide. And unfortunately what seems to be winning is sort of the libertarian the market will sort it out. Because the market won't sort it out. Because you for whom the data is about no say in this market according to the laws of the country, data about you isn't owned by you. So when you make a phone call, you can record who you call and how long you talk. The phone company can. The phone company owns the data that they collect. You don't own it even though it's about you. They collected it. It happened in a transaction with you in the phone company. They own it. They can resell it. They can do whatever they want with it. So you're going to deal with that. And I think it's really important we do deal with this. Because our notions of privacy are changing. Our notions of liberty it's very fundamental to who we are. This happens to me and it probably happens to some of you at least. You're on the phone with somebody and you're talking about something. Maybe you're talking about terrorism. Or some sensitive government thing. And one of you says the FBI isn't listening. And it's a joke and you know it's a joke.  Just the thought that you might be being observed changes what you do. The fact that I know this conversation here is being recorded is changing things I say and don't say. If this wasn't being recorded there were things I would have said in the early interview that I'm not saying because I don't want it recorded. So we really hear in the early years of building information age and despite some of you in the crowd grew up with the internet these are still the early years of the information age we get to decide what it gets to look like. And these aren't technological limitations. These will be legal and social limitations. That we will say we will have laws about data collection, data retention, data ownership, data accuracy, data audit or will not. We'll have laws about police access to data. The Fourth Amendment which protects those of us who are Americans against the government grabbing our data without a warrant without due process doesn't work for data that you don't control. So it's written in the Constitution to secure our persons and papers. But our papers aren't in our homes anymore. Our papers are at our ISP. Our papers are held by Google. Our papers are at our phone company. It's our SMS messages. Those are our papers. But we don't have control of them anymore. So the Fourth Amendment doesn't apply. And recently there's been some court cases in here. This is where we as a society get to decide. And really everyone here should be thinking about this. Because these are problems that in two, three decades we will be looking back and saying why didn't we think of this then? And some of them are hard. Medical data is incredibly personal and private. Yet is enormously valuable in the aggregate. Wouldn't it be neat if everybody's medical data was anonymized and available everywhere? To do massive lifestyle studies or massive follow on drug studies or environmental studies to have that data. What are the effects of living near power lines? Let's pull the health records of everybody who's ever lived near a power line. That'd be neat. How do you do that? How do you maintain the personal privacy of your own health and at the same time get as much of the societal benefit of the aggregate data? This is hard. This isn't easy. Well, we're going to have to do it. Good question, by the way, wherever you went. I'm going to go to this side. There's a question over there which I can barely see. Yes, you.  You must be louder. The question was, is society really going to decide? The answer unfortunately is no. We'd like it if society would decide. What's going to happen is we would get what happens naturally. There's a sort of fetish belief that if you leave things alone you get the best result. And some of it comes from libertarianism.  I'm hoping people here have gotten beyond that. Larry Lessig's written an enormous amount about this. That a lot of people believe that the myth of the internet is it was left alone and it's why it's so cool. It's because it wasn't left alone as why it's so cool. It was forced in a certain way. Probably not. I mean unfortunately the power is in power and I don't mean this partisan like. I mean this sort of in general. Corporations like this free and open access to our data. And they all do. So for us to get these sorts of privacy laws and you can see some of them are in Europe and are pretty damn good and some they don't have. It's going to take the kind of intervention but in a lot of ways that's the way pollution went. You didn't see pollution laws until things got really bad. And even then, I mean even today it's kicking and screaming. To get any sort of laws that protect the commons of the environment our health, any of those sorts of things. So likely we make changes around the edges. I got a call and I'm not going to name the name of a corporate CIO but I'm a major phone company who's asking me about what they can do for privacy. And a lot of things I said they couldn't do but some stuff they could. And that's where you play. You play around the edges.  But I expect or we can have any privacy left we are but it's going to be different. The privacy we get out of the Web 2.0 sites, the Facebooks and the Myspaces and the live journals are very different sorts of privacy than the ones our parents are used to. So you're going to see some changing notions of privacy? And this is important. A lot of the legal rules around privacy use the secret metaphor. So you have a basic right to privacy in this country. But it assumes that privacy equals secrecy. So if somebody knows one of your secrets you lose your right to claim it's private because somebody knows. That actually doesn't make much sense. It really makes no sense now. Because the argument is well you put it in an email and the ISP knows that therefore it's not private. But sir that's everything. Well I guess you have no privacy then. So it just doesn't really work. And this is another example of how the old way of thinking is sort of crashing against the new technology giving us these results that make no social sense. So unfortunately that's what's more likely to see in the near term. And we're seeing some of it now. Results that make no social sense. Anything else? Let's go to over there. At the end. Another techie question about AES about timing attacks. Who doesn't know what a timing attack is? See if I have to do it a little tutorial.  Anybody? Wow. No. Come on. The general class of and the phrase I coined for these are side channel attacks. These are attacks against in general crypto systems or security systems that rely on additional information than just the inputs and outputs. Traditional cryptography is you have the inputs and outputs and you have the mechanism in the black box and you try to break it. Side channels are things like timing attacks power attacks. Paul Cotcher did some phenomenal work in differential power analysis. Radiation. I did a paper on hamming weight attacks. I've seen papers on cash attacks that if you're dealing in a processor with multiple processes and if you're in another process I can just watch when the cash gets dumped you learn information. These are all side channel attacks. They are incredibly devastating. There aren't algorithms that survive these. And while you can in some cases build the implementation base much more than mathematical base build implementations that protect against some of them and again Paul Cotcher does some great work in this. He's cryptography.com if you want to see his site. He has a bunch of papers on this. It's hard. They often require more access. You need physical access. They're very powerful against smart card chips for example. Because you have the chip. If your badge had a cryptographic protection the side channel attacks would be very powerful against those because you've got the badge in your hand. They work less well against I have an encrypted file on my hard drive and you want to break it. Because you don't have access to the process. But they are very very devastating attacks. Now on the plus they are kind of high tech attacks. So your average criminal won't be able to do them but you're more well funded criminal will. Anyone want to ask me about voting? Voting has been in the news this week. This is actually some important stuff going on. The California yes? You want to ask about voting? Favorite voting system is optical scan. By far the best. Because what we want out of a good voting system is this is hard. Especially hard in this country because we expect the results before we go to sleep. It's not like India where you can take two weeks to decide who wins. We will not stand for that in this country. And yet you could probably do better if you allowed that. So voting has to be accurate and fast and anonymous. All the things we need. Optical scan and people don't know the machine. We have them in Minnesota. It's what we use. It's like those old standardized tests from the 70s. You'd fill in ovals on a piece of paper for your vote and then you'd put them through an optical scan reader which would do the tally. So it's night. You have the piece of paper and in my state it's both sides. You put it through the reader and the reader tells you at this point whether your vote was complete. So if you under voted, if you voted in fewer elections that are on the ballot, your ballot is going to have 30-40 different elections on it. Another reason voting is so complicated. We vote for so many things at once. All different districts. So if you under voted, it'll beep. If you over voted, you voted twice, more than you should, it'll beep. And then you have the opportunity to take your paper back or say, never mind, let it go through. And then the tally happens in the machine and the paper drops into basically a safe. So what this does, it gives you automatic auditing. You have a paper audit trail incredibly important and you have the fast tally. So this system really is the best one out there. And if you have any poll in recommending voting systems for your local county, city, state, whatever government bodies selects them, optical scans and the machines to get. So that's your question. What's gone on this past week is we're starting to see results from the California voting audit. State of California did a real live review of three voting systems. The Sequoia system and the SSS system and I forget the third. And this was a real review. They got source code. They were able to do red team you know, just you know, let's get the machines and see what we can do types of analyses. The source code review I think showed up this morning or maybe last night at the California site. I blogged about it this morning. And that blazes a real good post. He was one of the teams. He was the head of the team that reviewed the Sequoia code. Really, they basically broke all of them. The other news is there's a new report from Florida on a, I think it's also a Sequoia system where they did a source code review. So we're starting to see more of these. The annoying thing and I wrote about this from the California review is the conditions were really onerous. I mean, you had one month to do the review. The companies you know, dragged their feet and releasing the source code so people had less time. They weren't forthcoming. And the state really didn't push. So while, you know, and what I said is a lot of ways were lucky that these voting machines sucked so bad that the researchers found so many vulnerabilities in the short time they had. You know, next time you won't get so lucky. The bad guys aren't going to be limited to a month for three weeks. So while these are good steps, they're really only small steps. Brennan Center also released a new voting report on post-election auditing. So there's a lot of good stuff happening in voting security right now. This is also a tough problem.  It's not the way to think of it. I mean, voting is something that's incredibly important for like one week every two years. So it's really hard to get people trained on systems, getting familiar with them. I mean, user interface is extremely important and very different because you can't build a system with a learning curve either for you, the voters or for the election officials.  The person behind the desk has no more experience than you do. Actually, being an election judge is fun and it's another thing you can do if you're on election day, besides being an election judge in your in your county, in your city. You sort of see how this stuff works. You'd be kind of amazed. So that's what's going on in voting. All right, let's get a question. There's a question right there. I can't hear a word you're saying. This is actually a really big room. He's getting the microphone. I know that would answer another question while he's walking up here. Are these algorithms functionally better than they were five to ten years ago given the increase in the average computing power? Will there be some point in the future where we will reach a point of diminishing returns? So he read that. So we don't even know it's his question. Okay. It's his handwriting. So now we know. Encryption, you know, in computer security most of it favors the defender. Right? I assume that the DefCon crowd knows this. But we don't have to explain this to a lot of corporate security audiences. Right? I mean, you know, the defender has to defend everything and the attacker has to find one way in. Right? All my source code has to be secure. You have to find one more for overflow. The odds really favor the attacker in computer network security.  I mean, we break into everything. And so there's a natural flow to, towards the attack, towards the attacker. Cryptography is the exception. In cryptography the math favors the defender. I mean, think about it very basically. If I add one bit to my key I double my key space. Right? It makes the attacker do twice as much work to break the key. And, but for the defender I have to do, you know, one-enth additional encryption work. So 64-bit or 65-bit. Right? So if I take my key as I have a 64-bit key and I double a key 128-bits I have to do twice as much encryption work. More or less. Maybe four times. But there's some small linear increase. For the attacker, the increase is 2 to the 64th in work. Right? So you get an enormous benefit. And when you look at AES with a 128-bit key I mean brute-forcing it is more computing power than there is on the planet. In the foreseeable future. So it's not like you reach the point of diminishing returns. It's that you reach the point of who cares. Right? His question was, are encryption algorithms more secure today than they were five years ago? And the answer is who cares? Right? Because the encryption algorithms are so much more secure than every other aspect of the encryption system that that's not where you're going to break it. I mean there's been a couple of articles recently that are coming up in court cases on how the FBI breaks into PGP encrypted traffic. And it's kind of interesting. They don't break PGP. They install a keyboard logger. Right?  Or look up access data. Access data is a company in Utah. It's a really cool company. And they build cracking software for pretty much every encryption algorithm.   They don't break the cryptography. They don't brute force the key. They try the keys in most likely key order. You know something? In their list of the top 100,000 possible keys you know, break a good quarter of every instance out there. You know, their record is basically 60%. What they do, and this is phenomenal that if they get, you know, usually they do a lot of sellouts at the FBI. And so what happens is the FBI you know, comes into your house and they take your hard drive and it's got encrypted files on them and you're of course an unsavory character and the FBI wants to read them. So they go to access data and say can you read these?  Yeah, and guess why? I mean, they're running windows and windows you know, shove stuff into cash and it ends up at the tail end of some MS Word file and four years later it's still sitting there and what it was was the screen image of what your key was. Or something else. And they have an enormous success doing that. So here you have a really good encryption product being broken by, you know, the memory management features of the OS.  But that's what happens. So the key length is the wrong discussion to have. You know, what I like to say, it's like you know, we're putting a huge stake in the ground hoping the enemy runs right into it. And we can argue whether the stake should be a mile tall and a mile and a half tall, but honestly the enemy is going to walk around the stake. Right? The crypto is so much better. And whether it's the user interface, whether it's the key management, whether it's the computer security, the network security that something else is going to break first. You know, whatever I do analysis of encryption products, I pretty much break all of them. I never break the cryptography. Even if the cryptography sucks, it's not worth bothering.  And that's annoying, right? Because what I said was that cryptography favors the defender. Unfortunately, you know, the weak links is where it favors the attacker. And that's why you see the FBI reading PGP traffic all the time. PGP is great. But the FBI just goes around it. Yeah, I mean, I see because I do a pretty complex encryption management system on my laptop.   You know, how often do I have to scrub the free space of my hard drive to get it out of there? You know, what works? But these are the things I'm thinking about. I use, the question about my process is my laptop. Actually somebody asked me this in email instead of I should do a blog post on this. So I will. But basically I use PGP disc. I mean, encryption is all about trust. You must trust the people and I know the people at PGP, I know John Callis and the guys there. So I trust him and his team. And I basically divide my hard drive into the things I need immediately into the long-term storage. You know, the only thing I'm not secure against is someone grabs my laptop out of my hands while I'm using it. It doesn't have that sort of on-the-fly array stuff or encrypt stuff.  But I set tiers of longer-term and short-term storage stuff I need, let's say once a week. I encrypt with another key and just sits in some, you know, back file somewhere. Encrypted zip archives. And I only pull them out when I need them. The stuff I use every day, you know, my email and the files I'm working on is, you know, PGP has sort of other options to make that work. And I use that. The best, of course, best security for your laptop is not don't put your files on it.  You know, this is the problem of ever smaller devices. You know, we are keeping more and more sensitive stuff on ever smaller and easier to lose things. Our cell phones might have our calendar for the last five years, our entire contacts list, all of our email. You know, on a little cell phone. Right, our little USB, some of us have USB tokens that have our entire desktop.  That's sort of very valuable and easy to lose. So I like to see encryption on these smaller things. I have nothing good for my Trio, which is a POMOS. I try to do that by just not keeping a lot of stuff on it. But I haven't seen really any good encryption thing. My laptop, I run Windows. Just, you know, and yeah, yeah, yeah, but that's where I get my tech support. He's taking notes. This is great. Oh, he's allowed to, he's the presser. Now everybody's going to know. That's even better. Oh, I want to see. Who's got a good question? Like a good one. Nah, I don't want to do another crypto question. You'll be judged by your peers on the quality of your question. Okay, cyber warfare is quick. There's a really good essay on cyber warfare on my website from Cryptogram. It's too long to do, you know, now. One of the nice things about writing Cryptogram is I can sort of summarize what I think and never speak of it again. I'll probably never answer this in Cryptor Hard Drive question again, because I'm going to write about in the next couple of weeks and then I'll just make you read it. So cyber warfare, I'm proud of the essay. It's a good one and find it and read it.  It's just, that's sort of where it all is. Quantum computing is also on there. Oh, this is good. All right, we're going to do it. Wow. Security vendors protecting us from people they accept money from. I wrote about that and this is sort of the question is, will your antivirus software ever tell you about the FBI trojan on your computer? You kind of want to know that, don't you? Right? Maybe you should buy your antivirus software from the guys in Finland, because they're less than I don't know. Yeah, that's a big one, right? And we saw that, I wrote about that with the Sony case, when the Sony built that trojan and Symantec and all the others basically were told to ignore it and they did. So there's a whole lot of stuff going on there. All right, I didn't know what it was. I think they did. It was kind of hard to you couldn't get straight answers out of them of whether they actually colluded on purpose or by accident. It seemed like collusion on purpose. All right, I got time for one more question. There was a woman I heard back there. Say the word again? Virtualization. What do you want me to know about it? The security implications? I mean, it's like anything else.  As soon as you build an extra complexity, you have to secure it. And it's hard to do.  So before you leave, I want to remind you again, if you don't get cryptogram, go to shinire.com, sign up or get in blog form. The man this goes to absolutely nobody, I built it so I don't even get to see it. I'm sorry. I doubt it. No, no, no. So supposedly I was told there's going to be a Q&A session after my Q&A session which seems redundant. But maybe I'll ask you questions or something. And I don't know where it is, but I was told a guy with a hat would take me there. Now this is entertaining. Enjoy your conference.


What's up everyone? We are back with another snack crate taste test. This is one of the best things that I've ever ordered on the channel that we're continuing to do. I'm not sure how much farther we're gonna continue this. I think we might take a little bit of a break and then come back hopefully near the ends of fall or something like that. But these are amazing. It is in Canada. So normally getting like munch packs or other taste like mystery boxes like this where it's like a taste test for a video. Normally are expensive especially with shipping if it's coming from the United States. This is shipped from within Canada. So it's free shipping since I am in Canada. And this only costs $25 Canadian for roughly five to six snacks.  Very generous size. So we absolutely love these. As you guys seen on our previous videos the last one was in Belgium. And that was just an amazing box. Everything was delicious in it, this one here. I'm not really sure what to expect because the country we're doing today is Columbia. Ooh, okay. So I'm really excited but also I'm not familiar with the snacks that they have there. So I'm not sure if it's spicy stuff or different stuff that we're just not used to. So it's definitely gonna be very exciting. So unlike previous boxes too because SnackCrate is getting so many orders they used to be on time but now they're a little bit backed up on their orders. So this one came late and then the one after that we ordered for the month after that that we're gonna be doing in the next video came at the same time or roughly the same time. So if you do order and you don't get your order right away just give it some time because they have a lot of orders they're shipping out just because of their great prices. So we're gonna open it up here. And this time that's weird. The last one it gave us a little card that said everything's gonna be moved online. This time they put it inside. So that's a little bit different. It just says some of the snacks that you might receive and kind of a little description of them type thing. So that's pretty cool but looking in this here this is why I love SnackCrates. Two chips right on top. Usually with MunchPak it was very rare to get like even one chip in it and usually it wasn't that great. So this one here is gonna have two. So I'll let you pick the first snack here. It looks really good so far. Very good, yes. Good first impression. Yeah. Okay, we'll go with this one here. This one here. Margarita, but it looks like Lay's. So it could be a Lay's company in Columbia. Like Lay's owns them but I don't know. Kind of hard to explain. And it does say Pepsi company here. So that's what I mixed. Their version of Lay's maybe? Yeah. But it says Polo Witch. Chicken. It's chicken. Yeah. So hopefully this is good. It literally smells like rotisserie chicken. Wow. That is crazy. They got the smell right on. But the chips look plain. Like a classic Lay's. But smell really good. Let's try this. It's very salty. My God. Wow. That tastes like chicken. Yeah, so that tastes exactly like chicken. But in my opinion, I kind of describe it like chicken skin. Like fried, crispy chicken skin. That's what I taste more of. Yeah. You know what? When I first tasted it, it reminds me of turkey. Turkey flavor. More of a chicken. In my opinion, yeah, for sure. It's definitely way saltier than any other chips I've had. No, it's really salty. Like Lay's classic chips are salty. It's like three, four times as much salt in this. So obviously this is our first snack. I love chips anyway, so I'm still gonna put this in first place for now. But it's likely going to be bumped, especially with the other chip, if it is better. I'm assuming we're gonna bump it out, but so far that's actually really good. And the flavors spot on, it's just salty. Salty, yeah, for sure. Okay. Let's go with the chocolate. Let's go with these. Well, what is that? They look very interesting. Chokies? Chokies? I'll show you here. I don't know. Not really a good name. It may be pronounced different in Columbia. Chokies, chokies.  I don't know. I don't know if their chocolate's good, but we're gonna try it. I'm not expecting that. It's so airy. Mm-hmm. That's a dark chocolate. That is really dark. Really dark chocolate. Wow, it doesn't look dark chocolate looking at it. It looks like a milk chocolate coating. Ooh, it leaves a really bad aftertaste. Better. Dark chocolate. Yeah. I don't like it. Yeah, but he likes dark chocolate. Yeah, but the overall flavor and experience, it's almost like, I don't know what the inside is like. It's very airy, like a risk-crispy treat. Yeah. But covered in dark chocolate. I don't like it because the aftertaste tastes like- I get that. It's bitter, but it's not just bitter. It tastes like earthy. Yeah. Yeah, it's gross. Sure. I don't know. You have to taste it, too. It's the second place for now, but it's gonna be bumped. I'm assuming. Oh, yeah. I'm gonna pick the next one here. This is why I absolutely love snack crate because this is a full-size bar. This isn't a mini Kit Kat or something like that. So this is a gold mega. All it says is wafer and I can't read the rest because I don't understand it, so we're just gonna open it up here, but it looks chocolate coating, maybe peanuts in it. I'm not sure. See, hopefully it's not messy. See if I can open it. Wow. Look at that. That looks so good. It's not the first bite. Sure. I'll put my hand here so it doesn't crumble. Whoa, that looks good. It smells so sweet, but it looks like caramel and Rice Krispie and wafer. It's hard to actually taste the chocolate though because there's so much going on. There's caramel, wafer. You got the chocolate. You got the Rice Krispies. I'd almost put that at first place. What would you think? Oh yeah, this is first place for sure. That's really good. That is delicious and it's huge. Definitely first place worthy. We'll see, there's two more chips in here, which is awesome because like I said before, Munch Pack, you rarely get one, so this you get three. Yeah, let's try a chip. Let's try these first. These ones are like the smaller bag out of the two that we have left. Choco, chocolateos, chocolateos. Lime flavored. They almost look like Frito. Yeah, is this from Fritos? Yeah, Frito Lays, Pepsi Company. Interesting. It doesn't smell like anything. It smells like Frito. Yeah, they definitely look like the barbecue Fritos we have here. Oh my God. Really lime? Yeah. Ooh. A punch. Ooh. Too much lime. I wasn't expecting that. You know like some chips over here in North America have like, I don't know, lime flavor, but it's like a hints of lime. Oh yeah, like very subtle. Very subtle. This is like, you bit a lime. Oh yeah, no, this is like. I don't want any more. You like Fritos and it tastes, the quality tastes just like Frito the chip, but like the lime flavor. I'm not, no. This tastes like it should be like a key lime pie. Like you know like. Yeah, like you're not expecting it in a chip. Yeah. No. Last place? Yeah. I like dessert chocolate way more than that. It's strawberry flavor and tangy fruit. I was not expecting snot green color. I'm just, that's not what I was thinking here at all. That's a gross color. Oh, you already have yours. Yeah, I have the strawberry one there, so. Yours looks so much better. Okay. That's right. Flavor though, that's so good. Do you want to switch? Do you want to try this one? Sure. It's a good strawberry flavor too. This one's better. Yeah, I like that one more. Or would you put these in first place? I'd put them in second. Jumbo flow, I don't really like the name of it. That is amazing. I kind of want to put this in first place. Which might get bumped from the next snack, but. Yeah, that's better than the wafer chocolate in my opinion. Very good. First place, so chocolate's just dominating. Usually we do chips number one, always. But chocolate in this video and the last one are just dominating. So we have one snack left. Yeah, tostacos, tostacos, tostacos, yeah. Yeah, it doesn't look like it's from Lays though. This one. No, no. Nope, from a company in Columbia, it said. I don't know what flavor it is. Pecante, it looks like chili on it, so maybe it's spicy, I like spice. Whoa.  That's cool. Yeah, that's really cool. Not much flavor, it's like a plain Dorito. I'm gonna have a little bit more to try it, but. It's still a good flavor though. That's a good tortilla chip though, just puffed out, but this one, it's almost first place for me, what would you say? Just cause it's so simple and so classic. Yeah, I'd say definitely first place. Yeah, you agree? So we do have a chip that's first place now. I can definitely taste the subtle hint of like, I don't know if it's chili or like it's very subtle, but. But it's not overpowering on the salt, it's like the chip. Another one was? Oh yeah, the chicken one. The lime one was just so out there. The chocolate was amazing as well, but this one here would be first place for us. Just delicious, so I hope you guys enjoyed this video. We do have one more munch pack that we did get. I think it's supposed to be Canadian themes and we are Canadian, so it'd be nice to see what snacks are gonna be inside. But I hope you guys enjoyed this video. After the next one, we might take a little bit of a break and then come back. We will see, it depends on what you guys think as well and how many people are loving this series, but we hope you guys enjoyed this video and we'll see you guys in the next one. Take care.