I hope you've all now digested what that in operator is all about and how to use it. So this brings us to that point of the task to ask, are there transcripts in our table of transcripts that are not in the ensemble table or in our table of mutations? So do we have transcript IDs in GNAS mutations that do not appear in GNAS transcripts? You've already written the code? Yes? Cool. Awesome. So, what should I write? In the same expression? Sorry. I'm excited about the expression. Well, we get a result. I was surprised. Right. I would be surprised too. So that's good. We need to figure out, is that the right thing to do or what did we do and maybe we'll need to explain it to our duck. So if we try to explain this to our duck, we would tell our duck, so hey, what we're doing here is we're creating two vectors and we're asking whether the result of the first vector is in the result of the, whether the first vector is in the second vector, whether it's contained in there. Okay. And then our duck might say, I don't understand. What's in that vector anyway? So we say, well, just have a look. Let's see what's in that vector anyway. So let's have a look what's in that vector. And then this appears and then the duck asks, well, what is that? And we say it's a table. So and the duck says, well, I'm, I'm seeing numbers and I'm seeing, I'm seeing IDs. So which one, what, what does, what does the in operator actually use? Does it use the IDs or does it use the numbers? And we say we don't know. We thought it would be using the IDs because that's like what we wanted to do, right? But we better check. So if we look at that table, that's what that looks like. So if this is using the names and the first entry here is false, this would mean that this name is not in that list. But that's not actually the case. I see 26, 5, 6, 20 here and I see 25, 6, 20 here. So that actually should be true. If it's using the numbers, that would mean 32 is not contained in this list here. And that actually seems to be true. Why is it using the numbers? Because a table is actually a named vector. So the contents of the vector are the counts. The names that we see are just attributes of the counts. So the actual contents, what the table is is counts. But the words, and we can use them, are things that are annotated to each count. And we can get them from the table with the names command. So we can use names for vectors, row names and call names for data frames and so on. So this is close, but as you've noted, there's something funny about it and actually trying to validate it, you will probably have noticed that this doesn't quite give you what you wanted. But it's close, exactly. So in this case here, the 26, 5, 6, 20 appears 32 times in the list of mutations that we downloaded from Intogen. It appears 12 times in the transcripts that are annotated by Ensemble. Any suggestion what we should be doing instead? We do what? Without table. So not tabling it, but using the vectors as is. That gives me a long, long, long, long list. And as far as I can see, they're all true. Get that in a moment. So right now they're all true. So what happened here? What were we doing? We were looking at a vector of 900 something or 703 mutations, 703 mutations, and we were checking whether they were in the transcript table. So there's a lot of redundancy here. However, the result actually tells us what we wanted to know. Regardless of redundancy, they're all there. If any of them would have been false, then the falses would have been spread all over the results, and that wouldn't have given us the correct result. Incidentally, if you have a long vector of truths and there's a single false in there, it's easy to miss if there's a false. There's two commands that we can use. The command all applies to a logical vector and asks whether all elements in that vector are true. So here the in command in operator returns 703 logical elements. If we wrap this whole thing into an all function, we get true. The result of this is yes, they're all true. Similar to that is any. This is even at least one of them, something. So that of course will also return true. Actually they're all of them. So all asks, are they all true? Any asks, is any of them true? How would we ask, is any of them false? With the exclamation mark, yes. And that's false because there's no false in there. So there's any false among these truths and so on. So any false is the same thing as all true. Right, but now in this case, this gives us the correct result. We know that they're all contained in there. But now how do we use unique? We can use it in two ways. First of all, table implicitly uniques something. So we could use this and wrap these table expression into names which returns the vectors, i.e. the top parts that you were looking at from the table. And that would give us the smaller unique list of truths. But we can also, instead of using table, realize that we don't actually need the cons. So doing table first and then extracting the names of doing that is unnecessarily redundant. So we can do unique instead, which gives us the same result. OK, well, that's satisfying. It seems to indicate that in principle, the intergen table and the ensemble transcript table seem to be talking about the same thing. Now we should think about which of these mutations we need to use. So how many of each mutation type do we have? That's the command that you're very familiar with now. How many of the type? Usually I hear a word from this side, which is table. And from that side, I more often hear a word which is unique. So table will count them. Unique will show us how many there are. So let's table them. Table what? GNS, what? Mutations? Mutations. GNS mutations. And the column is most severe. So we have 43 frameshift variants, 445 missense variants, 111 synonymous variants, 62 splice region variants, and so on. So that's good. Can we plot that? The relative proportions? Yeah. Can you give us what we want to do? Right. But what if a table true and false? Could be, yeah. OK. Yeah. So if you nest expressions like that, they're always evaluated from inside to out. OK. OK, so we have different things with different frequencies. Is there a common plot to look at different things with different frequencies? What do we usually do? Histograms, that's the distribution of values themselves, not quite what we're looking here. Here we're looking for the size of categories. So we use bar plots a lot. And in the Excel world, we use pie charts a lot. Usually bar plots are preferred to pie charts, but let's do a pie chart for once. So this is a pie chart of the result. Most of the variants we see immediately are splice variants. We have a small number of splice acceptor variants and splice region variants and stopgained and synonymous variants, frame shift in frame insertion. So splice, that's a bit of a problem. We don't know what splice region variants do. Are they silent? Are they non-silent? So that depends on the context. We don't have enough context to make that call. We just know that it's a mutation somewhere in the splice region, close to splice donor or acceptor. But we don't know anything else. So from interpreting them as missense or deleterious, it's a difficult call. So we can just exclude them by sub-setting. So let's exclude splice variants and also splice acceptor variants. So anything that contains the word splice, let's build a little selection here. How do I select rows that have most severe some things that contain splice? Using what? Later, but right now let me just define the selection. Define the indices or a logical vector that indicates these rows. What have we used previously to find patterns and streams? Grab. OK. So let's use grab. What are we looking for? What's the pattern we're looking for? Splice. So conveniently, this is listed here. By the way, the one thing where a pie chart is kind of more convenient than a bar chart is getting the category labels not to overlap or not to be truncated in a bar plot is often nuisance. And you have more plotting space for category labels in a pie chart. So we see splice acceptor and splice region. And it's lowercase splice. And we can just use that. So we look for splice. And where do we look for that? GNS mutations, dollar, most severe. 86 hits and a vector of numbers. So we want to exclude that. So how do we use row index numbers to exclude something with the minus sign? So GNS mutations is, what do I do now? Square brackets. Possibly something in the rows. Possibly something in the columns. How do I use my cell? Is this a selection of rows or columns? Columns. Rows. Rows. Let's go with rows. Well, the selection, however, is for those that we don't want. So we just put a minus sign here. And the columns, all of them. So before we do this, we have 703 observations. We already saw that cell has 86 items. So we expect 700 and 600 and 300. And 617 as a result. Yay. That seems to have worked in whatever way we wanted. OK. And the last validation we need is we need to check whether the reference nucleotides we have in our data are actually correct for the data we downloaded from GRCH38 for our 100,000 nucleotides. And that's a bit of a step, right? So we have a vector which goes from 1 to 100,000. And we have different numbers of the start positions, right? So we would expect that our vector from 1 to 100,000 somehow corresponds, if we map that to the genomic coordinate, 57 million, 4844, we would find a C there. If we map it to 4429, we would expect an A there. And so on. So now we need to map start coordinates to our vector coordinates. Simple arithmetic, but easy to get wrong. So what's the first nucleotide in our 100,000 nucleotides? So our vector might seek index 1. Where is that on the genome? The minimum value that we found. And where was that? I think that was somewhere in the sequence analysis script. Right, so we downloaded 58,815,001. So the index 1 in mySeq corresponds to 58,815,001. So in order to map mySeq coordinates to genomic coordinates, all we need to do is to add 58,00815,000 to the index. And then we get the correct genomic coordinate. Or we need to subtract 58 million, something, something 0,0,0,0 from the start position, which we find in the integer file. Then we will find the index in our mySeq file. Then we can extract the nucleotide at that index. And then we can compare, is this actually the one that integer tells us is in the right position? So integer minus offset equals, so can we do that? No, yes, no? Oh, that's very simple. It's just math. Offset is the difference between the genomic coordinate and our index. So genomic coordinates is 58 million, something. Our index in the mySeq list of nucleotides is 1. So in order to convert our mySeq index into a genomic coordinate, we need to add 58 million. Or in order to map a genomic coordinate into our mySeq file, we need to subtract 58 million, something. Right? So let's try this. We can just do this by hand. We look into the genus mutations and start position 57484420. And we just looked at mySeq analysis. And we saw our first nucleotide is 58 million, something. So nucleotide 1 is this position. So we add this to the nucleotide to get the genomic coordinate, or we subtract that from the genomic coordinate to get the nucleotide. And we get the index in the file. And that's, whoa, what's that? 1,330,580. Negative? Yeah? I just have a question. Our chromosomal coordinates, are they always in nucleotides? Clean units in nucleotides? Yeah. Right. These are just a number of nucleotides from the positive stand in the beginning of the short arm, the P arm. Chromosome. It's per chromosome. So the coordinate is somewhat incomplete, specified as a chromosomal coordinate. It would be 20 colon 20 feet. So this is just a number of nucleotides. The 20 is implicit. But we get a negative number. What? So do I need to add this instead? Or what's going on? Why is this a negative number? The wrong genome. It's the wrong genome. It's the wrong genome. Intogen is not using GRCH38. Intogen is using GRCH37. That's the kind of stuff that will make for a happy day when you add vodka. You realize this, and you've gotten this far, and we can't just take these coordinates. And you have to be super, super aware of that. If you just blindly trust that and you take the 57 million and then you take your genomic coordinates and you get the nucleotide there, it's wrong. It's just not in the right place. So molecular biology is messy. Bioinformatics adds a level of messy to that. And you have to be super, super careful at every single step. If you only remember one thing from this workshop, it's this experience. We do something which is computationally absolutely correct, but doesn't make biological sense because our data sources are somehow not compatible. Not just the build, but actually you also need to put in the patch level. The patch level. If you've looked closely, wherever it says ensemble something, it says GRCH38.p13. So there's also patches which are done alongside major releases. And strictly speaking, you have to know the release and the patch level to be able to correctly identify and map a genomic coordinate to the actual victim. So we need to recover from that. Fortunately, everything we've done is in the script, so we can recover very easily. The easiest way to recover is just to toss out MySeq and download MySeq37 instead. I need the nucleotides that are correct for GRCH37 and then validate them. So everything that we've now done to prepare for our Intogen table stays in place. And our transcript table is still kind of correct except that the coordinates in the transcript table wouldn't match anymore. But if we simply download the nucleotides, at least we can tell where the mutations map to. Note one thing, however. Remember that we checked whether the transcripts match. So the transcripts in the transcript table were from the ensemble GRCH38. The transcripts in Intogen come from ensemble GRCH37. So the transcript names are not changed. It's just the position of the transcripts on the chromosome that changes. If the new annotation, which has apparently added something like 1.3 million of nucleotides upstream of that position to chromosome 20, if that would affect the nucleotide sequence of one of the transcripts, then it would get a new ensemble ID. But since the sequence between 37 and 38 appears to be the same for these transcripts, they maintain the same transcript ID, even though that on the new reference genome they're located in slightly different coordinates. OK, so how do we fix that? Well, we need to make a new table. So we need to find a gene source for GRCH37. So we go to ensemble, if I would be. And on the main ensemble page, the location of the archival GRCH37 link is a little more prominent. But you can also find it on this page here. So here, there's a link on the US East mirror, still using GRCH37 question mark. And if you click on that, you find visit our GRCH37 dedicated site. And this has a different URL, grch37.ensemble.org. This means we need to look for slightly different coordinates. So we can probably do that by just typing humanGNAS. Something confuses it. I'm impatient. Maybe it finds everything. Let's just look for GNAS. Come again? But I get this as the top list. GNAS, humanGN87460 looks correct. Next, do you mean here, humanGNAS? OK, well, that's nice. That looks very similar. We can see these coordinates here. So this is around 47, 410,047, 500, 57, 500,000. So let's download the data for 20 colon 57, 410,001, 257,510, 0,0,0. Cation, we send exactly, if you look up, location. Right, but this is the exact location of the GNAS gene, but it's not 100,000 nucleotides. So I'd like to keep that the same. Just the different genome assembly, but we'll work with these 100,000 nucleotides. File yesterday, 37. OK, is there a way to convert that? Yes, there are ways to map GRCH37 to GRCH38. It's a bit involved, but it's possible. So basically what you need is a file of offsets of every single change between one and one. So you need to read this very large file, and then you need to apply all the offsets from one coordinate to the other coordinate to. So essentially it works similar to what we saw before.  Then these values of 1.3 million are, of course, dependent on where on the chromosome you are, and that can be a bit involved. So what I did here is I just clicked on the location, which gives me a map by location, and then I entered into the location field my string here. So then I get to chromosome 20, 57,410,000 to 57,510,000. Now how do we download the nucleotide data from that? We've already done that once. Export data, output as fast A sequence. The location is this, no flanking sequences, genomic unmasked. Click on Next, go to Text, File, Save As. And now we give it a file name of chromosome 20, 100 kilobase pairs dot 37 dot fast A. Don't append 10. And the file is here, and if we look at it, it confirms that this is correct. So GRCH37, chromosome 20, and these are the co-ninates we want. Now we need to read that fast A file with our read fast A function. My GRCH37 is read fast A of chromosome 20, 100 kbp37 dot fast A. Right. Now we're back in business. So to recover that was relatively quick, but it was super important to have noticed that. So being back in business means that now we should be able to apply an integer coordinate, track the offset, and get something from our MySeq index here. So let's verify that. Offset 37 should be 57,410,000, because that's from where we downloaded the data. And now let's see what we have here. So GNS mutations of this value here should be a C. MySeq 37 of this value here minus offset 37 is a C. Well, that's not bad. I'd be very annoyed if that weren't the case. Is this just being lucky? What's the chance of being correct by chance? One in four. One in four. So we don't trust it yet, right? Let's try another one. Here we have a G in this one here. It's a G. So what's the probability of that happening by chance? One in 16. So we're getting better. OK. So I'd like everyone to pick a random nucleotide from the table and check whether they get the right nucleotide back from MySeq 37. If anyone finds a wrong nucleotide, that would be important to know for two reasons. Either our mapping is wrong, we need to know that, or your data is wrong, and we need to know that, too. So pick a random start position on the integer mutation table. Apply the offset. Look for the index position in MySeq 37. Pull out whatever nucleotide is found there and confirm whether that's the same as the nucleotide in the reference column of the mutation table. Now, of course, I'm essentially crowdsourcing this. If you weren't crowdsourcing this in the class, I would write a loop. And for every unique start position in the table, I would pull out what the reference coordinate is, and then compare reference coordinate to the found coordinate, and tell it to complain loudly if there's a mismatch. So if you were able to check one, please let us see a blue post-it. OK. As I glance around the room, I see no reds and lots of blues. So assuming that you've not just all reproduced the first one I tried, but looked at several different ones randomly in our file, we've seen no contradictions, which is good, which means we now actually have data that we can integrate. Because our identifiers actually mean the same thing. I'm always really, really thrilled if something like that works. We can pick a coordinate from a site in Barcelona and take a 57 million number, and we can look into something. We download it from a site somewhere in Boston, probably. And we find that we get the same letter there. This is pretty cool. Anyway, so now messy. I'm not going to give you a long sermon about how to build data models and how to integrate in a more principled way. I have a little PDF file here about data models that I share with my bioinformatics students. This is an annotated file of essentially a presentation. And these are presentation slides with extensive annotations. So this actually should kind of make sense if you read through that when you have time. Very often we use relational data models when we build data models of the data that we want to work with. Relational data models are defined by entities. This is like a data frame in R. Entities have attributes. Each of the attributes would be like a column in a data frame in R. And there are many of these entities in our database. Each of the entities would be like a row in a data frame in R. But these relational models also have relationships to other things. So for example, if we have a protein, we can store the protein ID and the taxonomy ID. And the taxonomy ID might relate to a second data frame, which has taxonomy information, i.e. taxonomy ID and the species name. And they're related because they share an ID. So generally, we build relational models by having tables or data frames that somehow have shared information. But that's all I'll say about that. Read through that when you have a few quiet moments. Now in order to integrate the data for our plot, we need genomic coordinates because that's what the sequencing experiments and variant calling and whatever return. We need the coding sequence. We need the codon positions and the translation. And we need the mutations that are mapped to the sequence of interest. So for example, from the mutation table that Indogen has provided, we can't tell whether adjacent nucleotide positions map to the same or different codon. They could be the last position of one codon and the first position of another codon. Or they could both be in the same codon. So if we count our mutations and map them to amino acids on the amino acid sequence, we need to distinguish them. Now Indogen has a column of protein positions. But as we've already seen, that depends on the transcripts. So what we should be doing is actually taking the nucleotide sequence, figuring out the gene model, piecing together our sequence from the gene model and translating that, and then mapping our mutations into that. And for that, we need a gene model. So what's a gene model? Did you ever come across the term? Yeah, in principle. So exons and introns is part of the concept. The gene model actually contains the exact start site, which may not be the first nucleotide of the first exon, but somewhere in the exon. Usually there's some untranslated 5-prime UTR. And the last stop codon is not going to be the last nucleotide in the last exon. So we need to know these as well. But other than that, the exons and the introns of the internal exons and introns are all mapped to CDS. So coding sequence, CDS, identified by the ranges of genomic coordinates that refer to the coding sequence. So there are two major sites of integrating information that we use in bioinformatics. One is the NCBI entree system, which is quite useful to explore NCBI data. They've recently done a lot in actually cross-referencing information into the EBI world, which is also very useful. In general, though, working with things programmatically, the EBI site is more useful. And the big integration facility at the EBI is Biomart. So Biomart provides integrated data. Now, we need integrated data for GRCH37. So be careful. Don't just go to any Biomart, but we need GRCH37. And we already have GRCH37 loaded here, GRCH737.org. In the top menu, you find a link to Biomart. So follow that link. So what we're trying to do now is to download a gene model. If I were to tell you how long I took to find, where to find download of a gene model information, I would probably be slightly embarrassed. But it took far too long. If you look into the genome browser and you click on the translated exons, it's immediately available. Things are colored in the right way. But then if you try to download what coordinates this actually corresponds to, this is horrible. It's just not done with that in mind. But finally, Biomart to the rescue, Biomart has what we need. So let's download that from Biomart, GRCH37. So getting Biomart data involves four steps. But first, we need to choose the database. There's Ensembl Genes 91, Ensembl Variation 91. This is the database that contains all the SNPs. Ensembl Regulation 91 contains also expression information. So what we need is Ensembl Genes 91. I just noticed that in your script, I'm actually navigating to ensemble.org and downloading this. That's actually wrong. You should be going to GRCH37 ensemble, choosing genes 92 and choose human genes GRCH37. But we'll walk through that. So Ensembl Genes 91. If this were my project, I'd make a note in my journal now because this is not something I'm writing into the code and it's actually wrong in your code. The data set we use is human genes GRCH37 P13. And once we've selected that, we get to choose what aspects of that integrated data we want to see. We can filter and we can retrieve attributes. So to filter, we can define regions, genes, phenotypes, gene ontology, multi-species comparisons, protein domains and families, variants. There's a ton of information that's available there to restrict your search to some things. So in terms of gene ontology, you could look for genes that have particular go term accessions. In terms of phenotypes, you could choose from a long list of known phenotypes. And select gene information that is associated with these phenotypes and so on. So what we are looking for here is gene level information. And we are going to select with gene stable ID. So what was the gene stable ID that we were working with? Well, it's in here. The ensemble gene ID for GNAS is ENSG087460. So we copy that, paste that into the filter. Clicking outside of that text window anywhere will actually load that filter restriction and then apply it. So this is what we want. We want information about this gene. But what data do we get back? Well, that's contained in the attributes. So filters take a subset of the data. Attributes define what information from that subset we actually get. The attributes we want to get for our gene model is a structural attribute, i.e., information about where in the genome this is found. So we click on structure. And we can choose attributes at the gene and at the exon level. At the gene level, we want the gene ID and the transcript ID to make sure we have the right gene and we know which transcript this refers to. Of course, different transcripts have different gene models. In terms of exons, we need the exon rank in the transcript, just to be sure. So first, second, third, and so on. The genomic coding start, the genomic coding end. So this is in our gene model where in the nucleotide sequence this all begins and ends. And the exon stable ID, we could compare this. And I think that's all we need to define here. So definitely, if you find some time, explore this some more. Look at the data that you can get there. This is, even if you're not working human genes, quite likely in any other model organism, they also have similar data available. It's very well curated, very useful. So one of the things we need for integrating information all the time. Note that the Bioconductor project has a Biomart R package, which allows you to set these parameters in a programmatic fashion. But even though you can do that, it's still useful to basically manually try this out from time to time so you know what options are available and how this selection works. So selecting gene IDs and getting this list of attributes, we can then click on Results and see what we get. Gene stable ID is the correct one. We see exon ranks, which is what we need. Genomic coding starts, at least for some of them, exon stable ID, and so on. So that's all good. It allows us to export this into a file. As CSV, TSV, HTML, or XLS, let's just keep the default TSV and click on Go. And this downloads a file into my Downloads folder, which is called martexport.txt. And if I look into that, I see, indeed, we get lots of information, much of it, which is probably useful. So I'm taking this. I'm going to name it GNSGeneModel37.tsv and put it into the project folder. All right. So we have GNSGeneModel37.tsv. Then, of course, it's not yet useful as a file. We read it into a data frame, as we've done before. The read command is read what? CSV, read the limb. The file name was GNSGeneModels. Any parameters we need to set? Does it have a header? It has a header. Good. Anything else we need to set? They're tap separated. That's default. Well, we've said we can be OCD about that and specify that anyway. Anything else? Strings as factors. Thank you. Strings as factors is false. And go. So GNSModels, 435 observations, six variables. GeneStable ID, transcript stable ID, exon ranks, coding start, coding end, and exon stable ID. Yeah. Why which transcript? Hang on. Hang on. Hang on. Hang on. Hang on. Hang on. Hang on. Let's back up. OK. So remember, we specified the genes as attributes, the gene. Let's hang on. Let's step back. So the filter is GeneStable ID. And the GeneStable ID is ENSG87460. So that's an ENSG, not a transcript. It's a gene. So we're looking for data that are annotated to this gene. So there's lots of data annotated to that gene. But what we're specifically looking for is defined when we go to attributes. In this case, we're looking for structural information. And we need some information defined at the gene level and some information defined at the exon level. The information at the gene level is GeneStable ID. Well, we actually already know that because we've only had one, but it doesn't hurt. Basically, as a sanity check to see that we've got the right gene ID in our output data. And the transcript stable ID. At the exon level, we want the exon rank in the transcript, the genomic coding start, the genomic coding end, and the exon stable ID. That allows us to integrate things with the nucleotide data, which we've downloaded. Once you have this defined, the right filter on the right data set and asking for the right attributes, you can click on results. The results very kindly show you what the results table looks like, what's contained in it. So you can be sure, yes, I'm indeed seeing the right stable ID. Yes, I'm indeed seeing transcripts. I have data for what I'm actually asking for. And once we're happy with what we're doing here, we can use the export function, export all results to file, choose a TSV format, click on Go. That will put the data into your standard browser download folder, where all your downloads end up. Then you go into your download folder, you change your file name, and you move the file into your project folder. So if you're stuck with that, let's see Red post it. OK. No, that's a trouble. So that change, click on the minus sign. Exxon, we want it. Exxon, we want it to be square. We're going to get it in this browser. Exxon, we're going to get it in this browser. OK, now click on Results. That's the first one. Who else? Exxon, we want it to be square. I don't know what this is. Oh, save that. Save that. Save that. This is cool. Exxon results in your file, TZ, and just click on Go. Save. Save. So this is somewhere in your download folder. And rename it. And move it from your download to your project folder. Or save it directly to your project folder. Okay, are we all there? Excellent. So what I'll do next is there's a couple of talks of putting the data together, which is essentially more of the same. So what I think we'll do here is I'll just type out. You try to follow along as well as possible. I'll just type out what we're doing here to put the data together. Then once we're done and happy with the result, I'll take the code and I'll post it into my code snips and I'll upload it. And when you come back from coffee, you can download it, run the code, and then you should be all set and have the same data locally, right? So the first thing is we want a data frame which contains only our GinaS2 gene model according to the following specifications. So we choose data for ENST something 371095. This is a transcript that codes for a particular protein which has these identifiers. So it's an ice form of P36092, GinaS2. That data frame should be called GinaS2 model and store the column start and end for each CDS segment. And then we need to make sure the segments are in the correct order. So we do that. Our spec calls for a data frame called GinaS2 model is a data frame, and it needs to store column start and end from our GinaS models, genomic coding start, but not all of them, just a selection. I'll need to think about what that selection ought to be. And we need a second column here called end is similar, genomic coding end, and again a particular selection. Now these are numbers so technically I wouldn't need strings as factors, but it doesn't hurt. Now I need to define what that selection is going to be. Selection is going to be all rows in my GNAS models $transcript stable ID, which are this transcript. How many do we have? 12. Well that's kind of what we would expect for the 12 exons that the UniPro site told us that we would have. So let's build this little model here. 12 observations of two variables, starts and end. Okay, that kind of looks like approximately what we want. Now make sure the segments are in the correct order. Are they? How do we know? What's the correct order? So molecular biology is really, really messy, but I'm not aware of transcripts that are spliced out of linear order. So where an exon ends up before another exon due to splicing. I can't guarantee that this never happens. Some organisms make very, very strange RNA modifications, but I think generally it's safe to assume that the order of exons is determined by the order of the starting indices. So let's have a look into that, into our model gene as to model dollar starts, and we see the following. 12, 1, 11, 2, 3, 4, 5, 6, 7, 8, 9, 10, oops. That tells me that these start and end coordinates are not in the correct sequence. So the 12th one should actually be the first one. That's the smallest value here, 4, 7, 4, 6, 6, something, and indeed 4, 7, 4, 6, 6, something is smaller than 4, 7, 4, 7, this here. So if we wouldn't have checked this, we would now have started assembling our coding sequence in the wrong order. You can never be too careful. We did check it. We caught this, so the correct way to order this is to say GNS2 model is GNS2 model. The rows should come in the order which we've stored in our vector ORD, and we want all of the columns. Now looking back into here, this is smaller than this. This is smaller than that. This is smaller than that, and so on. Now they're in the correct order. The order we get is our original 12th, then our original 1st, then our original 11th, and the rest came through in sequence. Why it comes through from biomarked in an incorrect order or in a non-canonical order, I have no idea. It just shows that there's no guarantee about these things. You have to be careful. You have to look at your data all the time. Never blindly trust that you'll be getting the correct thing. An error like that is really hard to catch, because syntactically everything looks fine. You have a start ID, it's smaller than the end ID. You have a segment. You start putting things together, A, B, C, D, E. You get a nucleotide sequence to translate it. It makes no sense whatsoever. Because very likely you'll already start with a frame shift at the first one. All right, so that's done. Now we create a data frame for GNS2 protein annotations according to the following specs. Actually we'll do that after our coffee break, and I'll lift over the code and walk you through that, and then we'll be ready very quickly to plot all of this. So three o'clock, good time for a coffee break, half an hour until 3.30, and then we're going into the final lap of this grueling workshop. So drink lots of coffee.


everyone and welcome. My name is Melissa Armell and I own the Stock Swoosh and today I'm going to talk about the importance of education in reference to trading the market. A lot of people take trades, risk money in the market, and many people are active without the proper education. Some people have had some education, some people have had no education, some people have had a lot of education, and still are losing money trading. Why is it important to have education? It's important to have education before you know how to trade because there's so many different things to be looking at when you are putting money at risk, and your money is also important. So to just take a trade without knowing why you're doing it, without understanding it, without having what I call conviction in a system or strategy is pretty much gambling. So you don't have the high odds or good odds of making money then consistently in the market if you're gambling, if your mentality is gambling. Now you might think it's not because you might have read something online or seen something on TV or maybe you're following some random trading room doing a trading room trial or month to month in some trading room and you're just taking their trades, but if you'll understand the system that even that person is trading, you don't have a philosophy, a protocol, a mentor even, anything that you would follow that you know consistently works when you're taking trades in the market, you're really gambling. So when I teach people my method, it's a method you can follow and do every day and it's what I get up in the morning and go through the own protocol that I have, which is my checklist every day. So because I use the checklist, because I have the protocol, it's not gambling, it's also something that I've been using for the last 10 years or ever since I created the whole system, took me about three years to figure the whole thing out. But there's a reliability in it because of the length of time I've been doing it myself and teaching people as well. And there's a protocol which I'm prepared before the market opens doing the system of strategy.  You're taking risks based on what? Based on gambling, without any knowledge, without enough information, without really knowing or having any conviction that it's going to in fact work.  You don't know what the system is and you haven't traded it until you take the class and start to actively trade in the market. That's true. But because of my experience over 10 years trading, then also having the business for over six and following even a trial in the room with me, or even if you've been on my marketing list for any length of time at all, or asking me questions, directly asking me questions or watching some of my videos, I'm sure many people can tell that there is a consistency to what I'm doing, which is I'm actively looking for gaps every morning to trade. So I'm actively looking for stocks that are gapping. That is what I focus on. It's how I find those specific picks that you learn in the class, how I take the setups, how I take the trades, how do I know that IBM is the one or whatever, whatever stock I happen to be watching that day, how do I know that that stock is going to sell up on the day before it sells off to short it, or how do I know that a stock like Netflix is going to rally when it gaps up to go along it, for example. So whenever you decide to take a class, there is a certain element of risk as far as you deciding to take the class, not knowing all of the information for the class. But anything that you do, if you want to decide to trade involves risk, whether it's spending money in a class, whether it's taking trades, you get education when you come and do my class. When you decide to take trades, you don't get an education from that. And it's very hard for people actually to pick up even the mistakes they've made on their own when they're trading. People think that they know the mistakes that they made, but most of the times they don't, because they don't really know still what they should be looking at or what they need to be looking at. And they think that they're helping themselves by actively trading without a specific strategy or system. And they think that their money is better spent trading than paying for a class, whether it's my class or some other class at some other place. But the benefit of coming to me is that you would take the class when you could be in the live training room with me every day. It's only open to students. And you could ask me questions in the room or obviously on email or call me as well too. But it is really important to have education if you want to risk your money in the market, otherwise it's gambling. And maybe you've taken classes in the past and maybe some of those classes, you didn't make any money off of those systems. Maybe that's true. But it doesn't mean that every class out there doesn't have something to offer. So even if you didn't learn really how to train with the class you took in the past and you paid money for it, you probably got something out of it that brought you where you are today to find me. To learn a system which is my golden gap system that not only teaches you how to read institutional money in a chart, but my system really teaches people how to make money in the market because you make money in the market by finding momentum and playing it. But it's about finding the good ones. It's about finding where it's going to rally and go and where it's going to sell off as well. Thanks everyone. If you have any questions, email me at melissa at thestockswish.com.


So when I arrived at my foster family playing, she did help me a lot because she tried to teach me English culture, different things. She took me to the community clubs that I knew and I'm glad that she did. I had my first English friends from that community place. The reason that I left my country was because I was helping to rise awareness about some political issues, but it ended up being my life in risk and I left from Afghanistan. So in general I can say my foster family helped me to learn different cultures, how to be a good cook, because when I first arrived, I was a rubbish dish. My name is Netri, I'm 19 years old and this is my story. Just received the decision from the home office, you have been granted a few days to no appeal. Oh my God, it's a big relief, thank you. Finally.


structures we have nowadays is power generation. If there's no power, we're pretty much screwed. Our next speakers will take a very close look at common industrial control systems used in power turbines and their shortcomings. So please give a warm round of applause to Reptep, Muradak, and Kors. Good morning, Congress. Thank you for waking up in the morning. We will talk about the security of power plants today, specifically about automation systems that are used in the power plants. You might think that this is another talk about how insecure the whole industrial things around us are, and more or less it is. So for years, we and our colleagues speak about problems in industrial security. We are happy to say that things are getting better, but the temper is a little bit different and feels a little bit uncomfortable. So anyway, we will speak about how power plants are built, what is the automation inside, what are the vulnerabilities, and the high-level overview of what you can do with this. But at first, a little bit of introduction. We are security consultants. We work with a lot of industrial things like PLCs, RTUs, SCADES, DCSS, whatever it is. We were doing this for too long. We've been doing this for so long that we have a huge map of context at such a point that we have a lot of contact with the integrators, the vendors. This has allowed us, in particular, to work for people who manage power systems, for example energy centers. So something that is very important to know is that everything we are going to talk about today has already been reported to the vendors concerned. And we are going to talk mainly about one of the vendors, it's Smenth. But we will see that similar problems can be found in similar solutions by other vendors. And it applies to other vendors that we will not necessarily mention today. Apart from that, we will talk about security problems that apply to real energy centers. We are kind of irresponsible guys. It may seem irresponsible, but in reality it's the opposite. To do some kind of research with these systems that are in the power plants, we have to access these systems to work on them, to know them. All these resources, they are limited. They are difficult to access for people like us. But for potential attacks, it's their job to invest time and research to carry out this attack. So we can assume that the bad guys already know all this and it's important to share this knowledge so that people can act well. So the energy centers are the most common means for human beings to get electricity. They have it everywhere. And the closest to it is the Lipendorf Power Station. And we can be surprised by the amount of information we can get from the internet. For example, on Google Maps, we can see that we get a very good way of getting electricity. And so we see what the building is doing on this page. We also see which equipment is being used. And if you don't have this experience, you can just look at it and you see which system is being used to automate it. It's a system that's called SPAT 2000 and SPPAT 3000, which are Siemens systems. And so it's SPAT 3000. It's a little confused, so we're not sure yet, but it's exactly the system on which we're going to take our look today. And we'll see again, it could be another system. It's just that we've seen this system more and more often than the others, so we're going to talk about it. So you can look at all the systems and all the electrical centers thanks to the people who monitor the coal. So these are the different types of power creation and we all see them by color. So we have the coal, we have gas, we have hydrolysis, we have nuclear and that's a lot. So what we're going to do today is those that work with gas or coal. So the plug. What works is a turbine. We unfortunately don't have a photo of a turbine, but I hope everyone has seen it on a plane how it looks like in a plane. Especially in terms of size and the way it works. On the website of different vendors, you'll see the different types of turbines. So these are the Siemens turbines, specifically used in power plants. So they're not only used in the electrical centers. We can also do a lot of other things with them, but if you compare with the previous slide, you'll see those that were used in the electrical centers. This is important because because of the vulnerability that we're going to have today. So before we talk about automation in an electrical center, you need to understand a little bit how it works. On the right side, and it's very easy. We'll simplify a lot of things to make it more suitable for all this sort of thing. We don't really understand everything. So the first thing you should do is to have a fuel tank. So we're going to have gas or coal and you put it in a combustion chamber where you turn it on, you put the fire on and it's going to generate a lot of pressure that's going to go to the turbine and because of that turbine, the turbine is going to start to turn. The turbine has an axis that's going to turn that's going to be used with a kind of dynamo that's going to generate electricity. So when we generate electricity, we put the power that's generated is not only for certain buildings but for the electric grid for all that is the local or national electric circuit. There's another point. When we generate this, we have a lot too much heat when we put the fire on and a big question is how do we do to cool that? The first thing is with a conversation. The second thing is to take this heat to warm water and we do hot water and the hot water is going to be turned into steam in steam and we put that in a second turbine which is going to be a steam turbine to optimize. It's an optimization in a way. So what is the automation in this process? The automation systems that are used is a system distributed, DCS and it automatizes everything in the cycle. So everything I just described. The vendors of these solutions want to simplify, want to simplify everything for their clients because instead of having 100 people working it's just a dozen. They want to simplify the whole process. They don't care how much they need how much fuel they need. They just need to launch the process and they need to know how much power we're supposed to produce at this moment, how many megawatts of electricity we're supposed to produce. That's what you need to know. The complexity that is hidden in these solutions because there are a lot of small things that are happening inside. So we're not going to talk about that. So this DCS system is not only for power plants but also for other things. We're going to use the same materials, the same software. For example, it's not only a software that you can install. It's an interaction between software and material. And so sometimes it starts with construction of buildings with the buildings. So it's a lot more complex. There are a lot of vendors. So as we said, we're going to talk more about Siemens, but there are several. So a short description of how things are simplified for operators. So for example, if you want to answer the question how do we regulate the output in megawatts, we need to control three things. We simplify again. First, you control, for example, a gas turbine. We need to know how much gas we're going to put in the combustion chamber. We need to control the flame temperature and we need to control the air that's going to be in the turbine. So that's simple PLC. And you can change from 100 megawatts to 150 megawatt just on these three settings. So the system itself that we're going to discuss, is the SPPA T3000. Again, as well as all other systems, DCS, other vendors, it's a typical industrial system with PLC, DATU, DHS, MIS, servers, OPC. It's the only thing that we have that is different for Siemens. They have two main things called OPC, which are the server, and the automation server, with the software that works on it. It's something that will belong to the other vendors. So if you read the manuals for Siemens, there will be a lot of different networks, a lot of different routes, it looks like there's no connection between the application network and the external network. In practice, you're going to see with specific sensors for the turbine vibrations, you're going to see all the... all the electrical centers won't have people who are doing maintenance. They're going to need to install on the system, install the antivirus, and they're going to need to launch OPC traffic, information on the production outside, for example, for regulators or on the energy market. So also for certain entities that monitor how much electricity there is for the energy market. So our presentation is going to be structured like this. We're going to talk about the application, then automation, and then... So we're going to start with... So all of this starts with the coordination of vulnerability. So at the beginning of December, in December, Simen published an AdViso about different vulnerabilities identified by us, but not only other entities contributed. So it's... The system is just supported by Simen. There's no specific integrator. So they're going to install their package, they're going to install the message on the critical infrastructure, and so we hope that everything that's already been patched up. There's a lot to talk about. We're going to talk about it, because we have a lot to say. So most of the... there are several vulnerabilities that are the same, actually. And they also say... they also talk about the criticality of things, but it's not necessarily applicable to an industrial site, because each vulnerability can impact the process, and we're going to jump on it. So that's the attack model, and we're going to... In January, we're going to give you that, we're going to do a white paper, you know what I mean. So the software application is the main resource that you would find in the SPP in the network. If you access the system, you'll find yourself in the app. If someone wants to launch the electricity generation, or to change the values of energy, it's there too. There will be other servers that are going to try to communicate with the app. They're going to download the app at the start. So there's a lot of open ports on this server. On this machine, we can see that this is the first point. It's a huge attack surface, absolutely amazing, but even if you want to compromise the Siemens or another one-third, you have a lot of attack, the attack surface. So all the installation of this SPP system are different. So depending on the version, the generation, you're going to see different windows, from 2003 to 2016. So we hope that they're all updated in the meantime, because the update for an installation like this is something very difficult. You have to put yourself in maintenance, and it's something we do once every six months or once a year. So you always have to find a window to install. So there's always a possibility to have an operating window for profitability. There's a lot of additional software, so for example, old Sieguins that don't configure Tomcat, so for example, we see the configuration of different software is aligned with the best practices of Benchmark 6 to secure an installation. It's a lot of fun. There's a lot of software in Java, so you have to talk about it. Surprise, one of the most biggest problems in the SPP 3.0 is the password. There are three things. All the installations before 2014 or 2015 have the same password, and we find that on Google. They published all their documents. So after 2015, the passwords are automatically generated, but until this year, it was difficult to change the password. So you had to know the process, you had to contact your integrator, and starting in December this year, it's a lot easier to verify, to change that. So even if you know you have this problem, you can't just change the password. So otherwise, you also have the complete diagrams and the two integrators for the integrators, how the system is built, etc. So it wasn't published by Siemens, it's simply an electrical center that has put it online, it's made by itself. So the most important thing is that the application services are a number of applications in Java. So my colleague is going to talk about it. Hello everyone. So let's see what kind of software is running on this kind of server. So, a light customer communicates with the server via HTTPS. HTTPS or HTTPS. So communication can be protected, thanks, by a firewall. In the case of a heavy client, we have discovered the service via a registered friend. So an illustration of the architecture for a SPPA. So if we organize that in different categories. So we have requests that come from light customers and are redirected to the good service. So we see that there are several containers that use RMI services. All the containers are represented, all different types. And all have names that explain themselves. So before starting to dive in before we show you some tools. So the first one is a deofuscator. The Java code is obfuscated. But we can avoid this problem by simply using it publicly available. And otherwise it is also convenient to see how the system communicates. It helps to understand the architecture of the system and the client's workflow. And so for SPPA it represents the TCP streams in a system that is readable for human speech. So this method is unsafe so they use it. So we have to go and see SPPA. So the first thing we're going to see is the Apache server. So we're going to look at the Orion directory on the config. And this directory contains sensitive files for example PCSystemConfiguration.xml or the AFC which contains the starting configuration of the different services or for the automated system. Otherwise there is a Tomcat where we can start using the visualizations. So this is the surface attack. So we have a distance diagnostic we have the configuration of Tomcat, Apache, we see the Orion server which can be accessed via HTTPS and in the file web.xml we have a list of all the services and applications and this list is huge. So some of the very attractive names like for example Browse Servlet which allows an authorized user to look at the content of the directory also on the operating system. So in the case of the problem operation, it's even better since there is a second one which is even better it's for file upload to the file. With the system right it's so with the target name parameter we can completely install things, that is without authentication we can install the good shell via Tomcat and we install that with the system right so some servlets which also have the name factory service inside so it generates the HTTP so they pass the requests to the HTTP and the search and we will look at the services and it's a parameter that we give them the service URL and and we see the all that ok, the parameters are simply put inside the request and it builds a request so now we have we have light customers who can accept the RMI and they can also communicate directly with the registry so if the application is missing some security security and there will be vulnerabilities inside that we can use and so we can simply exploit these vulnerabilities and recover the code execution for instance with the system and the right system and the first step we will simply use the lookup service a little bit to get a list of services except the the except I think they implemented a general for Aspepar and so what we did was we simply used the lookup service and it looks like a collection of another RMI service at least the public methods give the name of all the services and by using by looking at the names and services we have all the RMI services at this moment we implemented the RMI service factory so we have we can assume that that it will be we can also have another RMI service so we can combine we decompile we decompile the class and we find some factory methods which so like get service and inside we can find the name of the created service we can also guess so using public methods get service and the name we can have a reference to the to the next RMI service and so in the end we have the references on the RMI services which actually have usable methods so these services are a lot of public methods for the methods for the users authorized or not authorized understood and each step we have found a lot of RMI services and a lot of public methods so so the surface attack of this system is absolutely huge so when we list all this all the services available how does the authentication find feet so we will look at this question we will have the requests to the security service so the first thing is that the client will have a reference to the security service using a customer then the create security service will try to have a session in the manager of the session if the manager does not work there is an exception and the client will not get anything but if it works we get a PC service factory and so we will be able to instantiate the method create an instance of the security service the session ID and the client will be stored as login ID security service and the client will get a reference on the security service where it will be able to import the methods so these methods will be able to check in checking their ID so we have two security but how? how do we do to register? so we will with a security service the client will use this ID and it will end a bypass of the authentication and we will be able to delegate methods exposed by the RMI these public methods are huge and it is difficult to manage the security of these services so now we know all the inputs on these systems we know their security measures now so among the list of RMI there is one that looks particularly interesting is the admin service so we will be able to get an anonymous session via the run script method so we can call it without identifying so first of all this method creates an instance and at this stage it will imply another Java class in fact we will create a second class that will simply implement admin script and this class will be executed by run script so we create the class and so and this class is created by the arguments of run script and we pass the parameters so we have the execution on the system but otherwise there is another exploitation which is you can inject arbitrary classes in the code so you can use Java reflection to patch in the system and to have an influence on all the technology of the SPPR otherwise we also have the verification of privileges in the second class we are in the session service so so we use getLoginSession which gives a list of login session data of all the users with their name and their IP and their client ID except that the client ID is of the user we can have the client ID of the users who are admin and so we can use this client ID to have a second reference to the client security service to have a second privileged session we will continue an attacker an attacker a security service second time and to have all the private information of all the users of the system with the password passwords and their information to sum up we have both of these these two vulnerabilities allow access via HTTPS and firewall and so everything can be turned in general all the rmi communications are encrypted so the user names and the password in text are not encrypted it's much more critical ah sorry the rmi communication is not encrypted it's in text so all the password password no no session protection so if someone can attack an SPPA user and the traffic between this user and the application server he can recover a user name and a password password password password and he can reuse it to do operations on the system so he can even change the password of the user so I talk a lot about the names and password password password password we will try to understand how you implemented in the system Alex, you go hello everyone I will continue the discussion here we can see how the authentication works sorry on the previous slide we could see how the authentication works and now I will explain how it works so when the system starts it reads two files users 1.xml and pdata 1.xm so the user store and the store of the passwords respectively so the first one is xml simply it's the message 24 it's and it's going to be xml and the content content of this xml is presented on the slide is used to calculate a hash and create a password and so at the bottom you can see the verification of the password in pdata it's it's typical it's a typical for Unix and Linux there is a number of iterations there is the salt the salt and the salt that has been added it's the salt is the same for all users it's hardcoded so here there is something to to extract the password of course because it was developed you can see what it comes out and the tool can be used of course to check the password and it's public this tool there is a line so the app the first thing it's an attack which is absolutely huge it includes different components secondly it's with distance connections whether it's going to have maintenance it's going to have OPC someone else you should check it anyway and the last thing is the attacker can impact the generation of electricity he can stop the salt he can change the value generated electricity generated and he can simply collect information on the generation and so all this can be done from the application server that was just the app now we will continue we will talk about the purpose of the server is to automate in real time and it depends on the architecture of the electric center the role of the server can be different we saw three roles the first is automation it can be confused because it's on one side the function and the server name but when you look at the configuration and the information we realized that everywhere it's almost the same hardware and the same software and so we have we used this classification which is less confused for us at the same time on the other side it's also a little different from the model classification meanwhile the role of automation that the server is responsible for interaction with the input and output with the input and output modules that will control the equipment like for example the turbine the generation of electricity and so on the second role is communication and the third role is connection with third-party software and third-party systems so with the protocols so we will have the bus protocol 1004 and the last role is migration which is to connect to the older SPPA installations like the legacy system like SPPA 2000 or so so the role of automation the automation of the server the automation of the server will be utilized on an S7 on industrial PLC we will talk a little more about each role now so we will launch so we are based on PLC so we have security problems we control the the machines so it will directly access so there is no security system since it is the lowest level in the attack models without any changes or updates of a PLC you have to stop a technological system so there is always the configuration even the systems there is no security since the S7 is not super secure there is not a lot of information about the S7 protocol on the internet so we had to analyze it so it is a protocol for SPPA for the PLC so there is an exchange of data in this protocol it is a pretty simple protocol and maybe its description is available on the internet but we could not find it so we will show you its structure so there is some security mechanisms on this protocol so we can do attack models on it so for the protocol analysis we developed a dissector which is available so during the security test on the configuration one of the main things that we checked is the unauthorized access in writing in memory so it is defined by the position of their selector so during previous research by colleagues the privilege matrix was determined so it was able to show different things on the PLC configuration so this analysis is available in our repository so it is just a Linux so at the beginning it has downloaded some files from the application server so jars, script, batch and some other files to execute already we use a java vm a ptcperc vm the ptcperc vm is a system of completion to solve the problem of decomposition we wrote a PHP script which performs the reverse confirmation on the automation server we have RMI servers so in the case of the migration server there are rpc servers which are also RMI servers so the security problems on the industrial system so there is a possibility of spoofing the telechanged files from the server on the other hand there is a problem related to the use of default identifiers which are used especially via ssh for access to the server so cm admin and cm password then there are vulnerabilities that affect the rpc services that allow them to access files or rce and then there are vulnerabilities that affect the migration server if you have read about vulnerabilities that affect the rpc so this service contains a method where the first parameter defines the action that will be executed which will allow access to the content of any file present on the system and also to write content on files on the server so we will also be able to pass a file which will be able to be executed later so that's all for the automation server to summarize a little bit so it can be based on plc or pc industrial in the plc case we have specific security problems and industrial systems it's just a boxlinux with which it can be added so far we haven't talked about network equipment so during our research we were able to meet a great variety so firewalls, switches etc we tried to summarize all these information and to make a way for the network function for our SPPA the same type of equipment can be found in equipment provided by other sellers so these equipment they are often affected by many security problems it's a bit linked to the fact that usually there is no particular configuration at the beginning and they must be able to work just as soon as we start them so we're going to have problems such as the obsolete firmware identifiers by default a lack of configuration in terms of security all these things are quite common for network equipment and these are common security problems for the industrial systems I think that's all now we're going to discuss so the electric systems are huge we saw this, we saw that we saw a lot of things and everything is planned on this slide it's that we have problems in hardware, in applications in the configurations, in the mechanisms the use we don't even need to go over it we can impact the power generation it's what kind of disaster could be used you have to arrive with a attack like this so with blackouts this is not what you can do the distribution of electricity is not the problem of the generation it's a problem for the to verify that there is enough capacity in the network so what we're talking about is how we can so we're going to talk about the turbine but we don't have access to the real turbine so we didn't find someone who wanted to lend us one to destroy it but the point is we're going to we're going to guess and the turbine is huge it's huge, it's a mechanic which is breaking by itself and so there are different use modes and in certain use modes it's going to degrade even faster and so you can have a PLC of exchange but a turbine maybe less so it's not there but it's not huge so what we tried to do was to understand how we can help the electric power plants to find all the problems and analyze the structures and understand that all the installations are the same and that we can we can simplify our analysis to do what a simple engineer can do his test on the security connect to this place yes connect there and do a simple test and then call Siemens to fix something and we don't need consultants who are too expensive you should be able to do it yourself of course it's a DCS it's like other industrial installations there's a lot of similarity and it's the same places that hurt there's a good document of the IC 62 443 which that describes how the users should talk to the integrator and the seller and how the seller should confirm we suggest to each operator to read this document and talk to your seller because now it's a little different from our days between the different sellers but it's a document that describes how to talk with other entities of course read this slide read the whitepaper that comes out in January call Siemens call the systems change your password very easy at least to reduce the surface attack and there's a lot of in a network SPP 3000 it's Windows so we can also set up a monitoring and talk with your V6 admin to at least have the logs or Java applications you can't check with Windows systems but at least you can look a little bit on your network what's going on and finally it's not a problem of a Siemens DCS it's the same problem with our other seller that we didn't mention here we're going to which is we're going to make a big whitepaper with everything we found in it with our recommendations with our security assessment with our tools with other tools so for the with our sector with the we talk to Siemens we say thank you they did a great communication work with us and their production team who develop themselves what the seller does is if you are if you are if you operate a an electric center and they are and so they try to talk with their client to change the password to update and to sensitize everyone and so it's not just Siemens it's also the operators who can do things that have an impact on security so that's about it thank you very much do you have any questions thank you very much for this great talk we have 3 minutes for the questions if you have questions tell us if you use the the problem of the ears you go to the microphone 3 your question on the internet so with with the vulnerability identified would it be possible to take control from the internet I repeat with the vulnerability identified would it be possible to obtain control from the internet no that's the good news we we have tried to see they are more or less protected from the outside there are certain accessors and we are not going to talk about the internet we are going to talk about the network next question microphone 3 hello I also have an electric power plant and it's not very good for the atmosphere as I realized so my question is can you tell us where is the red button to stop the power plant I ask it for a friend we did not think like that but yes specifically if you are if you are an engineer do not go with the internet other question on the internet no question no question thank you very much and applause so thank you