Hello, my name is Simon. I'm a professor of quantum technology here at the University of Oxford and I want to make a video not about quantum technology, although that is a cool topic and I would like to make some videos about it. This is a video about the movie Iron Man and whether the stuff we see happening in that movie, all the fantastic fight scenes and crazy stuff going on, is allowed by physics. Is it a yes or no from the point of view of physics? On whether we what we see happening could happen. I think it's quite interesting because it causes you to think about some of the physics and maybe the answers are a little bit surprising. I should say it's not my intention to upset anyone and spoil a movie for them. I really enjoy the movie. It's a great movie. So I'm not trying to spoil it I'm just trying to think about it But if you would rather just watch it without someone like me criticizing the science, then maybe don't watch this YouTube video I'm really not trying to, you know, start any fights here. I just want to enjoy thinking through the science. And what else to say? I think that it's a good movie to choose because Iron Man is a technology-based hero. All his abilities and powers are supposed to come from the technology that he has built and that is in our own near future. In fact, it's set in the now But it's a little bit more advanced than we have because we understand that Tony Stark, the guy who wears the suit in Iron Man, is a genius And so he is able to go a little bit further, a little bit deeper than the rest of us can and effectively bring tomorrow's technology into today But we're not talking about sort of Star Trek level technology and doing things that physics would consider or that our best understanding of physics says is impossible. It's just making, you know, 10 years time, perhaps, technology possible now. So I think that's why it's a really interesting one to choose. So what can we say about this checklist of abilities that the suit has from the point of view of physics? First, the toughness of the suit, its strength against impact I see no problem with that. Physics allows for super strong materials compared to the stuff that we have in everyday life. What about the physical strength that he has? Maybe 10 or 100 times stronger than a normal person. He's able to do some pretty amazing stuff I don't see a problem with that in the sense that you can certainly imagine lots of small powerful electric motors Which are woven close to the surface of the suit because it's quite, you know, type fitting really So that it boosts his own power. I don't see a problem with that What about the flight of the suit? Can he really have little rocket boosters? I mean they're not rocket boosters or something else, but effectively little propulsion systems built into his boots and into his palms I don't see a problem with that either. You could imagine engineering something we understand today like a jet engine You could imagine engineering a tiny version of that If you had enough power to supply to it, it could accelerate air or and push it back out And that would give you the boost that you need. So I don't see any deep problem with any of those things What about the computer system? Well, the computer system is the most impressive thing about the suit in some ways because it seems to be way ahead of where we are. His Jarvis AI is fantastic Compared to anything that we can talk to on our phones or in our computer systems today So that's a huge credit to Tony's genius that he's got that working But again, no problem from the point of view of the physics as far as I can see because Of course the human brain is approximately as powerful as that and that's allowed by physics You would just need a system like that that's in some other way Distributed in the armor so I don't see any problem with any of those features But we should think for a minute about the power systems Oh and before that I should say of course the little weapon systems like little missiles and things Nothing to say about that from the point of view of the basic physics Why not? Why not have tiny missiles that are very explosive? Can't do it yet But there's no reason to think that we're anywhere near the limit on that Okay, so then the power system. This is interesting because if we tried to build Tony's suit today We could have a go at most of the components But what would stop it from working especially stop it from flying is that we would not be able to build a small light Power generator that would get the whole thing going and that's actually part of the film is that's the missing ingredient when the When the villain of the piece tries to make his own suit He can't get that bit working and he has to steal it from Tony So that is a really interesting heart of the suit really it's a miniaturized arc reactor We're told now an arc reactor isn't a thing from the real world So we can only guess at what how it might work But we can use physics to put some limits on what any kind of power system will do and check that it all works out So fortunately in the movie Tony actually tells us How much power his prototype his first version of the arc reactor produces? What could it generate if my math is right I don't know I says Three gigajoules per second. So we're told there that it produces three Gigajoules per second now That's actually a ton of energy a gigajoule per second is also called a gigawatt Three gigawatts is what a power a nuclear power station might produce a good-sized nuclear power station would have that as its power input So Tony has somehow created a little thing that fits in the palm of your hand But at least for a while can produce the same power output as a nuclear power station But hey, he's a genius the question is is that okay with physics can such a small thing in principle produce so much power Well any system that is going to produce power will have to consume something so have to consume some fuel It's only a little gadget so we can't consume a lot of fuel Let's suppose that it consumes one gram of fuel So a little amount of fuel which we wouldn't which we easily could fit inside the thing And we wouldn't even notice whether it been consumed or not how much power or how much energy I should say can we get out of one gram of material a little bit of matter in the maximum allowed by physics The answer this is really really interesting. It uses an equation from physics that you have heard of I don't care if you never even did high school physics You have heard of the equation that I need to use now to figure out the limits on what the arc reactor could do That equation is Einstein's equation e equals mc squared So there the m is the mass of the fuel which is one gram and the e is how many Jewels of energy are going to be released, which is what we want to know So we need to work out this c squared number the speed of light squared speed of light in meters per second Which is what we should be using is Actually three with eight zeros after it 300 million meters per second fast enough to go around the earth seven times per second By the way, so I'm pretty fast and we've got to square it We've got to square the speed of light. So it's going to be huge Let's do that three threes a nine will call that 10 and those eight zeros when squared will give us 16 zeros So c squared is about one with 17 zeros after it So one with 17 zeros after it so huge number When we multiply that by our little amount of matter, which is one gram That's a thousandth of a kilogram. So we'll knock off three of our zeros But that's still going to give us 14 zeros. So that's how many joules of energy we could get out of a gram Or how much is that in gigajoules where the gigajoule would be one with nine zeros So it's still we still got five zeros left. So that would be a hundred thousand Gigajoules Tony tells us that his device can produce three gigajoules per second So that would mean it could run actually for according to physics the absolute limit that a device was remember We're not working out Tony's we're working out the limit that he mustn't go above But so our device that's perfect generator allowed by physics would actually be able to produce the amount of power that Tony wants for What 30 about 30 33 thousand seconds before it would even use one gram of fuel So the bottom line Tony's generator can produce a ton of power same as nuclear power station But such a little thing could produce that much power quite easily The limiting machine that would be allowed by physics could actually do way more than his gadget So the the final thing we should just quickly check on the power side of things is is this enormous amount of power Actually enough to do the stunts that Tony does the incredible things the suit does can this huge power level We suspect the answer is going to be yes, right because the output of the nuclear power station But let's just check so one thing to do one way to check it would be today How much energy does it cost for Tony to go up 10 kilometers straight upwards 10 kilometers from the ground? That would give a rough sense of the energy costs of zooming around and as long as it's small Compared to the amount of power coming pouring out of his arc reactor, then we're going to be okay with things all right, so How can we work that out? We need to know how heavy Tony and his suit is or how much mass it has let's say Tony is 75 kilograms Let's say the suit is heavier than him But not crazily much more heavy because you know we don't get the incense the sense that it's a massively heavy thing So let's say that Tony plus his suit is 200 kilograms and then it's going to go up by 10 kilometers, which is 10 to the power of 4 meaning 10 with four zeros after it meters So we need to do a little multiplication, which is the weight or the mass of Tony and his suit times the distance upwards times The gravitational force on each kilogram that he's having to oppose by going upwards And so that number is 9.8. Let's call it 10 So we need to do 200 times 10. That's 2,000 times 10 to the power of 4. So that's 20 million Joules 20 million joules is how much energy quite a lot of energy, but it takes to take the suit up 10 kilometers and Maybe His his thrusters are not a hundred percent efficient It's not really possible to convert energy into thrust with 100 percent efficiency Let's even say even though he's a genius Let's say that those aren't those boot things are not great. So let's say they're 10 percent efficient But remember he's his reactor is producing three gigajoules per second So it seems fine even that extreme maneuver is within the capabilities of the arc reactor and given that I think we can also allow him to use some of his boost power You know in an offensive way and run his computers and I think there's no problem with the super strength Feets because if you look in a let's say a factory which manufactures cars You'll find big robots that can basically pick up a car a car and twirl it around those robots will not use Anything like the output of a nuclear power station. So Again, he's good. He's good in terms of having plenty of power for all the feats that we see him achieving So do I have any problem with it at all? Yes, I have a problem with it But it's it's and it's such a surprising thing if you it can't already spot what it is It goes all the way back actually to the toughness of the suit the fact that it seems to protect him from any kind of Impact we need to talk about that because it is actually a problem this the thing is what I'm going to explain It doesn't make any difference how strong the suit is the suit could even be stronger than the suit We see in the movie which does take a bit of damage even a suit that didn't take a scratch During all the violent things that Tony experiences would still not protect him because of a thing called momentum conservation Which must be obeyed and the suit can't help with So it goes like this. Let's think about the scene where a tank shell comes along and hits Tony in midair Now for simplicity we can say that Tony was essentially still Compared to the tank shell which came in from the side and whacked him now the suit is super strong So it stops the tank shell from tearing a hole through his body and going out the other side In fact, the tank shell will not be able to touch Tony's body because it's inside this invincible suit But still that event would kill him as we see it portrayed on screen. Why so the problem is this That the tank show comes in very fast It would actually come in I look this up at something in excess of a thousand meters per second comes in very fast And then it hits him now momentum conservation insists that after the event the momentum must be the same as before So that means that Tony will have to be moving backwards and the shell Well, it could bounce off But that's worse for Tony because then he has to move back even faster to make up for it So the best thing for him would be if the shell carries on moving with him at the reduced speed after the impact But even then he will have had to suddenly have accelerated very dramatically in a short amount of time This is the problem. So we could work out what that acceleration would roughly be if we say that the tank shell is 5 Kilograms, which again, I looked up and that's apparently about right for that kind of tank Tony himself We already said is 200 kilograms And so what this means is that if we say that the tank shell comes in and then they move off together They will move off at roughly in the ratio of their masses So then a 40th, which is the ratio between the 5 kilograms and the 200 Roughly a 40th of the speed that the shell came in So a 40th of a thousand meters per second, which is 25 meters per second. That is quite fast. It's not shockingly fast It's about 25 6 is one big 150 so that would be 1.5 kilometers per minute Which is the sort of speed that you might do in a car down a motorway So it's nothing shocking But what's shocking is that Tony has to go from a standing start to that speed in a tiny amount of time How much time or basically the amount of time that it would have taken the Tank shell to pass through him if it was going to pass through him, but it didn't hit him So then they're going to move off together. How long would it have taken to go past him? Well less than a thousandth of a second because it's doing a thousand meters per second coming in and He's less than a meter in this direction. So let's say a thousandth of a second being generous to Tony He's got a thousandth of a second to go from nothing to 25 meters per second That is an acceleration albeit very briefly he has to have an acceleration of 25,000 meters per second per second because he did it in such a you know only a thousandth of a second Okay, so an enormous acceleration. Would the human body be able to tolerate that? Well, my body is tolerating a force that's trying to make me accelerate right now Which is the force due to gravity. It's trying to make me accelerate downwards at 10 meters per second per second But it's not succeeding. Look, I mean, it's it's trying to pull my arm off right now, but my arm is fine So we are designed or evolve to put up with that kind of force and it's absolutely fine If you go in a fast car You know round a corner or if you go to a fairground and experience forces there Then you might experience several G several times gravitational force in one direction or another going around a steep corner Or on a roller coaster or something like that and again, your body can put up with that for a while and it's fun If you think about though very dangerous levels of force then I gather from looking online that a hundred G a hundred G Which would be a force that would accelerate you a thousand meters per second per second Would be the kind of force that may be fatal for a human or you may luckily get away with it If you were in say a car accident with those kind of forces were involved So that would be a thousand meters per second second would be around the Lethal level that you might or might not care where with but we're asking Tony to do 25 times more than that 25 times over what we might say is the human limit approximately the human limit now He's a fit guy and he's a hero of the film But I don't think it's reasonable to think that his body can withstand 25 times what other people's bodies can withstand So what would actually happen the shell would come in it would push against the suit Accelerate the suit the suit inside of the front of the suit would push against Tony and accelerate him But at a rate that his body cannot withstand the forces going through his body and trying to you know accelerate His internal organs and so on are communicated through his flesh and bone And they just would not be able to put up with that So what he would end up doing is being horrible say it But he would be pasted against the inside of the suit at the front inside of the suit So that when he crashed and they came and opened up his suit He would just be Tony paste on son the inside of the suit And there's no way around this the suit It the strength of the suit does not protect him from this now a similar problem comes When he hits the ground It's less extreme, but it's still I think if you run the numbers would be likely a fatal impact when Tony hits the ground We see him in the cup of the scene sitting the ground He seems to go into the ground only about a meter and so in that very short distance He has to decelerate from doing a very rapid speed like something like 25 meters per second to zero and again His body won't be able to put up with those internal forces. It will be pasted So the problem with the suit is that it can't protect him from momentum conservation Is there anything that you could do about this if we were trying to design Tony's suit for him and fix this problem What would we do? Well, this the solution that might be allowed by physics most easily would be to have thrusters That are not only in his boots and his palms, but which are distributed over the outside surface of his suit So that when he is about to receive an extreme acceleration event the thrusters can fire and cancel it So for example when the tank show is coming in at the moment of contact Thrusters would need to fire out the back of his suit to cancel out the incoming force And it would still satisfy momentum conservation by the way because the momentum would now be in the form of let's say the air Molecules or whatever it is that he's shooting out from his thrusters They would be moving off very rapidly in that direction so that he doesn't have to and similarly when he's about to hit The ground back thrusters could fire and allow him to more gently come to a halt But we don't see that in the movie. He doesn't have that feature What else could he do? Well, then you're into some if you want it to be like it is in the movie without the extra thrusters And he is just okay through these extreme acceleration events Then what we need to do is go is appeal to physics that doesn't really exist physics like in Star Trek for example, which we're supposed to be now 500 years in the future or something and We see a starship accelerate from a standing start to extreme velocities even faster than the speed of light because they can do that And what what doesn't happen is the bridge crew all gets pasted against the back of the bridge because of the extreme Accelerations this doesn't happen and why doesn't it happen because they've got something called inertial dampers Which is not a real thing It's just the writers of that story putting in something a mystery technology that makes it okay But Star Trek can have mystery technologies because it's so far in the future that they are Breaking the laws of physics as we understand it They do things that look to us like magic, but I don't think we can reasonably allow Tony to have that kind of thing What it would need in fact this inertial dampers type thing it would need inside his suit He would have to have a technology that can push on every molecule of his body with just the right force to make it accelerate Or decelerate correctly without having to pull or tug on the molecules around it because that's the problem It's the problem is that hit the insides of his body are not designed to communicate those kinds of extreme forces So we would need to take that problem away by directly pushing on every particle of his body with something like a fake Gravitational field would do it so if we could fake gravity if we could generate a gravitational field inside his suit That's just in the right direction at the right time. Yes, we could accelerate him safely But we don't know how to do that We don't know if that's even possible We would think that it isn't possible and I don't think Tony has got that kind of tech because if he had you know He'd be he'd be in a Star Trek type universe and not in the one we see in it So that's my that's my objection which is you know You could say it's a like nitpicking objection, but I think it's really an interesting one Basic to the physics of any kind of scene where you see someone Surviving something because they seem to be inside of inside some kind of protective enclosure. So there we are I think you know, it's a high score for the credibility of the Iron Man suit If you allow that he's clever enough Tony Stark is clever enough to build such an advanced power generator The miniaturized arc reactor then the abilities of the suit all make sense. They're fine The only problem we have is just in a few scenes I think actually in three key scenes in the movie where an extreme Acceleration event or deceleration event happens and we are given no reason to think that Tony has such advanced tech That he can prevent the Consequences of that acceleration event on his own body for even though it's inside the suit So there we are. That's my only criticism of the science really or the only one that's sort of fundamental in the physics And I hope that doesn't like annoy you because I mean, it's a great I'm not criticizing the film. I think it's a great film It's correct to have these dramatic scenes like a tank shell hitting him and so on It's great fun to watch but the physics of it is not is not possible as presented Okay, so I would like to make at least one more of these where I look at a different superhero and I'm going to look at if I do it I'm going to look at the superhero that is my favorite growing up as a kid and still my favorite Which is the Incredible Hulk and you might think that I'm if I was critical of Iron Man I'm going to have to be super critical of the Incredible Hulk But not necessarily maybe maybe not but not necessarily not necessarily Because I want to focus on just what physics allows and not for example what biology allows So we'll see but I try to be kind to the Incredible Hulk because he is my favorite Okay, if you want to see that video then do go ahead and subscribe And if you found this one interesting drop it a like or drop a comment in if you disagree with me You think I've been unfair or too fair to Iron Man and you think you know the science in it is either Perfectly fine or absolutely rubbish then leave a comment. Okay. Thanks a lot


Hey everyone, Rob Greenfield here and this is the off-the-grid zero waste haircut and beard cut That's right. No electricity, no waste, but a nice clean cut Now a lot of you know me as a guy who kind of lives off the land or sort of an earthy simple living kind of guy, but I still like to have a nice clean Head of hair and beard. I just don't like to have a big bushy beard This is about as long as I like to get So I have a few tools for that whether I'm in the city or in the countryside number one an old-school vintage manual trimmer and I've adapted a simple guard to have the right length and then a scissors for working around the edge That's what I need to keep nice and trim Plus one other thing of course a mirror I can do without but this thing makes things a lot easier So check these out. These are just like the electrical ones that most men use today except one big difference No electricity. I just squeeze the handle and that does the work Now most electricity today is made from fossil fuels This on the other hand is using an alternative renewable energy human energy and most electronics Today they don't last very long. They're designed for the dump This thing on the other hand It's probably about 50 years old Now that's the off the grid part, but where's zero waste come in well Most people when they cut their hair all of the hair goes to a landfill Goes to the garbage can and then the garbage goes to the landfill There it's designed to stay in the landfill not break down So this stuff here can actually last for decades or even longer Even if it does break down it doesn't become part of the earth It doesn't nurse the earth on the other hand when I cut this hair and I let it fall to the ground it just goes back to the earth to actually nourish the soil Plus here no cleanup is even needed not a broom a dustpan anything But if I'm in the city what I can do is just sweep up the hair and then put it in the compost Me I'd rather be free than in the landfill It's really easy to never think about it But in one lifetime the average person cuts their hair hundreds of times and each time They send a part of themselves to the landfill or back to the earth and Speaking of waste disposable razors That's another thing where you can use thousands of those or hundreds of those in a lifetime with this every single time No garbage is created and if you think about it you can spend Thousands of dollars going to the barber over your lifetime or you could put those thousands of dollars into bettering your community and Sure doing it on my own. I don't always have a perfect cut. Sometimes I make mistakes But what's important to you looking perfect or looking out for the earth? Now last thing What about fingernails? That's another thing that we might not think about I have a standard nail clipper But same thing I go outside I Clip away and those fingernails go back to the earth just like the hair does now a lot of us Haven't really thought about these things before maybe not once but these little things They really do add up for better or worse the question is which do you want better or worse?


Alright, so welcome everyone, also the people on Moodle, welcome welcome, we are just going to start. So today we will have a lecture about standards for analysis. So there's a lot of like kind of standard things in my mathematics. So I just wanted to run through a couple of these things. So of course again please register for the exam if you haven't already. Very important, I haven't gotten a feedback from anyone yet that they were unable to register. So I guess everything's going well there and I hope that everyone can find it in Agnes and can register. So if you have any problems registering then send me an email and I can contact the Prüfungsbüro and of course if you don't register, I think it's two or three weeks before the date then you are unable to participate. So if you're planning on getting a grade for the course then do register. Alright, so the overview for today, first I wanted to talk about different types of files and some formats which are commonly used in bioinformatics like comma separated files, FASTA files, FASQ files, GFF, VCF and PET files. There's a lot of different other formats like SOM and BOM and these kinds of things but they are generally very complex. Someone asked me to add PDB to it and I wrote a PDB parser like a couple of, well not a couple of years but I think around a year ago or something and the format is described in like a 160 page document so I thought that that was a little bit overkill so I don't think that it makes sense to kind of discuss that. So after we've discussed all kinds of different file types and stuff of course we will talk about some coding things because also there are some standards so I wanted to talk about the difference between things like unit tests, regression tests, integration tests and these kinds of things and then I want to say a couple of words about test driven development and agile development. It doesn't play a big role in academia but sometimes it's good to know what's out there and how you can kind of structure your programming work in a more structured way. I wanted to talk a little bit about documentation so code documentation mostly and the different types of code documentation which are out there and then I added a part which I stole from the R lecture or stole, well it's not stealing if you made the original one right so I kind of auto plagiarized it for myself and that is about R packages so I'm going to teach you guys how to make an R package and hopefully we will be able to do that after the first break, second break but of course like always before we start let's have a look at the assignments. I hope everyone did the assignments, I'm not so sure everyone did because I didn't get any questions. I did get someone emailing me but not about the assignments, that was about wanting to have the time travel paper from Kerry Mootis which is a really nice paper to read so if you're interested just send me an email and I also had someone who wanted to know about one of the papers written from freely available geodata sets so I also send that so I might actually upload the Kerry Mootis paper to Moodle and make it obligatory. I was very busy this week and I will do it this week, yeah that's what I say as well when I don't want to do stuff. We could actually do a prediction about that, did Commando do the assignments this week yes or no but we're not going to do that but no, yeah it is important to do them, it's the only way you can learn programming and since like last week the lecture had a lot of programming of course if you want to learn how to program you just have to sit down and get stuck by your head against the wall Google for half an hour and then if you can still figure it out then just send me an email but at least I will just go through the answers and I will also put the answers online and I was a little bit lax, I think I only uploaded the assignments on Monday or Tuesday so it's also a little bit my fault, like I should, so if I don't upload the videos and the other stuff to Moodle tomorrow so before Friday 5 then do send me an email to remind me because if I forget then it just has to pop into my mind again and usually it pops into my mind when I start preparing the next stream so that's generally Tuesdays or Mondays. Alrighty then so the assignments from last week were about gene expression analysis so I will show you guys my notepad plus plus window, this is of course not, this is the engine so again when you do some coding and when you are coding always make sure that you add a little header, so just say in one line what is the content of this file, who made it, hey you could add a copyright statement or when it was created but at least have something on top of your file so that people can see who did it, who they should contact with questions and what is in the file. Alright so the first thing that I did was load the preprocessed core library, it only comes up in like question number of five or six or something but the thing is that when you write code then the way that I always structure it is first do loading of all external libraries then setting my working directory and the reason why I do this is because then you can directly at the top of the script see which libraries are required. So if you would put it like all the way when it is needed which is probably somewhere around here, hey if you would do something like this then it is kind of hidden, so you don't want to hide things if you want people that use your script to know what they should install beforehand. So where did I store the data, well the data is stored in my case in this folder like D drive, project, lectures, blah, but of course on your hard drive that will be different. And then there were three files that you should load in, I think the first couple of questions could be done without loading the probe annotation matrix but there you need at least the array data in the arrays. So again here R provides this really useful retable function, of course you have to look into the file right so you take the array data.txt and you have to open it in notepad++, let me actually do that for you guys so you can see how the file looks like. So let's just open up the array data. So what I generally do is when I open up the file and I always use notepad++ or an editor which allows me to view kind of these kinds of symbols right, you see these little arrows here that means that they're tab characters. So if you would have spaces then those would be little yellow dots and the tabs are these little arrows. So it's directly obvious that this is a tab separated file and so of course here we have to specify that the separator used in this file is a tab. There is a header right so if I look at array data then I see here that there's kind of a header description which looks different from the other lines right so the other lines they have like numbers in them or sequences but here you don't have that. One of the things that you can see here is that the first column header is called sequence and then when you look a little bit further down you see that sequence is actually the second element right so it's not the first element. So that means that there are row names in here so you have to specify row names is one right because every row has a name and the name doesn't have a column header so in this case setting row names to one. Had the same thing holds for the arrays and here I added this called classes is character I don't know exactly why I did that let me see oh yeah because I don't want these numbers to become numbers because they are animal identifiers that we use right it's an individual ID so had the problem here is if I would say that load it in and convert it to the best possible type then the last column would be converted to either integers or factors and I don't want to do that I don't want to use factors in this case so I'm going to specify call classes and head that allows you to specify for each column what the class is but I'm just going to say character which means that it forces all of the columns to be of the character type and I'm loading the annotation matrix same thing just open it up view into the file see that there's a header has a set header to true and the separator is a tab so let's go to R and load it in and make our first box plot and this already I think is kind of a little trick but first let's go to R and let's show you guys how these different data files look so we're just going to read them in the array data takes a little bit of time and you can then use the head function to show like the first couple of lines right so we see here that indeed it has kind of loaded it incorrectly and we can see that there's like a bright corner let me see that's annoying so let's switch to R so you can see that it have when I when I look at the head of the array data I can see that it's more or less fine right it has row names because these things are repeated for each of the rows that I have if they would not have been repeated then there would be another column and then it would have been named sequence but that's that's incorrect and then here you see that there's a bright corner and a dark corner so those are the positive and the negative control of a micro array so the bright corner is always on and gives you kind of the maximum intensity value for each of the samples that you had and then the dark corner is the minimum intensity value of the micro array so these are just built in positive and negative probes which are either always on or always off has so you get an idea of what the range the dynamic range of the of the micro array is all right so let's look at the arrays and just show a little bit so this is just the description of the array so it has the original file name this is just the file that we got from the company so you can see that it's done in September 2009 and these kinds of things we have something called comp ID and this is the ID that the company assigned to our sample we have the strain so the strain here is either the Berlin Fett mouse or an F1 which is a cross between the b6 and the Berlin Fett mouse and if we want to know how many strains there are we can just say well show me the strain column and then it will show you all of them so we see that there's also b6 ends in there then we have two types of tissue so we have ht which stands for hypothalamus and we have gf which stands for gonadal fat and then of course we have different individuals which are marked by the individual ID from the mouse house so that's kind of how this file looks like and then we had the annotation matrix so let's show you a little bit of the annotation matrix so the nice thing about ours that you can just type in like a couple of letters and then press the top key and it will automatically like fill in the Meravigliata thank you for following me thank you thank you I love that feature that it gives me a little sound effect when someone presses the follow button I'm glad that I built that in so yeah thanks for the follow I hope you're enjoying the lecture so looking at the annotation matrix we see that it has the different probes on there where they are located on the genome what the which gene they are targeting and then with the gene a little bit of description and then the location of the gene that that is being targeted so that that's the kind of the dimensions right so the the arrays themselves so had this this file here it has like columns and rows so the rows couple to the annotation matrix and the columns they couple to the arrays and then when I want to make a box plot of this file of course since the first column is called sequence and it's not plottable right you can't make a box plot of characters so when you make a box plot and you just say box plot of the whole thing right so I would take my array data and I would just say box plot array data then I would probably get an error or it will make a box plot but hey you see that it will give me an error so kind of to prevent that let's go back to notepad plus plus so kind of to prevent that I'm saying well when I look at the files I see that the individual or the columns of the of the array data file are made by the company ID because the company gave us back the IDs so I can select from the arrays the company ID and then use this to index the array data and this will of course only select the real samples that we send and it won't select the sequence column so when I then try to create a box plot and I do LAS is to to kind of flip the axis and then of course I'm able to make a nice box plot or a nice box plot it's a box plot that kind of shows me what's going on so it takes a while to make and what you see is that that there the numbers range from around zero right so the lowest intensity is in in the order of zero and you see here that the highest intensity varies a lot for the different arrays and today you can see that these are all very very big numbers hey you don't really see a box for a box plot you just see that there's a massive amount of outliers which are on top and this of course has to do with the fact that when you get intensity data scanned by a laser then this intensity data of course is not a normal distribution and normal distribution would show up as a normal box but since the boxes are more or less all the way squeezed to the bottom you know that most of the genes are off right because there's no intensity and the genes that are on they look like outliers because they're they're very very intense compared to the average of the genes that are off alright so then the first question or I think the second question is is or the first question was use the dim function to view the different dimensions how many probes are there in the array data file so if we look at the array data file right so we just saw that the probes are on the rows so we see that there are five thousand five hundred fifty five thousand eight hundred and twenty one probes on the array there are seventeen columns one of them is sequenced so there are sixteen samples in our data file take a closer look at the header of the array data file is there any structure to the column names so let's see so we just look at the head so is there any structure to the names well they're kind of is because the names are made based on the tissue so HT and then the animal ID so that's kind of the structure and of course there's a sequence but the sequence that doesn't and the sequences of course just the sequence of the probe alright question number two use the plot function to plot the HT 2010 column of the array data file so I just skip that in the answers but we can just select of course this column and then we can use the plot function to just plot that so this will take a while as well and then we see here the image right so here we see the here we see the fifty five thousand probes and for each of the probes we see the intensity so you see that most of the genes are actually not expressed or most of the probes are more or less not giving you an intensity signal and you see that the genes which are expressed are all the way at the top but of course when you would make a histogram of this so instead of using plots you can use his for a histogram hey you would see that this is definitely not a normal distribution it's it's more of a kind of Poisson distribution but it's not really Poisson because they're not real numbers alright print parts of the other files to your our session and figure out what the different columns and rows are so we already did that select only the columns containing data from the array data file in other words do not plot the probe sequences use the buck plot function to visualize so that's the one that we did before right so how we take from the arrays so this is the description of the arrays we take the company ID column and then we use this ordering and select the individuals from array data and then make a box plot using this LAS is to and I use the LAS is to just so that the names of the of the different samples are like this right so that they are vertical and horizontal alright then the next question is look use the log to function in our to transform the micro array data save the resulting matrix into a new variable called log to array data all right so let me switch back to note that plus plus so the way that I did this is to just say oh I didn't save it in a new variable I just overwrote the old data which of course is also possible but you have to reload so I generally don't like these kinds of structures right so what I do is I take the the same as what I did the box plot on before I do the log to transformation and then I put it back into the original data frame but of course this is a destructive operation so that means that once you do this you cannot go back and that's of course not the best way to do this so a better way to do this would be to follow the assignments and say well I'm just gonna store this in something called log to array data and then I'm just going to plot the log to array data and now it's not destructive now I just make a new variable one of the drawbacks of courses is that I'm now just copying the whole matrix with 55,000 probes and 16 samples but depending on if your computer has a lot of RAM memory and this is not an issue but it could be that if you're very low on RAM memory that it's better to do the destructive operation and just copy paste it back in to where it came from because of course this doesn't duplicate the data because you just override it so I'm just gonna keep the answer that I had so I'm just gonna do the destructive operation normally I wouldn't do that but for this case like it doesn't matter too much and so we take the log to and then we store it back and then we make a box plot again so let's go to our and show you guys how that looks and now of course we see a very common structure so now we see that the minimum intensity is slightly above zero that's just the way that that it works had there's no real zero here because had we're looking at intensity so if you shoot a laser at something you will always get some intensity it's it's never exactly zero that is a really really nice emoticon test this out I like the emotes that which has all right but then what we see here is that the first four samples have a slightly higher mean compared to the next four samples then we see another four samples which have a slightly higher mean and then so it seems to be that the average expression of the array is kind of coupled to the to the tissue that we're looking at so that directly brings me to another point when we start doing the normalization procedure it might be that in in the brain in the hypothalamus there are more genes active than there are in gonadal fat right because if there are more genes active then you would expect the the average expression to be a little bit the average expression to be a little bit higher and so but you see that some arrays have outliers and you also see that that every array is slightly different from the other one so have to get rid of this and because this might be due to the fact that there's some real biology going on but it might also be due to the fact that there is some that there is some technical variation right it might be that the hypothalamus samples just had slightly higher concentrations because extracting DNA from hypothalamus might be more efficient than extracting DNA from gonadal fat or they're doing DNA to RNA so it might have a biological origin but this variance might also be technical so of course we want to apply a normalization to get rid of these kind of technical variation that we see or the large amount of variation all right so create a boxplot of the unnormalized data we did that and save the results yeah so create a boxplot of the unnormalized but log 2 transformation transform data so what do we observe we we observe that there seems to be kind of a slight kind of correlation between the fact that hypothalamus is slightly higher expressed or as a slightly higher average expression meaning that there might be some more genes which are active in hypothalamus than there are in gonadal fat but of course we can't really do anything with with this observation because we have to normalize our data first so for the normalization we are using the pre-processed core library so let me load the library so just that I have it ready and then the next question is load the library normalize the data using the normalize quantile function save the resulting matrix in a new variable select an appropriate name so let me see what I answered there so let's show the notepad plus plus window so here I do the normalize quantiles right so I do have a little header telling me what I'm doing and I'm actually just not adhering to what I said again because I'm using a destructive transformation again right so I'm again copying it back into the to the original matrix yes so you could save it in a new variable which is advisable but for some reason when I wrote the answers I didn't care too much about destroying the original data or being able to go back one step in the analysis all right so let's use this function and do the plot again and then we see this right so after we've done the normalize quantiles we see that every array has the same minimum value it has the same maximum value and all of the means are the same but also the standard deviations surrounding these means have all been harmonized to be exactly identical to each other can you explain the difference between normalization and Windsor rising so Windsor rising is looking at your data values and then fixing little errors like comma errors right so for example if I'm measuring a mouse and I'm measuring a mouse to be 10.1 centimeters then sometimes it ends up being written as 101 or 1.01 in the data right because a lot of times when you're in a lab and you're measuring things then you're just writing them down on paper and that of course creates like a 5% error rate because every time that you write something down you can make a little error so when you're looking at your data right and you see that one of your values is a mouse which is 10 times bigger than the average or 10 times smaller than the average you always have to start wondering is there a comma error right so a comma error means that you're then Windsor rising this one value into the range in which you expect it to be right you don't expect one of the mice to be a hundred centimeters right that would have you would have noticed that during the experiment so so Windsor rising is more or less a looking at your data by eye and blotting out values that you either don't like or putting values in the normal range so normally Windsor rising which was invented by the Earl of Windsor is something that you do to kind of get rid of mistakes in your data well normalization is something that that is done not to get rid of mistakes it is done to harmonize data right so in this case we are normalizing and like I told you guys last time there's two different ways of normalizing and one is normalizing internally and the other one is normalizing between and so since the idea here is to compare these arrays to each other if one of the arrays would be from 0 to 10 and the other array would be from 10 to 20 at the intensity values then of course every gene on between these two arrays would come up as being different and of course that's not true because like if we look at a grenade or fat sample and we compare it to another grenade or fat sample then we don't expect all of the genes in the genome to be different right so so normalization generally is a technique that allows you to get rid of unwanted variation which sometimes means that you get rid of some real biological effects as well well Windsor rising is looking at your data and really removing things which are obvious errors one of the things which also falls under Windsor rising which I would not advise people to do but I've seen people do it is that when you have your data measured right and you have a missing data point to put in the average on this missing data point and I would never do that because statistically speaking it does not really make sense to replace a missing value by a value which is which is the average and it can actually hurt you in the long run but Windsor rising is not really a normalization technique it's just a technique to get rid of them but when we now look at our data we see that that everything is the same mean but of course before we said the observation is is that the hypothalamus might have more genes which are active than compared to the gonadal fat so we we by doing this normalization step right we might have removed some really really important genetic variants or some real biological phenomenon and this is of course one of the one of the issues with normalization since normalization is kind of a blunt tool you can normalize away things which might be really interesting so you always have to be careful when you do normalization and you have to have a good reason to normalize but test since we are wanting to compare arrays with each other we can't have the fact that one array has a completely different scale than the other ones so again create the boxplot of the now locked and forward was it look normalized like in the lecture yeah this looks pretty normalized alright question 10 when looking at the other files we observe that this data is coming from multiple individuals and as multiple tissues measured first we want to know something more about the relationship between the different arrays so the first step what I generally do when I get back microarray data is to look at the correlation between the different samples right so the question number 10 is use the correlation function and create a correlation matrix save the matrix in a new variable and then question number 11 is use the heat map function to plot the correlation matrix as a heat map what do you see in the heat map all right so let's go back to the script yes so I'm doing this in one go because I don't want to kind of store it and define a lot of variables so I'm using the correlation function so I'm correlating again columns which have data and here you see that I use the Spearman method so I think that most people know that correlation comes in several forms so you have Pearson correlation and you have Spearman correlation Pearson correlation you can use when things are normally distributed if you have a distribution which is not a normal distribution you would want to use Spearman correlation because it uses a rank method so it's not sensitive to outliers and why did I choose Spearman well that has to do a little bit with how the box plot looks right so if we go back and then we see that it is not a normal distribution right the distribution on the bottom of the arrays is different from the distribution on the top of the arrays so you see that there's more or there's a normal I can show you this in a different way as well so let's just make a histogram of one of the arrays right so just take the first one and make a histogram of it and then you see this right and this is definitely not a normal distribution so you don't want to use Pearson correlation to compare these things you definitely want to use Spearman correlation and you can see here very clearly that some of the values are between 0 and 1 which is a little bit awkward because that's just the way it's a locked to transform data set but you see that most of the genes like more than 15,000 genes on the first array they are not on they are not working at all so they are off and then you see that there's a kind of an expression to the to the upside so there's not a normal distribution because there's not a normal distribution I have to use Spearman correlation instead of Pearson correlation in this case it wouldn't matter too much I think I think the answer would be very very similar but be aware that correlation comes into two or three different flavors depending on on what you on on on what distribution you are looking at all right so let's do the heat map of the correlation matrix this might take a little while but let's just do it so we see this picture right so and from this picture we learn a whole bunch of things because we can now say how our data quality was and if everything went well so the first thing that we observe in the clustering and so if we look at the two the dendrogram on the top or on the other side and what we see is this that on average hypothalamus samples are a lot more similar to hypothalamus samples than they are to go native fat samples right so we learn here that we didn't switch one of the cups we didn't on accident we didn't put a hypothalamus sample into a go native fat sample and we didn't or and we didn't swap a go native fat sample with a hypothalamus sample right so we can directly see that and we see that there is that that it's very clear that that the samples are assigned okay right they could all be wrong right all of the go native fats could actually be hypothalamus but at least internally there's an internal consistency here why didn't you use candle correlations that are spearmint for the non-normal distribution well spearmint is for is a rank-based method so it's non-parametric Pearson is also not a person is a parametric method so you can only use it for and candle towel it's also non-parametric one and it's slightly different but I think candle is preferred when you have lots of missing data right I think that's the difference between so Pearson is parametric spearmint is non-parametric optimized for when everything is there and then you have candle towel correlation and that is optimized when you have large amounts of missing data but you could use candle as well our provides three different ways of correlation but since there is no missing data because how you shoot every little dot with a laser so every little dot gets an intensity yeah I thought candle is better when you focus on rank-based method I would say that probably here dr. Spearman would disagree with that statement and your statement would be preferred by candle and that's of course if it's a it's a flavor right like there's many different you also have like by variant correlation methods you have so there's like hundreds of different correlation methods and all of them have their own advantages and drawbacks so I'm like someone like me would prefer a easy method and the easy method or the thing that what what I like a lot is that spearmint correlation is Pearson's correlation on ranked data so they are kind of interchangeable in a way but it's it's up to you which correlation you prefer and it also depends on your distribution but normally my kind of way of looking at it is if I have a normal distribution I use Pearson if I have a normal then I tend to always use Spearman but you could use candle as well and they probably are more or less the same right we can we can actually just check that so let's show the correlation matrix like this that's too big so let's look at the first five versus the first five right so this is using Spearman then we can use Pearson and then we can use candle I think I can just type Ken and here you see the drawback of candle correlation right it takes a long time to do this it is a very heavy method it might be more accurate but like the computational time doesn't really weigh up to the we're still waiting so which is not bad right if you're a bioinformatician you're quite used to waiting but this takes too long like it this is not a very valid method to use let's get a call I already have a coffee so we can we can wait a little bit but in this case if you would do all three methods would give you more or less the exact same same answer I'm thinking that this is just going to crash our it is actually already in a non-responding state so oh you can't see that you can see the header of the thing but alright so it's just freezing freezing freezing yeah cancel it that's easier said than done because it totally crashed alright so let's not do that let's use the task manager to get rid of the thing and then open up a new our window which is just like pain in the ass and have to resize everything which I already did at the beginning but I have to redo now so let's resize it like this and then resize it and then resize it a little bit more still like a couple of pixels and like yeah there we are alright so let's reload all of our data and make our heat map again so Kendall correlation seems to be preferable when you have non-parametric data and your data set is small I think that would be my my my observation here but the heat map shows us that indeed to have conatal fat is similar to conatal fat hypothalamus similar to hypothalamus and so we learned that we didn't mix up any samples one of the nice things that I that I that I think you can see here in this this is that there does seem to be three groups right because we know from the data that we had like the BFM is we had to be sixes and then we had to mix between the two and we can see here that these here are these three samples here which cluster together they are probably the BFMI they could be the B6 as well and then you see here another three samples why am I missing a sample here one two three four five six seven eight huh that's interesting and but there seems to be some grouping as well which we would expect right because there's there's three different types of animals in our data so we we would expect to kind of see these groups back as well but these groups are not as clear so we can't really discuss about that we might be able to see it when we would look at a single tissue and instead of here plotting the name of the sample plotting the species of the sample hence so then we might find out that these three are indeed related to each other we this is the pattern that we would have expected from a data set which looks like this all right so the next one is for each probe on the array calculate the overall mean on the log 2 normalized data using a for loop all right so I think that this was one of the harder ones although the answer is more or less given I don't know why I gave you the answer here that's an interesting one I already know what's wrong yeah I'm showing you guys the answers for the bioinformatics for the for the R lecture so the R lecture is slightly different questions that's not that's not what I want but anyway let's let's just do question number 12 right so I'm just gonna say q12 and I'm just gonna copy paste the code so what is happening here so I'm going to want to calculate the overall means for each of the probes right so I'm doing overall means is no or I could do something like this so is an empty vector and I did not do any computation yet and then I'm going to go through the rows of this normalized data which I didn't store so I'm just gonna have to store it here so I'm just gonna take out this part because I overwrote it right so I'm just gonna declare this variable so I'm going to go through each of the rows of through each of the probes I'm going to calculate the mean and then I'm going to add this mean to the list of means that I created so and this will go through and it will calculate all the means so let's go to R and then define that so I didn't define the variable yet so I'm just gonna do that here and then I'm just gonna calculate the overall means so this will take a little while right because it has to do this 55,000 times but then in the end we will get an overall means and of course if you're doing this on a laptop it will take some time but like I said that's bioinformatics for you all right so we get 50 warnings so we definitely want to check the warnings so it tells me that for some of the computations of the mean that it did it encountered non-numeric or logical value so it returned an NA so we can we have to figure out where that happens right so if we look at the overall means then we see that it actually happens everywhere so what went wrong so the question is what went wrong so first let's look at the input data set that we're using and then we see that here we should be able to calculate a mean of the first row so we do a mean and then it gives me NA so for some reason when it loaded in the data set it did one of these R things right and R is really good at screwing up your data especially when it comes to things like factorials and numerics so if we would ask for the class of this then it would say data frame that's logical so the first number here is actually a numerical value so the reason why it doesn't do if I would unlist it then I would just get the numbers and now if I would ask the class it's still a numeric value so it's a little bit weird that I cannot calculate the mean directly because I'm expecting that it would and if I do an unlist on it just to make sure that it's not a data then it works that's kind of interesting so the mean function here doesn't really work because it internally stores probably the numbers not as numbers but it stores them probably as factorials so to prevent that I'm just going to change the code a little bit from the code that we had and I'm just going to say well I know that when I unlist them then it then it definitely takes them out of this little sub matrix that you have which just as one row and then we just have values so let's try this again let's go to R and now use the unlisted version to see if we can calculate the means and that should be actually a lot quicker because computing and getting this warning is actually slowing it down quite a lot so it should be relatively quick in calculating all of the different means of each of the probes all right so no warning so everything should have been going correctly so here we see the overall means so we just look at them like this and now of course when we plot these we can see again for each of the probes and we can now see the mean expression of the probe and this already looks different when compared to when we looked at an individual array right so it seems that that genes and because now it seems that more genes are active and this is of course because the genes which are active in the brain generally tend to be not active in the fat and the genes which are active in the fat tend to be not active in the brain right so we see that on average there's much more probes that should have been expressed if we make a histogram of the overall means and we should now also see that this is the case that there's still a lot of genes like 14,000 which are not active but we see that this this this hump here is much bigger so there are more genes which are active if I just look at each of the probes and see if they're active in one of the two tissues all right then the next question is choose two groups which you think might be interesting for example the F1 gonadal fat and the BF my gonadal fat we can select from the array annotation the company ID or the atlas ID of the arrays in this group we can use this to extract the correct columns of the data so it had this little example so let's just copy paste the example into notepad plus plus so we go here oh I already had this so I did have that one all right so here what we do is we we say which of the arrays strain is F1 and which of the tissues is gonadal fat and then of course I can extract these from from the arrays right because now I'm getting the indexes in the arrays and then what I'm doing is this then I'm taking these rows and I'm only taking the company ID right so F1 gonadal fat samples so I'm saying take from the strain column only the end the things where it is F1 and in the tissue column it has to be GF same thing here from the strain column take BF my from the tissue column take gonadal fat and then take only the company IDs of these individuals all right so let's see so let's go to R show you guys the R window right so we have array array F1 and then we have a GF right so these are the two samples which so there's two F1 samples for gonadal fat and then if we look at the Berlin fat mouse BF my gonadal fat then we see that there's actually four gonadal fat samples of the Berlin fat mouse all right then we can now use the array F1 to extract the correct columns from the array data do this for both groups so of course when I look at the array data and then I can select the first 10 rows and I can select for example the array F1 gonadal fat sample and then it will just give me these two and of course I can do the same thing when I look at the BFMI right so in this case I get four columns all right then the next question was a more complicated question because now I wanted you guys to compute different things in one go right in the first four loop we only computed the mean and remember that but then now the question is calculate for all of these probes the mean of the two groups the group that you selected for example the BFMI gonadal fat and the F1 gonadal fat the log two ratio between these means and the p-value using a t-test and the t-test is going to go wrong here because t-test you have to have I think at least three samples and I think that that will go horribly wrong but Ted the let me show you guys and so the first thing that I'm going to do is I'm going to create variables which will hold my answers right so I'm going to create a mean F1 gonadal fat right so this will hold this will hold the means 40 F1 individuals gonadal fat going to do the same thing for the BFMI so I'm going to say F mean BFMI gonadal fat I have my log two ratios initially I haven't computed anything yet so they are null and then I'm have my p-values which initially are also null right then I'm going to go through the number of rows of the array data and then I'm just going to take this one element take the F1 individual take the same probe take the BFMI individuals calculate the mean and then store this as mean one and mean two then I'm going to divide mean one by mean two and take the log two this is then the ratio and then I'm going to do the t-test which probably will fail for all of them because I have only four individuals versus two individuals but I'm going to do the t-test anyway and then I'm going to calculate p-value and then I'm just going to add the things that I computed to the correct variable that is going to hold it for all of the probes so I'm going to add the computed mean to the means of the F1 the computed mean to the means of the BFMI the ratio to the log two ratios and the p-value to the p-values and we're just going to run this I don't think that the that the t-test will work but if we're lucky it does if we're unlucky we get just p-values which are all NA but at least like we try to answer it so probably if we would have taken two different groups if we would have taken the B6N versus the BFMI then probably it would have been three versus four or something like that and this of course will take much longer because now every time that we look at a row for this row we take the F1 individuals we calculate the mean we take the same row take the BFMI individuals calculate the mean calculate the log ratio then do the t-test so hey it's going to be kind of heavy and it's going to have to do more computational work than in the first loop so again we can take some coffee and we can just let this run and I'm very impatient so I'm just going to quit it halfway through so I'm just going to press the stop button and we're just going to see how far we are so we were almost done so I was a little bit too impatient because we stopped it at 52,000 out of 55,000 doesn't matter too much because now the idea was to create the volcano plots by doing the minus log 10 p values against the log 2 ratios right so we have the log 2 ratios which look like this and then we have the minus log 10 p values but we only start p values so these are all the p values so now we have to do the minus log 10 of the p values and we want to plot this so we are going to put the p values are on the y-axis so we're just going to say y equals minus log 10 and x equals and that's the ratios log 2 ratios log 2 ratios so the x-axis log 2 and the y-axis is going to be the minus log 10 p value and now we see kind of the characteristic volcano plot right you see kind of the zero line so here there's no difference in the ratio and here we see the minus log 10 of the p value right so which genes are interesting genes to look at those are the ones which are all the way over here and because these have a big difference between the F1's and the BFMIs and they are also very significantly different right this is 1 times 10 to the minus 6 this is 1 times 10 to the minus 5 right so the interesting genes are more or less in this area when we look at the up-regulated genes and this these genes are more or less the interesting genes when we look at the down-regulated genes all right and then there was an additional question to color the dots of the volcano plot by the distance of the origin did I do that no I didn't color them but you you can do that right you can you can just compute the Euclidean distance from each of the points towards the 0 0 and then hey you can color them based on that error 404 what's not found to test this out of what are you missing I will make you guys a really nice colored volcano plots in the additional question but this is more or less how you would look at it came up on R on my laptop you got a 404 error on your laptop on R that seems strange like R generally doesn't give 404 errors is that a 404 for loading the data because that can 404 with a file not found then you have to download the proper files and set them in the correct working directory you do have to extract a zip file though I had some people in the past trying to directly load the zip file into R that does work because R it has this zip I think or G zip so you have a zip file extract but you can use this zip function to directly load from zip files if you wanted to but I would advise you to just extract the zip file and then go and set your working directory to where you extracted it but this is more or less how you look at microarray data right so and it's just a very very basic introduction let me look the recording is now going on for 54 minutes and we only discussed the answers which is perfectly fine so let's go back to the PowerPoint so for everyone watching on Moodle you are going to miss one of the amazing breaks so the break is going to be pandas I think I think the first one is pandas all right so I will stop the recording


Welcome to another edition of RCE. Again, this is Brock Palin. You can find us online at RCE-cast.com and you can find our entire back catalog as well as subscribe and your favorite podcatcher RSS reader. I also have here Jeff Squire from Cisco Systems, one of the authors of OpenMPI. Jeff, thanks for your time. Hey Brock, here we are another episode. Weather's turning nice and that means that SC deadlines are probably approaching, right? Yeah, yep, yep. So a number of SC deadlines are starting to come up very quickly here. So if you're planning on submitting something SC, be sure to get your proposals done and get those submitted. All right, what do we got for today? So we have with us today Dr. Gorgolecki of the Stanford Reproducibility Center. So Chris, why don't you take a moment to introduce yourself? Well, thank you for inviting me to be here. My name is Chris Gorgolecki and I'm a co-director of the Stanford Reproducibility Center for Reproducible Neuroscience. And my goal in research and science is basically to get more neuroimaging data publicly available to more researchers. So why is reproducibility important? Well, basically in the business of trying to discover the nature of the universe, we perform lots of studies and we want to make sure that if we say something today, we can use the same data tomorrow and arrive at the same answer. It's basically the the fundament of doing science. So in the recent past, there's been a number of high profile retractions in the neuroscience area. Was this you've blogged about this a little bit and written about this at reproducibility.stanford.edu. Was this part of the motivation for creating this center or was the motivation before that? The motivation was before that. The issues highlighted in that paper were basically bringing up that certain analysis methods are flawed and insufficient and the paper itself was a little bit sensational, sensationalist in nature. So the situation is not as bad as it is described. But it does show that it is beneficial to be able to take existing data and reanalyze it again with newer methods or corrected methods. And that's where we come in. And this is where the ability to replicate the results in a formalized fashion is something very useful, especially with historical existing data. So now the the most easiest one to talk about, and I think the specific article that you're referring to is the 15 year old bug in the FMRI algorithms for analyzing data and whatnot. How how does something like that happen and how is it not found immediately? I mean, the situation was not as dire as it might seem. There was a confluence of different things and and the paper did find some problems with a certain piece of software. But there wasn't the explanation of all of what was the major finding of the paper. But to answer your question, how things like that can happen? It's basically the researchers bias. So if you get results that confirm your theory or if you get something that is quote unquote, statistically significant, you're less likely to go back and search for a problem with your analysis. However, if you don't find what you expected or your results are not so significant, then you go search for a bug. So if you have a bug that biases your results and gives you higher rate of false positive findings, fewer people question those results. And it's harder to to find something like that. The other thing is that in neuroscience, human neuroscience, so human neuroimaging analysis is very complex and has many different stages. And this is just one of those many stages. And it's hard for a single individual to go through all of that code and debug it. And so I think you have answered my next question already is, you know, why exactly is this hard? You know, why is reproducibility hard? And if everybody is using, you know, the same tools and the same things, then everybody gets the same results and it makes it difficult to realize that there's actually a problem in there, particularly when the results are what you expect. Is that a correct characterization? More or less unpack what reproducibility really is into two things that you need to have to reproduce someone's results. You need to have the data, which is the crucial and very, very important part of it. And you need to have their code or codes, whatever you're going to call it. And that that second part can actually get quite complex because, especially in scientific software, codes or code is evolving very, very rapidly. We have a plethora of different versions. And there are a few papers in your imaging that are showing that things that we didn't expect to influence the results, such as, for example, the order in which you're going to link a certain binary with mathematical libraries can influence final results. Same with the system that you're going to run your analysis on. So we have a lot of different variables and a lot of different pieces of software produced and delivered by different groups that at the end give you the final results. So we somehow have to capture all of that to be able to replicate the results. So why did you focus on neuroscience? I assume that's your area, but why put so much effort into reproducibility for neuroscience specifically? That's a great question, and two takes on that question is, like, why people care about neuroscience at all. And that's kind of an obvious one to answer, because the human brain is the greatest mystery in the world. And we all have one and we all wonder how it works. So it's definitely a great problem to work on in general. But why trying to solve a disability in this particular domain instead of in a general way? Well, we're trying with everything we do. We're trying to be very, very practical. We're trying to be as close to the real scientists and asking real questions as possible to be able to relate to their problems and their limitations. For example, we don't want to ever be perceived of coming from this high horrors and telling everyone that they should use this particular procedure, or everyone should have 100% coverage in unit tests and every single script they write, because this is unrealistic. And in the same way, we want to provide practical solutions in a particular domain, because this way it's easier for our audience, target audience, which are the scientists to understand it and apply it to their research. So kind of in conjunction with my prior question, this is really more focused on reproducibility, not a strict check of correctness, right? Because correctness is also incredibly important in scientific research and discovering the nature of the universe and the human brain and all these kinds of good things. But what you're trying to do is make sure that people are doing it in a reproducible way and the correctness check is a secondary, but a different effort. Is that correct? Yes, that is correct, because correctness is an incredibly hard question. And you want, of course, we're doing some work trying to evaluate methods and trying to come up with the best way to analyze data. And part of what the infrastructure we're building is is aimed at providing neuroscientists with easy access to the best methods, even though those might be computationally expensive or difficult to deploy. So in that way, we're trying to improve the quality, but you're absolutely right. I can have a completely reproducible technique that is completely wrong. So on the center's website, you stay with the goal of harnessing high performance computing to make mirror. Neuroscience research more reliable. How does high performance computing make research more reliable? Well, basically, if we can give people access to high performance computing in one single place, we treat it as a resource, like basically like water in your tub. This way, we minimize the variability of what people are using to execute their code. And it's also is treated as a carrot. So in other words, as remember when I said, for disability, it comes down to access to data and access to methods that were used in the original study. So part of our goal is to increase access to the data and we're using high performance computing as a sort of a carrot, as a reward for sharing your data. So our scheme that we operate or soon will operate on there is, it has a very simple principle. You upload your data into our system, you can run all of those cutting edge method using high performance computing and in return after a certain grace period, that data will become publicly available because without having access to the data, there's no way someone else will be able to replicate the results. So let's talk about the data repository in a minute here, but I still wanna touch on the secondary problem that you've identified, this very hard part of, I wanna run the same program and get the same results and assuming that I have the same data. And you said something fascinating there in your last answer that you want to provide access to tools and potentially even have a single service where people are running all their stuff so that I think one of the implications is that you can have apples to apples kind of comparisons of results. But how do you account for even the drift of technology over time? So assuming that like you get all neuroscientists in the world to run on this particular HPC cluster, well, that cluster is gonna last say two to three years and then they're gonna upgrade it with the next generation of technology, whether that's the next processor chips or the next generation of numerical simulation libraries or whatever that therefore may give slightly different results, even with the same programs and the same data. How do we wrap our brains around that? No pun intended. Woo! Well, it's a great question. And it sort of points at the fact that sometimes we don't want reproducibility because we have improvements. It's again, back to the old, it's reproducible, but it's wrong. It's reproducible, but it's imperfect. It's reproducible, but it's suboptimal. So obviously if you wanna do apples to apples and keep on the bleeding edge, you have to rerun the same analysis with new versions of software again and again. And we're also just to kind of a side note, we are simplifying our problem just to the software layer or actually just to the software layer above the kernel. So we do assume, and I know this assumption is not 100% correct, that across different architectures and across different hardware, we're gonna get the same results because if we were to actually guarantee the same hardware, that would be just too much to tackle. But software is plenty. So we've got our hands full. So what tools have you created for the neuroscience community to encourage this method of operation? So I can kind of describe this ecosystem from kind of top down. So our vision, as I said, is provide science as a service in exchange for making your data publicly available. And that service, which hopefully we'll launch in publicly this summer, allows you to upload your data, edit it online. And the first step for possibility is being able to take snapshots, immutable snapshots of your data. Because to be able to go from results through methods to the data we have to know in what state this data was and dataset change, people have subjects, et cetera, et cetera. So we have this science as a service platform, but we have to populate it first with data. So to be able to do that efficiently, and this is why we're doing this within your imaging, we have to come up with widely acknowledged standard for describing and annotating your imaging datasets. So someone from a lab in Texas could upload the dataset and our tools would understand this. And someone from Helsinki would also upload a different dataset and our tools would also understand it. So we developed this standard, it's called Bids or Brain Imaging Data Structure. And that allows us to interface between the data and the computational methods. And then we started populating the service with tools, with basically those logs that people run on their data. And to make this widely used and appealing to people, again, we had to have the best methods across the field. And it's a heterogeneous field, it's not that gonna ask every scientist and they go, gonna point to this particular software package and say like, this is what I wanna use. And this is the only one. No, there are multiple competing packages, there are multiple ways of analyzing this data. So again, we reached out to the community and we had this workshop last year where we brought the leaders of methodological development in your imaging. And we asked them to take their existing tools, put them into software containers and make them work with our input data format. Having those tools, now we can, in a much easier way, put them on our platform and make all of this available to everyone. And on top of that, you're also building some of our own tools. Actually, we call those tools BidsApps because everything needs to be an app these days, I hear. So we are building some of our own BidsApps and Lab trying to make them as robust as possible. So this is different than the usual data management plan requirement of, keep all your data, keep all your code, zip it up somewhere and tell people where they can get it. You're going a little bit more in-depth in terms of trying to get people to do it in the same framework so that you can run it and other people can run it. And like you said, your tools would understand people from different parts of the field at the same time. Is that a safest difference between the other attempts at increasing reproducibility and data reuse? Yeah, we're trying to have this carrot rather than stick approach. People want to do science the best way they can. It's not that scientists are evil and all of them are just trying to pee-hack everything away. It's just the tools are missing. And we are trying to build those tools and we're trying to make those things easier. And here the carrot is processing and access to cutting edge methods. So basically if someone doesn't have the resources to run this very computationally expensive method that is hard to install and maintain and they have this option of just uploading the data to our servers and we're going to run this analysis for them. It's a no-brainer. They're going to do it because they benefit from it. And we take care of all of the reproducibility. The data is snapshot it. The method itself is in a container and all of the containers are strongly versioned. So you can always go back to that particular version that generated this result and to that particular version of the data itself. So we attract people by the methods they care about but provide this reproducibility for free for them. Can you give us a little more details on this? Like what kind of platforms are you going to allow people to run on and how much storage are you going to provide and where are you getting the funding for this kind of stuff too? Yes, so that's a great question. So this kind of research, in general it's not that easy to get funding for this sort of, we want to make science better research from typical sources but we're very happy to receive a generous grant from Laura and John Arnold's Foundation. And this is what is funding this right now. And this is mostly the development stage where we're building all of the infrastructure, all of the software around it. But the idea is that the architecture of the system will allow us to tap into existing publicly available high performance resources. So basically, there are plenty of programs where you can apply to get an allocation, for example, on exceed or on tag where you can basically get free cycles on a cluster. And this is how we plan to maintain this project in the future, basically paying for the compute. And most of the money right now goes into development of the infrastructure and the software. Storage will be a bit of a concern in the future. We haven't quite addressed the kind of intermediate storage but we have some interesting conversations with companies such as Amazon for providing free unlimited storage for publicly available data sets. And what is the... So all of this kind of leads to the... I'm a researcher, I have a whole truckload of data and I don't have resources to do the computation on or maybe I do have the resources to compute it. What would inspire me then to put this system, put all this data and whatnot into your system? Well, I mean, if you don't have the resources, then the resources would inspire you. But if you do have the resources, it's a little bit like having an external backup and a little bit of having a nice environment to do and track your computations. So I do hope that people who do have access to the computational resources might still opt to use this service because we're gonna provide this data management layer for them and otherwise maybe they just like us and that's what they're gonna do. Is there any fear that there will be more takers than givers? Like if the data is freely accessible, is there any fear that that will inspire some greedy actors to just take all that free data but not contribute any of their own? I see that as a benefit. We love takers. We love those, how do you call them, scientific data parasites. Yes, we absolutely adore data parasites and people who take freely available data and reuse them. This is why we're doing this because if they reuse the publicly available data, that means that more science is happening for less money. And this is a great saving. We recently estimated the data shared through the previous generation of this platform called OpenFMRI saved something around a ballpark of three million dollars. If that data were to be acquired, then novo, and this is how much it would cost but people just took existing data and did new science on it. So we are not concerned about having more people download data than people contributing data. So you mentioned OpenMRI. What is OpenMRI? So OpenFMRI is the first truly public database for task-based FMRI and later evolve into any MRI. And this is the form it is in right now. It's a very simple repository for storing human neuroimaging data. And one of the things that the way it stands out from other repositories that's completely public. So everything is public domain that is deposited in this repository. So there are no barriers or no hoops that you have to jump over to be able to access the data. And that's what makes us stand out. And we've been doing this for a good few years and building collaborations with journals and associations. So it's also a respectable place to deposit your data. And we're seeing more push both from journals as well as funding bodies to deposit data and to domain-specific repositories. So have you seen a shift in culture towards using not necessarily your environment specifically but towards using environments like what you've proposed? So I see the shift in sharing data. I don't quite see much of a shift in terms of using science as a service platform as yet. Although I hope it will come there. Like we see it is on a smaller scale. We see smaller services that require less data and less computational power, but that just provide it as a web application. So instead of having to download software, you go to a website and then upload a few megabytes of data and you get some fun result. Or maybe you do some simulation on the website. A colleague working in the center as well, Joachim Durnes, she developed a wonderful app for optimizing the design of an experiment. And that's all running on the web. So it's not as sufficiently expensive as the analysis that we're planning, but it shows that lowering the barrier of access to providing web interfaces is attractive for scientists. So do you have plans to move into other areas of science, either further more parts of neuroscience or new domains? We're probably gonna stay focused on human neuroscience, but that doesn't mean that we are at any level close to exhausting its breadth. It's a huge field. And right now we are branching out from MRI to other ways of acquiring brain-related data such as MEG or PET. So we're trying to combine these different modalities, but we're probably gonna stick to the existing solutions, but everything that we are learning here, we are very happy to convey those lessons and hopefully other fields could also learn from it the same way we learned from other fields, especially from genomics. And we have planned some integrations with more broader open science frameworks, such as the open science framework, which are not neuroscience specific. Now, jumping back just a little bit here, in terms of having giant repositories of scientific data available, particularly as this data accrues over time, does that actually make it more cool or improve the culture of reproducibility itself because this wealth of data, and then if you're bolting on the second part too, like, oh, well, the code that was used to analyze it is also freely available too. Does that just, you know, is the hope that the barrier to entry to reproducibility becomes significantly less and therefore inspires more people to do that? Yes, that is definitely the hope and the goal. And we already see that. So I can give you an example. There are a lot of methodological development. So basically someone proposes a new way of denoising data or normalizing data or analyzing data. It's gonna serve a purpose to answer a particular cognitive or neuroscientific question, but first we have to validate the method and we have to test the method. And that's in itself the scientific result. And without having access to open data, you could publish a paper showing, oh, I validated this method by running these analysis on this data, but no one would be able to actually replicate it on the same data because they will not have access to that data. So if they try to replicate it on a different data and they get a different result, they don't know whether it's the data or the method. And I'm not saying that replication on separate newly acquired data is bad. It's very valuable as well, but having access to benchmark data sets is crucial for methodological developments. Now going hand in hand with that, if you have access to open data, are you also developing these tools and the numerical analysis and computational aspects of this in an open source kind of way? Oh yeah, like we do everything in open source way. And it's wonderful. It's basically like joining this very big family of developers and hackers, especially now in the age of GitHub, it's just wonderful to see people collaborating on new features and new faces coming up almost every week on the bigger projects. So all of this is open source and it's not just open source in a way that, oh, here's the source, you can have a look at it. But it's basically open source in a sense that we have an open community of contributors and we'll open to new contributions and we perform the coding in an open way. Even when I would make a comment or have a comment on a line of code written by someone in the same office, I would still convey that in open way to GitHub because our goal is to build a community and work in this transparent way. So what license do you make the code available under and this is perhaps a slightly naive question, is the data available under the same license or a similar license? The license we use for code is usually Apache 2.0 because we are open to both open source as well as commercial applications. Whatever pushes the science forward. But software licenses do not have the right leglies to be applicable to data. And in terms of data, we're following the lead of other institutions and journals, such as CERN or nature, and applying the public domain license. So either PDDL or CC0. And that basically gives you the right to do whatever you want with the data. So you can use it for commercial purposes as well. So Chris, thank you very much for your time. We didn't get into a lot of the detail of how your software as a service for a reproducible neuroscience actually works as a technical contraption. But where can people find out more about the Stanford Reproducibility Center? So you can go to our website or at disability.stafford.edu, which is a high-level description of what's going on with very pretty images. Or you can just dive into the code and find us on GitHub. We are under the organization Poldrag Lab. And that's where most of the things are. All of the new imaging apps are available under the organization BIDS apps. And you can always shoot me an email. I'm happy to answer all questions. And if you're doing something similar or you have some comments, how can we be more efficient or better what we're doing? I would love to hear them. Okay, Chris, thank you very much for your time. Yeah, thanks for your time. This was great. Thank you for having me.