Let's talk not about XMR or Monero, but rather about Mirai and computer vision and the security of connected webcams and the embedded devices that go with it. That's about it, I think. Let's explain it better than me. Okay, that's right. So you can call me Michael. My name is Michael Schlofenben. It's a really long name. And it's because it's a German name. I spend a lot of time in Germany. In Munich, for example, I have a laboratory, a research facility, and I'm building a small fabrication facility in Munich at the time. I have another office in San Francisco, and I travel quite a lot because I do workshops. And the intention today was to have a hands-on workshop. So I see there are some laptops out. You're welcome to follow along with things that I will type later on. But from the four-hour requirement, we have so little time that it's going to be mostly demonstrative. It's going to be mostly demonstration. And so we're going to be taking a look at these topics and have as many demonstrations as possible with very little hands-on time, unfortunately. So let's, I forgot to get my laser here. There's a few different things that we're going to focus on. We're going to intentionally de-emphasize the cloud portion because it's so little open source. There's not very much transparency in typical cloud providers and computing machine learning things like IBM Watson and so on. So we're going to de-emphasize that a bit and try to keep inside the edges of the network and do everything locally with projects like, well, for example, OpenCV and Sphinx, Python, and these kind of types of things. And let's just get started then. This is a typical how a network typically looks when you're looking at it from the IoT or low-power device perspective, where you're possibly not always connected. You have an intermittent connectivity flow. So you do have some gateways that are sending your data out to maybe the cloud on the very end or other connected computers over data centers. But these are your devices here, your device profiles. And in some cases you might have the same thing on the other side. This is just a very simplistic version of what you might find. But we'll be looking at, for example, microphones, webcams, and trying to understand how the processing works at the edge, mostly today. So these are the typical computers that we use in a workshop. This is what we would have if we all had nice tables, if there was a power socket at each table. If we had a stable wireless network and an Ethernet network, then we could do things like with these new unit, what does NUC stand for? New unit computer, next unit computing. These are the typical things that we have in our workshop that we travel around with. And if you're expecting a four-hour workshop today, we won't have that. We don't have the facilities for it. But you can always contact me after the workshop. I'll put some cards up here. If you have a private question or a question you didn't get to, or if you'd like to try to schedule a workshop in your university or something, find that a good idea. I'll get the cards and put them up in front, or if you have non-relevant questions like the hardware wallets that I just passed out a while ago. So I'll try to have some. I don't have new cards, but I have old ones from Black Hat. So I'll put those over there. Those are the type of computers we usually use in the workshop. Today we're going to stick to the edges and everything inside the network, so do local processing, which we can do, well, demonstrate, at least on a notebook. So for computer vision, at least in the open source world, OpenCV rules the day. This is what we'll be using today with the time we have. This is an overview of maybe some of the history. It was a project started in 2009. No, 1999. I knew there was a nine there. And it grew slowly at first. It was just an Intel-based laboratory project until it kind of gained traction and was adopted by the open source community, in which has kind of overtaken the maintenance and support, until today it's quite a large project, OpenCV. It's relatively well known. And, yeah, it has a large and active community. There's a couple URLs there for anyone who wants to know details that they're very useful. In particular, there's a lot more than this. We'll get to that later and we'll have a couple demonstrations. These are the use cases. So you might have robots trying to follow a line or anything that involves vision, controlling door access if the door is open or closed. For example, what, I don't know, estimating how long a suitcase has been without its owner in a train station or how fast traffic is crossing a line. These are all use cases, very good ones for OpenCV. And what else do we have there? Motion tracking and so on. There's a couple more exotic use cases. Like if you build a robot which looks like a human, you enter a club and then you're trying to get a view of all the androids, criminals in the club, then you do something like this. I don't know if you caught that joke. So this is the terminator. So the terminator had computer vision in his eyes. This is a couple more typical examples, edge detection and finding the nodes of objects in order to play chess or whatever you have. And this is a typical Python program for OpenCV. We'll do a C++1. I think, let's do that now. In fact, we have a few more slides, but I'm going to blow past them because we just don't have time for that. It's obviously computer intensive. And here is, this is not good style, you know, like smushing lines together and using if conditions with no brackets and so on. But I did this so that it was easier to pack the text on one slide so we can see it in large text. And it's really not that difficult to understand. I hope the tabs worked out. So this is a basic kind of Hello World project for OpenCV. We're just displaying a file, which is a bitmap file. And once I just copy that over, let's see what happens if I try to demonstrate it. Let's try OpenCV, not Sphinxlib. So I'll make this a bit bigger. It's not on the screen because I guess I have to hit, what, escape? Okay, I guess, yeah. Then let me see. The best way to fix that is probably do this. That looks better, doesn't it? Okay, so, and then that means I can't see it, but you can. Okay, so then I will figure it out. So I can't see anything. Just tell me if I'm typing something crazy, right? So it's probably like display. No. So that's it. That's what we had before. And if I, let's see, is there a make file here? Not yet, but I can make one. If I make a build it directory, I go into build it.  What we had before here was a CMakeList, I believe. Is that right? And then if I take a look in there, there's some definitions because it's a CMake project. The OpenCV developers, they really like CMakes. So they make it easy for us to use. That's what it looks like. That's defining our project. And there's nothing here inside the current directory. So if I do CMake and then I give the location of the CMakeList file, which defines the project, then basically CMake will start working and it will, it will, oops, it will, let me just clear that. And then it will give us these files here. So remember before I listed the directory, it was empty. Now we have all of these things. And our make file, I don't know how long it is. It's a bit too long to show. Sorry. But what that means is I can type make and then it will make the project. How fast did it go? About 10 seconds. Let me, why is that cut off? I can't really, maybe if I can, it's not very nice to have it cut off. Something like that is maybe better. And after building, we now have, from that C++ code, we have a, what is it called? Display, something like that. Yeah. And then I have prepared a smiley something, smiley crazy image. So this is basically just showing the image, right?  So we know, for example, the CMake library is working, the CMake project generation is working. We know that we have OpenCV correctly installed and all of that. Anyway, so then I guess, how do I get out of there, escape? So then what do I do now if I want to go back to slides? Okay, there's the mouse. Oh, okay. This is really confusing with these slides. So we're about halfway through. We went there and there. And we talked about OpenCV use cases. We did a demonstration, C++. We saw the Python equivalent code for that as well. And usually we do this in a laboratory environment in a workshop. But there's no time for that today. All right, so this is back to the conventional network diagram that we have from the IoT perspective. What else we could do with a camera if we had a camera attached, which maybe we'll get to an OpenCV demonstration with a camera so that we can see, detect a smile, for example. So there is a microphone on most cameras. So let me just see how that might work. Voice to Text Recognition is one of the use cases that we can do with, not OpenCV, but with generic IoT edge processing for things like microphones and cameras. This is the history that we have seen with the type of recognition on the edge over the years. Since 1950 we've seen progress and we're getting pretty close to very full vocabulary systems today, even on the edge. One of the projects that is open source friendly is called Sphinx. So I'm not sure what CMU stands for. Is that Carnegie Mellon University probably? I'm making a pretty wild guess there. I could stand for something else. This is CMU Sphinx. I've been using it quite a bit and it's very stable. It works in Python and, well, you can see that it uses kind of a neural network type of processing scheme. So let's just go past this deep learning and let's do another demonstration for that. These are all things that could be happening on IoT devices like in cars, drones, and so on, right? So actually, I've already got this set up. Alright, so I can't see this either. But because of that I'm using Python modules for this, I can show what I've installed beforehand because we wouldn't have time to do it otherwise with the unstable network. And the ones that are most important are, let's see, speech recognition, for example, is here. I can't find, well, I can't find the mouse. So we have speech recognition. There is a pie. It's above. Pie audio is important. Pocket Sphinx is the actual library that does the text analysis. So I'm just going to Python and let's do the one. Let's do this one. So this is what we will do. And I hope it's going to use the correct microphone. So let's see what happens. Hello, Faustim. We are in Brussels. So now it's recognizing takes two and a half seconds. Hello, Faustim. Hello, Faustim. We're in Brussels. Okay, so Faustim is obviously not in his vocabulary. Because it's not in the dictionary. Right, exactly. So the one thing that's interesting, I can show the source code in just a minute. It's actually doing two different types of recognition. In this case, it's doing an edge-based recognition, which is all local. It's using the Sphinx library. And it took two and a half seconds to detect this quality of text with this resolution. Hello, Faustim. And we're in Brussels. So it got a few things wrong. The next one, it used cloud processing. So it used 50,000 computers in Iceland and everywhere else. And in a less-than-open-source manner. However, it did a slightly better job. In most cases, it gets close to 100%. Things like Faustim, I guess it won't. Catch that, but it does it a bit faster as well. Even with this, I'm using the same network as everyone else. So it's quite unstable. And then you're probably wondering what this looks like. Oops, I just... I just wiggled the cable, unfortunately. Oh, yeah. Oh, that's the one that came unplugged. So, intermission time. How are we doing with the hardware wallets? Has anyone stored funds on the... I accept tips. Somebody answered before me, so... Okay, yeah, you're probably wondering what the source code looks like. Let's take a look. So, yeah, this is not too easy. Right, so this is a bit longer. That's why I didn't put it on a slide. It's maybe 100 lines long. But it's only long because I'm using the two different kinds of analysis, right? The first thing we're doing is with the Sphinx library. So we're setting up the microphone. And we just do a listen for five minutes in order to gather benchmarks of the room and ambient noise, basically, so that we have a baseline of what's speech and what's not. Because you could use this, yeah, in traffic or in a silent car, all these different situations, right? So after we get the baseline done, we print this here, say something, and then we start listening for five seconds, I believe. And after that, we start with the Sphinx code and the logic. So that's... Where is it? Try, print, excuse me. Right, so that's here. You can see that at the end is a recognition logic. So this is quite a useful library. I do recommend it, especially here in open source context, where a lot of people enjoy Python. And then just a little bit below, you might be interested to see how that works with the GCP, the Google Cloud Platform, that starts here. And we're taking a timestamp there so that we know how long it took. And this is here, the important... Oops, not there. One line up, I guess. Yeah, there it is. So this is the one where we do recognize audio of using the Google Platform and it's going to tell us what it thought that we spoke. Yeah, all right. So the only thing left to do... Hello. Anyone see the mouse? Yeah, the tab is not doing too well. There we go. All right, so this is typically... This is what we cover when we have four hours. We take a look at the Google IoT Core, what type of services they have, the same thing with AWS. That's the console that you see typically with the GCP, the Google Cloud Platform. This is... Where is AWS? AWS is, I think I skipped it accidentally. All right, so this is the structure or the architecture of a typical AWS flow where you have Lambda functions and device shadows, all of these things which are useful for IoT devices. This is what we could be using, for example, in combination with the OpenCV library. The one thing we haven't covered now, but I think we won't have time for it, is the history of MIRAI, what happened there. If you're not up to date, the authors... There were two of them or maybe three, a third collaborator of MIRAI have been caught and they pleaded guilty in December, which is kind of an interesting part of the... the story, but the MIRAI, just to give a overview of that, the MIRAI botnet was one of the things used to attack Dyn and the whole Dyn-controlled network, a lot of other high-profile groups and companies were brought down for a long, long time. That was, I think, in 2016. We typically look at the source code and how it was controlled and what the scanning feature was and what IoT devices it attacked, like set-top boxes and webcams and how we might protect against that in the future. We can also develop our own webcam using Tesla boards or Beaglebone Blacks or Mino boards and use Node.js JavaScript or Python. That's always a lot of fun. That takes a few hours as well, but I think because we're running out of time, I would just invite questions of any kind. I think anyone have questions on any of these things, even MIRAI, yes? I remember you did the voice processing on the edge of the network. So why did you do that? I mean, normally in IoT, these edge devices are very, very small and then they send the data in the cloud and do cloud processing. Why did you do it on the device? Well, if we look back at the... Yes, I'm sorry. The question was why we did the processing on the edge internally on the local network, so to speak, instead of sending the data out to the cloud. Did I get the question right? Yeah, but you did both, right? I did both. Yeah. So yeah, I wanted to show the difference a little bit about the utility. If you do have a stable network connection of using the cloud for some of this processing, it gets even more obvious when you do computer vision because tiny devices, for example, BeagleBone Blacks, they just simply can't do any real-time smile analysis and they can do some face detection, things like that, some edge detection, but it's very, very poorly granular. So that's the reason I did it internally to our local network as well as externally. That's kind of in a nutshell. The typical devices we use are NUC devices, which have, I think, core i5 processors inside. It's very fluid. Everything works very well, even computer vision. In this case, well, I have an older network, an older notebook, and it works okay. I tried doing this on a BeagleBone Black yesterday and completely failed. So I couldn't even bring the necessary Python libraries onto the board. Any other questions? Well, I kind of missed the connection between Mirai and computer vision. Is the computer vision targeted to be used on Mirai control devices? Can you repeat that? Is somehow computer vision targeted on Mirai control devices? So the question is, is open CV or computer vision part of the Mirai scenario? And I would say no. The Mirai botnet was strictly TCP-IP. It was a network thing where they were just launching a denial of service attack via IP packets on other devices and bringing them down in that way. So there was no real computer vision or something like that. I brought that into this presentation because I consider it a future risk where if you have this type of DOS or denial of service combined with a analysis of the surroundings, either audio or voice, all of these things, temperature and vision, you could do a lot more damage if you consider the type of sensors that are in a fuel plant, a processing plant or an energy plant of some kind. And then bringing the computers down is one thing, but understanding the heat and exchange systems is another. If you manipulate those, then you can do a lot more damage. Any other questions? Could take one last question. So to link the two previous questions, basically from a security point of view, it's not good to send personal data like audio and video to the cloud, but then the cloud has the processing power to process it. So could there be a middleman where you send something to a local central unit that does processing, for example using Sphinx, the processing that things cannot do, but still you do not send the data over the internet to the cloud. So do you foresee a middleman in this exchange between things and the internet? Yeah, so the question, well, first of all, I think you answered your own questions. A wonderful answer. The question is, we have seen how it works inside the edge, the processing that's available and what kind of power you need, the resolution and the results, what quality you get as opposed to sending the data out to the cloud, which offers a much higher resolution and processing power. So it's quite a bit of a trade-off and part of the question is, if it makes sense to have a middle solution, maybe a hybrid of some sort, where you're doing things like using the Sphinx library for audio processing, I think that's what you're suggesting part of the time. And then if you have a stable connection or you have the need, it's expensive to go out to the cloud, so in all of those situations, you do the trade-off analysis and then send the data out selectively. That would be kind of what I consider a middleman scenario where your middleman is doing your work for you and storing or forwarding the data according to a rule set, right? Yeah. So it sounds good to me. It's based on trade-offs. If you want to send sensitive data out to the cloud or if that's too sensitive, you want to keep that internal, just do local processing. That makes a lot of sense to me, doesn't it? I think it's going to be the future. In my opinion, if that's the future of this type of sensory network and processing of optical and audio data, I just think there's too many scenarios to speculate on, right? Sorry, that's a bad answer but non-conclusive, I suppose. I find the hybrid approach very good and in every case, you need to look at what the security profile is and so on. Actually, antivirus are using this kind of stuff to analyze data, so it's based on confidentiality. If you need low confidentiality, you do it local and if you need greater confidentiality, you update it to the cloud and let the cloud give a better response. Leave it at that. Thank you very much, Michael. You're welcome.


Hi, I'm Angela. I know you're already using CESA, so I'm excited to show you how you can add a fun color theme and icon to your class. Let's take a look at CESA when you're signed in as a teacher. Opening our class app, you'll notice on the right a panel with different tools. You can pick a color theme and icon to personalize your CESA class. This helps students know which class they're signed into. To choose a class icon, tap the icon you see here. Then select from the options available or use photo to upload any photo you want. Once you pick, tap the X to exit. The wrench icon in the upper right will take you quickly into your class settings. Let's tap the wrench, then scroll to class theme. Tap the class theme and you can see a variety of colors to choose from. Pick your favorite color or a color that represents your class. Tap the back arrow and X to exit.


The study found that auditory detection of frequency modulation, FM, for low FM rates depended on the use of both temporal, phase locking, and place cues, depending on the carrier frequency, while detection of FM at high rates depended primarily on the use of place cues. The study also found that FM thresholds were better for the 2, than for the 20 Hz rate for center frequencies up to 4 kHz, while the reverse was true for higher center frequencies. For both FM rates, the thresholds, expressed as a proportion of the center frequency, were roughly constant for center frequencies from 6 to 10 Hz, consistent with the use of place cues. For the task involving detection of changes in the temporal fine structure, TFS, a bandpass-filtered complex tones, thresholds worsen progressively with increasing frequency above 4 kHz, consistent with the weakening of temporal cues. This article was offered by Brian C. J. Moore, Unvine.


It means a lot. I think the industry started honoring women sixth year in a row now. So it feels pretty much nicer to be holding this trophy in my hand, I think again in my sixth year. And it just gives you a lot of confidence and conviction to say it's no longer a man's world. I think there are lots of women in the advertising, marketing as well as the media fraternity. It's just that I think they need much more recognition because as far as hard work is concerned and putting in the effort is concerned, I don't think there's any dearth of that coming on because that I think biologically women are made that way to be putting in their best. But I think it's a lot got to do with recognition. The industry needs to recognize them more. So I think this is to that extent a very very good way of rewarding the women because it feels nice when good things are said about you.


I know that Africa doesn't always come across very positively in the international media. It's a continent known for conflicts, diseases, famine, all kinds of negative things. But I'm an African and I can tell you that Africa is a continent of greatness. In fact, Africa is a superpower when it comes to megafauna. With the last continent on this planet that still has its full array of what we call megafauna. Elephants, giraffes, hippos, rhinos, these gigantic animals which actually co-evolved with human beings on the planet. So you can imagine that it's very interesting that these animals have survived on a continent that is actually well known for being quite poor and maybe unable to deal with so many things. We've actually protected these extraordinary animals that are a global heritage. And why? I believe it's because we have an ancient, ancient relationship with these animals. It's not just because we evolved. It's in our DNA. It's also in our folklore and our stories. There is a tribe in Kenya and East Africa called the Maasai. They are pastoralists. And they have a story that goes that a bride was leaving home for the last time and walking off with her husband. Her father had told her, don't look back. Don't show regret. But she did. She disobeyed her father and she looked back. In that instant she was cursed and she began to turn into the first elephant. And this is why the Maasai believe that humans and elephants are related by blood. They believe that our spirits reside in elephants. And if the Maasai has come across a dead elephant on the savannah, they will stop and bless it by placing grass on the skull of that elephant. Grass is the most important part of their currency because they're pastoralists. And they believe that elephants would do the same for people. That if an elephant comes across a dead person, that elephant would stop and actually cover that person with bushes. And amazingly, scientists have actually seen this. Well, I'm a scientist. I studied elephants. I did my PhD at Princeton University. Like so many scientists, you know, spent decades pouring over data, analysing these incredible animals. What we're beginning to learn is that they are far more complex than anyone could have imagined. You probably all know that elephants have an incredible memory. They've got a brain many times larger than ours. They can hear from immense distances. If we were a herd of elephants in this room today, elephants in clusters would be able to hear us. They wouldn't just hear us. They would know what we're saying, and they would know who's speaking. They don't just listen with their ears. They listen with their feet and their trunk. They are incredible. And every year we discover more and more amazing facts about elephants, and every year we begin to lose them. One of the most amazing things to me is that elephants are beloved everywhere in the world.  They're on our pajamas. They're on our books. We show our children films about elephants. And children around the world grow up with these extraordinary animals in their lives. Well, in Africa, we see millions of visitors flooding into the continent to experience elephants and wild animals for themselves. How many of you have been to Africa and have gone on a safari? That's great. Amazing number of you. Those who haven't, I can assure you. Come to Africa and your lives will be changed forever. It's really extraordinary to meet animals like elephants. It's almost like they can read into your mind. Well, I love elephants because they're so much like human beings. They live in families. They have matriarchs. Here's a matriarch photograph I took of her standing beside a baby, shading it from the sun while she dusts herself. It's not even her baby. That was another female's baby. She is an aunt. And she's taken care of her sister's baby. And as soon as the sister rumbled and called for her baby, the matriarch nudged it with her foot gently, lifted it with her trunk, and the baby was taken to its mother. Elephants laugh. They make tools. They're incredibly intelligent. They understand the landscape. And they grieve. They cry and they return to the dead elephants of their relatives. They touch the bones. Every single year they return and they touch the bones, especially the tusks. They're extraordinary. There are many secrets about elephants that I think we'll never understand. But as Africa develops and it's developing at a massively rapid pace, people are moving to cities and we're disconnected from this great heritage of ours, these incredible animals. Very few Africans actually can go to our national parks anymore. And we're getting disconnected and we're losing touch with these extraordinary animals. This is Satau 2. He's one of Africa's largest tuskers. A few days after this photograph was taken, he was killed with a poison arrow. The poachers chose a poison arrow because it's silent. They couldn't be tracked. This poor elephant suffered for days before he fell. And as he was suffering, he was sending messages out to all the other elephants, telling them how much pain he was in and how much danger there was. Why? Because of the demand for ivory in countries half a world away. Primarily in China, Japan, the Philippines, Thailand and those countries. Because of the resurgence of the ivory trade as a result of a decision taken in 2000 and 2002, we have seen a catastrophic killing of elephants across the African continent. 30,000 elephants every year. That's one every 15 minutes. And as a result, we're seeing population declines in some of the most important areas. Tanzania has lost 76% of elephants in just five years. The Congo, 90%. Sudan, 95%. Some countries have lost their elephants altogether. Well, in Kenya, we saw this happening and we predicted exactly what would happen the moment the ivory trade was reopened. We saw poaching begin to increase. In 2012, it peaked and then it fell off. And it wasn't because our elephants had been decimated. It's because we took a national action. We launched a campaign which was driven by civil society in partnership with the government as well as companies like Amarula, which is their icon as an elephant. We also involved the media and we created a massive message that these are our elephants and we want to defend them. There are seven steps to what we did. First, we used evidence. We didn't just gather data on the dead elephants across the savannas. We also went into all the courtrooms. I have 10 lawyers who went into courtrooms and tracked every single wildlife trial and Kenya is the first country in the world to have a digital database of every single wildlife trial that gives us real-time data on what's happening in those trials. We mobilized thousands and thousands of people to join us on the streets to demand justice for wildlife. There is no leader in Kenya who can ignore the fact of thousands of people, especially youth and children, demanding justice for these extraordinary animals. Probably the most important thing that we did was we went to the media. We didn't just go to the newsrooms and tell these stories. We went to National Geographic, BBC, we went to Disney Nature and we asked them to give back those films that are made about Africa's wildlife and allow us to share them for free to air across Kenya. It might surprise you to hear that those amazing award-winning documentaries that you all grew up on, they've never been seen in Africa. They weren't made for Africa. So what we did was we asked for those films to be shown back in Kenya. And then we went another step. We said, well, we actually want to tell our own stories. We got a crew together. We got a Kenyan narrator. And we went into the field to record and document our own people, our own heroes, our rangers at the front lines, our scientists, our communities and tell their stories. And the amazing thing happened that we hadn't really anticipated. Five million Kenyans watch our TV show. We have twice as many viewers as National Geographic for our own local production. 40% of Kenyans say they want to volunteer for wildlife. And this has happened in just a space of two years. As a result of this, we've been able to expand our shows across the whole country, bring them into classrooms. I think it's just completely transformed the national consciousness about wildlife and the environment. We cultivated political will. We didn't just go to our president, who is a great champion for elephants. We went to his wife, Margaret Kenyatta. She's a great matriarch in her own right. The only first lady in the world who is taking on a wildlife initiative. She's the patron of the Hands of our Elephants campaign. And Judy Wahungu, the minister of environment, who pushed through necessary regulations and new legislation to support this initiative. We supported new boots on the ground. I'm not talking about ordinary rangers, just going into the national parks. I'm talking about elite forces from many different agencies working together at the borders, on the highways, in the parks, at the airports, and even at the seaports. And this has had a huge effect because it allows us to detect and arrest the criminals. We lobbied for the destruction of our ivory. I know for some people this looks a bit controversial. That's $15 million worth of ivory going up in flames. And it's not the only time Kenya has done it. Kenya has done this five times. But we believe that the only people who should wear ivory are elephants. And so there is really no need to store or keep this ivory. We've destroyed it so that it will never, ever appear in any international market.  Many other countries have now done it, including the United States, France, Gabon, even Hong Kong and China. And we pushed through new laws. Kenya today has the most severe penalties in the world. And the proof is in the pudding. This is one of Kenya's most notorious wildlife traffickers. His name is Faisal Muhammad Ali. It took us seven months to arrest this man. If I tell you about the trial, you won't believe it's far too fantastic. Evidence went missing. Witnesses disappeared. But thank God to this campaign. We created so much awareness in the country and just zero tolerance for criminals against wildlife that he was eventually convicted and jailed for 20 years. He's serving that sentence right now and he's been fined $200,000. He was the very first ever ivory trafficker to be arrested and tried in a Kenyan court. And since then we've had several more. As a result of that, poaching in Kenya has declined by nearly 90% in the last five years. And I'm so proud to have been part of the team that put together this strategy. So I would love to invite all of you to join me in Kenya, in Amberseli, where we have the world's most famous population of elephants. The Amberseli elephants are all named as individuals because they each have very special personalities. This is Tim. Tim is 49 years old.  He is so friendly, although he's massive, he's so friendly. When I go to Amberseli, if I see him and I call his name, he will turn his head to me and he'll even walk towards my car. Prince Albert, I would love to invite you to come and meet Tim. I'm sure that that introduction would lead to a lifelong friendship. But it's not enough to just save Kenya's elephants. As I told you, elephant poaching is happening across the entire continent. We're still losing elephants at a rate of 7% per year. And at that rate, elephants will be gone within our lifetimes from the wild. Within the next few decades, we could lose elephants from the wild. It's just a reality that's just impossible to accept and we will not accept it. And this is why I'm here in Davos. It's to really raise the level of urgency. We have to work together. And I believe that all the people in this room and the people here at Davos can work together to save this incredible iconic species. Imagine a day when the only elephants left on Earth are the ones in zoos, the ones on your mantelpiece. You know the little carvings you have on the mantelpiece or on your children's pajamas. It would just be unbelievably awful. So there are three things that I think we can do. First, don't buy ivory. It's kind of obvious. Don't buy ivory. But I think we should go a bit further than that. We need to make sure that our government's ban ivory trade into perpetuity. So I'd like a quick show of hands. People would be willing to write to their ministers or their presidents and say, let's ban ivory into perpetuity. That's fantastic. So please do it, because I'm pretty sure that if your presidents and ministers get such a letter from you as members and people who are some of the most powerful people in the world, it would make a massive difference. Secondly, support the grassroots efforts. We have people on the ground risking their lives every single day. They're parts of government. Some of them are in NGOs and other organizations. So please do support them. If anyone wants to know of organizations that they're out there, I'm very happy to share that information with you. And third, I'd like you to consider joining us in what we're doing in terms of creating global awareness. A Wildlife Direct we discovered that telling the stories about elephants and wildlife to Africans, not just the stories that are made by BBC like David Attenborough. He's fantastic. We love him. But when you tell a story by a local person to their own people, it has a magical effect. We have mobilized millions of conservation champions in Kenya. We call ourselves wildlife warriors. And I think we need to go way beyond Kenya. We need to take this initiative across the continent and to Asia where the demand is and probably across the whole world. We need everybody in the world to know about elephants, to understand them, to care about them enough that they will actually take actions. I'd love to work with all of you to make this possible. I'm working with National Geographic who have agreed to help me to scale this project. And I really wish that we could all work together to make this possible. Thank you.


Okay, so I would just like to start the session with Charles Druckmann who was at Janelia and has recently set up a lab at Stanford University and he's going to be talking about his work on cortical dynamics. Good afternoon everyone, can you hear me well enough? Okay, let me just start my timer so that I know that I'm on time. Okay, so I'm going to start a bit broadly given the fact that I assume not everyone here sort of is a full-time neuroscientist. This might seem so broad as to be a bit naive but I think it's actually worth thinking about. Right, so animals are pretty incredible and the things they do are amazing. Right, so you can go to the extreme example like a bee finding food and communicating it to its friends by a dance or you can go just to you know a mouse doing whatever it is when mice think they're not observed and you know just going about their daily business in their natural habitat and you know meeting predators like ducks and very important things like that and then yeah he makes it in the end. Right, so the the reason I bring it up is I find this pretty incredible. Right, there are these animals they have this little things inside their heads inside their bodies and somehow that is the thing that we're trying to study. Right, so sometimes when you get lost in the businesses of you know dendrites and neurons and this and that and you know this kind of neuromotorator that kind of neuromotor it helps to actually think how amazing is this phenomena and how challenging it might be to understand that. Right, so a slightly more adult version of that is that we're interested in understanding the neural computations that underlie behavior and what's been very interesting in the past few years is that our ability to record neural activity for animals that are actually behaving they're performing some kind of task is increasing remarkably right so Ken showed you a very nice example of this this morning I'll show you another one so this is the spatial temporal pattern of activity in a few hundred neurons recorded in cortex by g-camp imaging in Carl Suboda's lab and what and each one so we talked a bit about custom images this morning but just to make it clear each one is each dot here is a neuron their the level of their activity is color coded so that warmer or pinker colors indicate high activity and that's just a bunch of neurons that you can record and I'm just playing the activity over time while an animal is doing an extremely simple two alternative force choice task with a small delay and what I want you to take away from this is not the pretty colors but rather the fact that there are all kinds of weird spatial temporal patterns of activity here that don't have any obvious relationship to the task right so all the mouse needs to do is to figure out whether a pole was given up here or here and then report it a second and a half later and yet you find all this weird stuff popping up right so to me that immediately raises a question of what are we going to do with that right so there's one option which is to say we're going to do absolutely nothing with that right so technically in order for the brain to work and solve this task all it needs is that the spatial temporal pattern of activity visited by stimulus a whatever it is will be different than the spatial temporal pattern of activity listed by stimulus b I only show you the pattern for stimulus a but you can take the pattern of stimulus b duty conic analysis will take you five minutes they're very different this is all the brain needs and maybe we're done right so I think that's one attitude but I think there's a more interesting attitude that tries to take to take the details of this data seriously right and tries to be bothered by the fact that this is perhaps not how I would expect at cortex to look like right which brings you to the question the other immediate question then what exactly did I expect cortex to look at right can I actually do this can I look at the spatial temporal pattern of activity of a few hundred neurons and say with confidence this is totally consistent with the animal doing computation x but it's really not consistent with the animal doing computation y or conversely is this consistent with theory of cortical function a but not theory of cortical function b and I think what the ability to record this data highlights is that we're very far away from it and I think this is something that we need to deal with in my personal opinion the reason that we're very far away from being able to do that is that we just have terrible intuition for parallel distributed computation right so maybe should speak just for myself I have terrible intuition for parallel distributed computation right so if you take 500 thousand neurons I don't know take a billion as many as you want each one connects to a thousand others they're all constantly transmitting information between themselves computing something with your dynamics and then you want to ask yourself what should this thing look like while it's doing it and why does it look one particular way or another you quickly realize that you have no intuition for this mode of computation so it becomes very difficult to check it against data right so put so the central notion behind this kind of thinking is that the brain does this and the principles of how it does it are embedded in the details and we need to learn how to understand these details and how to deal with these details if we want to learn the unique computational style that the brain has you don't have to write you can just do machine learning in general and understand intelligence but if you are seriously interested in understanding how the brain does its own version this is something that we have to deal with every time that we see something surprising I think we should think about why that how that fits into our perceptual model and if it doesn't then you know try to rethink things a bit does that make sense so related to more to talk before then you know we're very used to structure to function questions in biology when it's physical structure right so synapses and wings and things like that and how they relate to actual function but now that we're able to record dynamics and detail we see if they have their own rich structure and that structure somehow related to the function to computation and we need to sort that out and then we can sort out the structure to dynamics part right so just like more it showed us all this beautiful structure it would be really weird to just ignore that and say oh what the hell you know there's a different way for the brain to work it doesn't have all that so we're not going to bother about these details I think it's just the same thing that we now have the ability to record these dynamics they're much more complex than we would have naively expected so I think it really makes sense to try and look at this okay that was perhaps an over long computation introduction because the course the question is how do you do this and this is the main thing that my lab is interested in so what when you talk about computation one can talk about it in general I don't even know exactly what that means so need to pick something specific the thing that I like is short-term memory which is the ability of animals to generate persistent and selective representations from transient stimuli okay so the classical example is a delayed match the sample task so the monkey sits on a chair he fixates at a screen you show it an image there it was and then there's going to be a delay period from a few seconds to a few tens of seconds and then you're going to show it the second image and the monkey's job is to say whether they're same or different so the first image was the general research campus building the second was the Eiffel Tower so if the monkey is smart he'll say they're different and if he's really smart he'll say that apparently the same length they've put end to end and equally popular as tourist destinations so this might seem like a pretty naive computation it might be very simple but there are some key properties of it that I really like right so the first is that every time that you introduce a stimulus to a brain then you can't control the computations that are happening right you give a sensory stimulus all kinds of things are happening pattern discrimination pattern completion bandwidth equalization many many things that have been described in the literature over the years and you just can't control those but having this delay period in the middle in which you're no longer presenting sensory stimuli in which hopefully the main computation that's going on is just this maintenance of this one particular thing that the mouse or monkey needs is a very unique opportunity and the other thing is that I think this is the simplest computation of building a model of the world which is I think what brains are for right so for instance if I turn my back to you which will be extremely rude but I'll do it anyhow to make the point right then you know the image of you is totally disappears from my retina but when I turn back I'm not surprised that you're here but but that's because I did that for a second if I would do that for 20 minutes then I'd be pretty surprised if you're here I mean I know this is Canada so people are extremely polite so it's possible that I would have to do it longer but but and you know but if I do it for two hours I won't be surprised that the table is still here right so you know it might be too fancy to call this building a model of the world and I want to say too much about it but just to say that this business of generating persistent representations of the world to me is I think extremely interesting and just indicative of a more general part of even more interesting things that animals need to do okay so what's the simplest way this could work so imagine you have a single neuron this is time on the x-axis and firing rate on the y-axis so I'm not supposed to use my the laser pointer but I'm not sure I'll deal with that so we'll see how that goes so there are two interesting times the times of the first image presentation the second image presentation and a very sensible way to do this would be to just you have a neuron that codes for this image since it codes for this image when you show the image the firing rate goes up and then the firing rate remains stable and elevated through the entire delay period right so this would be a very reasonable way to build this computation and when these experiments were done in the early 90s and when neurons like that were discovered it was extremely exciting because neurons have biophysical time constants of tens of milliseconds and it's not obvious how you generate something that suddenly has a time constant of many seconds right so this was extremely exciting extremely interesting from biophysical perspective but from the conceptual perspective it's not that exciting right you need to generate a persistent representations to just have a neuron that codes for it and you generate persistent activity in that you're on but what was then observed is that actually these neurons are extremely rare so they're about three to five percent of the population and if we look at most of them they just look different so ignore the different colors their different stimulus conditions that doesn't matter for now just look at the temporal profiles so the first one starts up and then goes down this one in the bottom left starts low and then goes up this one on the top right only remembers that needs to respond in the end and there are all kinds of weird things but whatever all this is it's not a stable and persistent representation and this immediately leads us to something weird so we all believe that neurons code for things with their firing rate we know all the monkey needs to do is to remember this one thing we know he's able to do that because he does the task and yet what we find is that the activity of almost every single neuron is constantly changing during this delay period so in cases like this when our intuition is tricky it's worth trying to think a bit more abstractly so bear with me while I do this so imagine you want to represent points on a plane not images that's a stimulus of the mentionality to one very reasonable way to do this is with a linear population code so each neuron has a preferred direction in space you're on one likes this top left direction you're on two likes this top right direction and imagine I want to encode this big point in space there's one very easy way to do that which imagine just the activity of your one is at half so I'm scaling down this vector by half and it's a linear population code so I'm going to sum across all the neurons you're on two had activity of one so I didn't scale it down I add vectors like you add vectors tip to toe and I find myself encoding the right point in space okay but now imagine that your own one is ramping up and you're on two is ramping down so now the vectors scale differently they scale differently they'll add up differently and I'll find myself encoding a different point in space which is again just our intuition that if neurons code for things with their firing rates the firing rate changes so should the representation however this is this intuition is extremely misleading every time that you have more neurons than dimensions of stimulus that you're trying to represent this can happen sort of by force when you have a divergent architecture like the classical phylamocortical projection where there are 50 times or 100 times many more neurons in cortex than in thalamus but often we're just interested in situations which we have you know a hundred million neurons in a cortical area and we're encoding basically one simple thing okay so how does that change the picture well it changes it dramatically you don't need to go to a hundred million dimensions it's enough to look at just three dimensions so now you have three vectors you need to arrange them in a plane if you choose this arrangement this is called the mercedes-benz arrangement in the applied math literature believe it or not this could be called the peace sign arrangement but I don't know if there was DARPA funding or something and then now the picture is totally different so if you want to encode this point in space you can do it just fine by having the activity of neuron one at one but you could also do it equally well with the activity of neuron one at half and you're on two at three at minus half they will add up to be exactly the same thing and you don't have to have negative activity that's only an artifact of using three neurons which is just done here for simplicity and of course it's not that you just have two patterns of activity we have an infinite number of patterns activity shown in the same way sort of level of activity in a bar plot below and how they above sorry and how they add up below and since it doesn't matter which of these patterns you choose to represent there's no change in the coding we and later others chose to call these non-coding or null patterns but of course it's not that the entire network could only represent one point in space that would be really weird there are different patterns of activity that do cause a change of representation right so here activity is changing and also the place that's being coded for the position in space that's been coding for is changing and since these patterns do cause a change of presentation we call these coding patterns does that make sense so an efficient way to think about it is again in this space of neural activity so you can present the activity of a network at a given point in time by a dot of a space equal to the number of neurons so here this dot is above on the one axis so neuron one has some activity and it's on zero in neurons two and three so there's no activity neurons two and three and the dynamics of the network are just how this dot evolves in time and the realization is that every time that you have more dimensions more neurons sorry than dimensions we're trying to represent there's going to be some degree of freedom or null space we call it null space just because of the relation to linear algebra or a non-coding space that if you change the activity along that direction then nothing will change representation and the animation in the bottom right was just me pushing the activity up and down this direction okay so of course when we observe a bunch of neurons we don't observe the full dimension we typically observe a very small fraction of them if we observe only one like in some of these classical experiments then I can make this neuron look like any arbitrary psth that exists just by the correct dynamics along this degree of freedom which won't cause a change of representation right so what this sort of somewhat abstract thinking has taught us that even a computation is simple as keeping track of a perfectly persistent stimulus can come with more complicated dynamics you might not easily expect okay but the real question is which someone if it were slightly less polite audience someone might ask as you know um is oh um yeah I'll come back to this you know is this just fun linear algebra of a board physicist or does it actually have anything to do with the brain uh so there's going to be one way to show that before I show you there are going to be several ways to show you before I get into that I want to make two comments about this the first is that this is fundamentally a theory of population coding right so if I observe as I mentioned before if I observe one neuron at a time then you have no way of knowing whether it's changing because it's taking apart in a meaningful pattern or a non meaningful pattern but there's something even more important that what this work tries to address is what is the unit that we choose to ascribe meaning to when we do a neuroscience experiment right so it used to be when you could only record one neuron at a time that everyone would try to interpret the brain through to the picture of single neuron activity that's kind of slightly in the past now and then people have switched to ensembles right but usually they mean kind of the same thing right so for instance you would have one selective ensemble which would have simple dynamics and another non selective ensemble which would have whatever dynamics you want but still it's you know these picture of one ensemble that's important another ensemble that's not important this is a very different picture so there are no two ensembles here there's just three neurons there's one ensemble and the unit that we're going to ascribe meaning to is a specific pattern across this population right so one pattern is going to be extremely meaningful and another pattern is going to be extremely non meaningful and if we choose to believe that that has fundamental consequences consequences for the kind of experiments we want to do the kind of perturbation techniques we feel like we should develop so it's an important thing and I hope to convince you that it's worth your time thinking about that okay so um what's the extreme version of how you could tell that this is not just linear algebra for board physicists so imagine I can go into a circuit I can map out what I record a bunch of neurons map out what I think is a non-coding space and what I think is the coding space and now I can do an experiment that specifically pushes activity only along the non-coding direction you would expect there be no behavioral effect and I can do a different set of trials in which I'll push the activity specifically along the coding direction and then you might expect that I would be able to generate a behavioral effect does that make sense that's not quite what we did so the technology to do that is very difficult we're pursuing it now but that data though very interesting is a bit too raw to talk about what we did is a zeroth order version of that experiment which we basically just bonked the network with a large perturbation then was up to us by analysis to figure out whether in certain cases we have perturbations that go along more in one direction along another direction sort of sort everything throughout by analysis instead of by doing the proper experiment but we're now also trying to do the proper experiments okay um oh I'm going the wrong way sorry okay uh so onto the data so okay so as I mentioned before this is a two alternative force choice task with a delay so a pole goes up in one of two positions and the mouse is trained so that if it goes up in one position he likes left and if it goes up in another position he likes right and this is what it looks like while he's doing it pole goes up there's a delay and then he likes pole goes up there's a delay and then he looks and they look awful cute while they're doing that the second thing that you want to do is you want to record dynamics from a relevant area and this is the point to say that my lab at at this time doesn't do experiments so all of this work was done in collaboration with the so both a lab uh and the first part was done by Newell Lee who's an extremely talented post who was an extremely talented postdoc in Carl's lab now he has his own group in Baylor and if you're looking for a postdoc and you're doing experiments I would definitely say go to Newell he's fantastic he's looking for postdocs um and then later Kevon who's a postdoc who's joined between Carl and myself joined to the project and he's also now doing the the more sophisticated perturbations that will hopefully soon will have enough data to talk about them in detail but none of this could have happened uh without this collaboration and without Carl's generosity to try to sort of ask some of the you know more out there theoretical ideas with his labs time and resources okay so previous work in the so both a lab has told us where one should record so the experiment that it was done I'll just go over it briefly is basically you take cortex you inhibit different parts of it you do this on a grid and then you're looking for an error that has a specific pattern of deficit which is when you shut it down during the sample period then there's no when you shut it down during the sample period there's no behavioral deficit since this is a memory area but when you shut it down during the memory area there's a very strong deficit in the animals ability to do the task you find that that's this area in the top left you call it something in this case ALM for anterior lateral motor cortex that's where it is and then Newell goes in with electrodes I know in the beginning I showed you some calcium data all the rest is going to be electrophysiology since it's much easier to talk about sort of fine-scale temporal dynamics with electrophysiology and then you look at the neurons in first of all you find neurons that have persistent activity during the delay period you find neurons that have selective activity during the delay period this neuron has this particular dynamic this neuron has weirder dynamics it ramps up from one trial condition and kind of ramps down for other trial condition this neuron has dynamics but no selectivity and you can find pretty much whatever you want in this zoo of neurons very much like a Ramos classical data okay and now the experiment in this case the perturbation is going to be photo inhibition which is you try to delete spikes by activating inhibitory neurons this was calibrated in separate experiments to destroy basically all of the activity in about a one millimeter area so again this is not sort of the precise activity space specific perturbation that we want but it has this very interesting flavor if we're going to observe the dynamics perturb them and then actually observe them as they recover and observe behavior as it recovers and try to then relate one to the other okay does that make sense good so let's just get into the data this is a sample neuron raster plot above shows spikes as a little tick and then psth below shows you the average across trial firing grade for these two different conditions so this neuron has very sort has activity during the delay period it ramps up very strongly there's much more activity for the right trials than the left trials and then when you do the perturbation on a subset of trials then add as advertised during the perturbation the firing rate goes down to essentially zero but then what was interesting is what happens after right so if you stop this perturbation early what you find is that the firing rates are recovering which is not obvious since basically all of the models of these networks work by having sort of these recurrent dynamics that are meant to supply memory but what was even more interesting is that they recover with the correct selectivity so blue was above red before and blue is still above red after right and given the fact that they recovered the correct selectivity it wasn't a surprise that the animals are actually totally fine with this perturbation even though you're taking the activity in the relevant brain area and wiping it out for a few hundred milliseconds but what was even more curious is that if you have a sharp eye you might notice that it looks like the activity is recovering awfully close to where it should have been had you not done this perturbation right so you can see that the the blue the the light blue trial is the non-perturb version of the dark blue trial and you see that they're coming up awfully close to each other it's not some artifact of just post-inhibitory rebound you can look at this for neurons that are ramping down and you see something very similar and you can do the statistics carefully to show that this is a real effect in many neurons but importantly it's in many neurons but not all of them right so there are neurons like this in which the original activity in light blue looks nothing like the recovery from perturb activity in dark blue and this is something that we'll return to in a few slides okay so why was this surprising to us so memory models circuit models of memory are very popular there are dozens of them and none of them have this prediction basically so the sort of the short version of the reason is that if you want to build a memory network then you're basically turning taking something that is a pulse of information turning into a step of representation right so what you're building is essentially an integrator and then an integrator has a very different prediction so in order to generate this wrap you actually have an integrator that's feeding into an integrator and then if you stop this integration during the delay period maybe even shut it down to zero you would expect to have this persistent gap it has to do with the amount of time that you did the perturbation which we just don't see in our data even though we tried in multiple ways okay so this is why we found it quite surprising but there's a very trivial explanation that could have been true right so imagine this brain error is just a readout of where the interesting stuff happens so then this is kind of what you'd expect to see right so when you shut it down you can't see anything but the since you haven't affected where the real stuff happens then you know you just see that it recovers when you stop perturbing it this is an argument that's very hard to absolutely shut down because you can never say that something didn't happen anywhere else in the brain let me give you a couple of lines of evidence why we don't think that's the case so the first has to do with playing with the amount of inhibition so imagine you what I told you up until now was for inhibiting an area that's roughly a fourth of this brain area you can then max out the inhibition and catch this entire brain area and there's almost no difference in the behavioral effect but as soon as you put some inhibition in the two hemispheres then you find that there are massive deficits in behavior and indeed if you totally shut down the two hemispheres the mice become at chance right so what this data strongly suggests is that what is preserving this memory is some kind of interaction between the two hemispheres as if the other hemisphere somehow has a copy of the relevant dynamics and is feeding it back into the hemisphere after you just perturbed it and the second line of evidence is you actually cut the corpus callosum which i thought there would be no way the mouse would still you know work after that and yet nuo did it and the prediction that comes out is the right prediction right that mouse is still able to do the test now it's extremely sensitive to unilateral perturbation so you know once we saw something like that it led to an immediate question of you know what do we exactly mean when we say that activity needs to recover right what do we mean when we say there's a copy of the dynamics on the other side i mean you know one wouldn't seriously expect every single idiosycratic twist in turn of every single neuron psdh to be stored in the other side right and then to somehow be magically restored by a bunch of axons going through so in order to look at that we turn to the same activity space analysis only now we're going to look at things along specific dimensions which are rotations of the space sometimes these are called modes the classical example is principal component analysis so imagine you have two neurons each dot is a trial these two neurons have some firing rate on each one of these trials and i'm just scattering all of the trials on one plot so what principal component analysis tries to do is to find a direction in the space which is just a particular weighted sum of neurons that maintains a lot of the variance right if i take a bunch of trials just to make visualization easier i project it down i project all of them down this direction activity space maintained some amount of variance right but if i switch to a different direction which again just a different weighted sum now 0.25 times neuron 1 not 0.75 this will retain a different amount of variance and eventually as i twist things around all the way i'll find this direction which is the first principal component which captures the maximum amount of variance when you project that data down does that make sense good that's not what we did this is the point where usually people fall asleep so i want to make sure that we did something similar conceptually but distinct which is linear discriminant analysis where now you have two trial types red and blue and what you want is not to maintain the maximum amount of variance you want to maintain decode ability you want to maintain your ability to predict what the animal is doing if i now take the choice that was just the first principal component this is not a very good choice because in this bottom left area things are heavily mixed but now as i twist the vector around again then my presentation gets stuck okay as i twist it around again i find this direction which once you project down then things are easily decodable just by throwing the threshold there in the middle between the two groups right so this is linear discriminant analysis it's of course a very old technique and i purposefully chose the data so that the first principal component and the first discriminant component are different but this doesn't have to be the case right so if the data were otherwise arranged they could be the same but there's no reason a priori to think they might be okay just to be over the deck take about okay so we find that direction and now of course what does it look like in the data so we take the neural activity we project it along this linear decoder and then we just look at a single trial over time that's one single blue trial this is one single red trial they're very different these are all of the trials from this session they are really different and it's very easy to decode what the animal is going to do okay and you can average across sessions it's an even stronger effect but what's interesting is not just these dynamics this is what we were kind of expecting from single neurons having decodability the question is what happens during the perturbation right so i'm keeping the the unperturbed data in these dashed lines in the background and now showing you in thick lines the perturbed data so when you do an ipsilateral perturbation of course during the perturbation the difference in between the two shut down because the firing rates are all going to zero but once you stop the perturbation things kind of spring back up and there's again they're again highly separable but what was more interesting is actually the bilateral data so now we shut down both hemispheres the animal is a chance most of the animals are a chance many of the animals are a chance and now the presentation indeed looks highly mixed right but we can use this mixing sort of as a feature not as a bug so if this really is the pattern of activity that's going to be relevant for what the mouse is going to do then we might be able to predict what the mouse is going to do even though the mouse himself apparently is unable to do it because he is still a chance and how would that work so this average is a very wide average in the sense that there are some trials that recover beautifully like this red trial and there are some trials that recovered so poorly that they went all the way into the area that's normally associated with the lick left decision and if this really is the important direction in activity space then we should be able to take these specific trials look at where they stand at the end of the delay period and then see how that relates to behavior so that's exactly what we did we took this direction we've been trials according to where they fell at the end of the delay period so if they fell into this area on the left which is the air on lick left trials if they fell into the area in the left which is the area that's normally associated with a lick left decision if they fell all the way here into the maximum case then the mouse will be right on a hundred percent of these trials if they fell into the area that's normally associated with the opposite decision then on these particular trials the mouse is going to be completely wrong right so we can basically predict what the mouse is going to do even though the mouse himself is unable to do it you can look at lick right trials and it's quite symmetric and this is the picture that we get right we have this one direction activity space that have dynamics that can easily tell apart what the mouse is going to do and these dynamics matter even when you do a perturbation then it causes the mouse himself to be confused but that's not the only direction activity space you can look for more directions you have more than one year on so generally you're going to be able to find one directions depending on how the variance is spread there's a limit to how many directions you can find but almost always you'll be able to find two so then we look for one on purpose that's very different so we saw that this direction recovered and now we want to look for one that's the opposite right we want to look for a direction activity space that actually doesn't recover at all you can do that and this is what the projection looks like so as you can see the light blue traces look very different from the from the blue and red traces which is where the dynamics normally are but it's not surprising that you can find that what's more interesting is to see what it looks like when you're not perturbing the system right so one hypothesis is that has nothing to do with the normal dynamics of the system so when you look at this particular direction in control trials you'll see that there's no variance in it right it captures nothing of the normal activity this is actually very different from what we find we find that this perturbed mode captures almost as much variance as the first mode which is the one that sort of oops uh yeah i have to speed up sorry i'm almost done uh which captures uh i i i did a really way too long introduction sorry uh which captures sort of so it has a similar amount of variance to the first direction which is the one that will determine the animal's behavior but when you try to decode things out of it it's much much much worse right you can barely decode things only in a subset of trials so what this might remind you of is that we have this type of uh we have this direction of activity space on the right which you can't decode anything out of and the brain chose to leave it to be not robust right it didn't bother to build robustness into it and you have this direction in the left we can use to predict behavior one to one and the brain chose uh to make it robust right so when we wrote the theory paper about sort of these different about sort of this interpretation of neurons through directions in activity space you weren't smart enough to make this prediction uh that happens often in theory but here you know no one thought there would be robustness so maybe that's an excuse but if you were to build the system this is what you would do if robustness was expensive to build you would put it into the components that are important and you probably might not put into the points that are not important so if you're building a space shuttle you probably should put it into the engines robustness but not maybe into the system that the astronauts use to send emails or something um and this is exactly what we found but the thing that's so interesting about it is that this is the correct explanation is not on the level of single neurons so this is a scatter of all the neurons their projection on the selective and non-selective modes so the neuron that I showed you in the beginning that looked like it recovered beautifully was the one that I purposefully chose uh that has a projection of one on the selective mode and zero on anything else and the neuron that I showed you that looked like a mess was again one that I purposefully chose that has a projection of zero on the selective mode and high projections on the other modes so its activity looks by eye like it's nothing similar but if you look at more stuff of all of the neurons there are mix right these were just two cherry picked examples so the level of single neuron is not the level where you find the correct explanation to this robustness and it's not in the level of selective ensembles versus non-selective examples so this is you know not real data that would show you how it would be if there was a selective ensemble but you would have a bunch of neurons that have a strong weight on the important modes and no weight on the non-important modes and you have a bunch of other neurons that have no weight on the discriminating modes and a lot of weight on the non-important mode this is about as different as can be from the picture that we find okay so there's a modeling section on how does one obtain such robustness which I'm going to totally skip over um okay I'll totally skip over it uh yeah just to say that we can do it right so in general modeling can tell you one of two things right and could give you a serious hypothesis for how things happen or you can just tell you that you don't need to invoke witches and dragons to assume that this thing is true let's just say that we don't need to invoke witches and dragons to assume this is true you can sort of build these very reasonable circuit models that have this property so I'm sorry I had to skip that section but let me just summarize our findings so we find the surprising robustness of the detailed trajectory and the important thing was is that the right way to explain this robustness is really by looking at these directions of activity space and then this goes along with our theoretical model that not all population activity modes not all directions and activities place are created equal of course the issue is that when you do a bunch of populations recordings what you're seeing is a sum across all of them superposition of all of them and then you need to somehow dissect that and in this case you actually really needed the perturbation experiments to be able to do that and that's not trivial in any case so I'll steal just one minute to say a philosophical epilogue here so I think this piece of work was very interesting not for the stuff I showed you but for the amount of questions that came out of it later which we're still trying to follow up well I think the stuff I showed you was interesting otherwise I wouldn't have bothered with it but but you know why were we able to why did this thing work it worked we had a very straightforward conceptual framework for how the theoretical question we're interested in should play out in data and I think that's why we were able to design the right experiments get them done and know what's the right analysis to do that ends up being very tricky because in order to do something like that you actually do need to know what exactly is a circuit trying to compute and you need to know what it would look like as it does that right so any of the two alone would be enough so if I knew what the circuit is trying to compute I can just look in the dynamics and say aha that's how it gets done and then write that down and we have an answer if I had a book that magically says what are all the computations and how they look like in population activity then I could just look and say oh mouse is doing computation 23 now the difficulty is that we know neither in fact I don't think we even have good conceptual modes of either so what we're trying to do is simultaneously solve both of them and that doesn't always work out right so I think a lot of the job of theory and computation is judiciously picking out cases in which this link is particularly strong to the experiment and then you can make a lot of progress and it's often difficult because you know if you don't have a theory of how the activity relates to something you're interested it's quite difficult to even know what's the right experiment what's something that you might even think it would be reasonable to try to look at okay I'll skip Warren McCulloch's famous words so okay so Warren McCulloch gave this talk in Harvard what is a number that a man may know it and a man that he may know a number which I think is very inspiring I think it has a lot to do with what we're doing neuroscience but this is a particularly human chauvinist point of view but when you know when things well down to the fact that he thought that we would understand what is the nature of being a person by understanding brains and neural computation and we would also understand you know what is the computation but seeing what humans do and I think what we're trying to do is kind of similar only in monkeys or mice right so this business of understanding information processing I think will tell us a lot about what it is to be an animal and vice versa sort of through trying to struggle with the things that the brain actually does we learn a lot about computation and I think that's something very exciting and I'll end there sorry probably still all my question time sorry for cutting you off but yeah we discussion time yeah start over there quickly use your microphones yeah so fantastic talk thank you very much what I'm curious about is whether you guys have examined this in naive animals so in particular you know you might expect that the robustness is only a product of the animal having learned that that is the null space for that problem and in a naive animal you wouldn't see that so that's a fantastic question okay so just to clarify the question is how come there are robust directions right evolution didn't encounter optogenetic perturbation of an entire hemisphere so first of all we we tried the usual substance we thought oh maybe this will just be you know robustness to noise and that will give it to you for free that's not true so then we have to track it through learning and in fact there are there are kind of two hypotheses that there's nothing stable in the beginning and learning stabilizes a direction but if you work with nonlinear networks it's not obvious that the hypothesis is it actually the opposite thing that there's a weird mix that everything is kind of robust and kind of non robust and learning is either about tuning the connections so that the most robust thing is used in dabbing down the rest or some weird combination for the boat we're trying to do that unfortunately you can't track the same neuron by ephys from day to day so you need to do it in imaging and unfortunately speaking about population dynamics through imaging is very tricky so on a neuroinformatics note that's something that we actually tried to do we looked at data collected from a few thousand neurons when that what from the animal doing the same task in imaging and in ephys and we systematically looked at the differences of how just sort of doing or sensibly the same analysis on these two data sets and there are a lot of complicated difficulties so we're still working through how much we think has changed you know how to do this properly given the fact that we have to do it through imaging but yeah we really want to do that and we really need to do I agree yeah yeah there's initially some robust dimension yeah yeah no I mean that exactly that that's the reason you would want to do these experiments but again this you know suddenly we're talking about directions right we because otherwise what we'd be comparing histograms of robust neurons that just feels you you see that sort of this kind of thinking actually opens up I think very interesting sets of experiments that I think the wise I actually wondered there's a paper that came out recently from Aaron Batista and Byron you where they sort of you know did a subsequent analysis on their perturbation of the BCI paradigm and they show that the sort of learning of new mappings happens with a fairly fixed in a fairly fixed part of the subspace and you sort of remap everything to a fairly constrained space which I think indicates that there are at least my interpretation you guys probably be able to give a much better one is that there is a fairly restricted subspace that learning maps things to maybe just you know because there's whatever biological constraints on that being the sort of optimal subspace but it's also wondering more generally so you know the task you're using here is fairly low-dimensional it's fairly simple and straightforward and you know I'm not super familiar with mouse anatomy so I wonder like what else does this brain area do and would you see you know this sort of stability and you know consistency of null space probability in you know different behaviors that the animal's engaging in or in sort of you know once you expand the dimensionality of what the animal's actually doing okay those are two long questions and I'll try to give short answers to both so I'll start with just explaining I think the first question to people who might not know the terminology so BCI is brain computer interface where you take a bunch of neurons you choose to relate that to somehow something that the monkey is interested in and then the monkey's job is to turn this brain computer control into something specific in order for it to get a reward so that's an extremely powerful scenario for relating neural activity to you know what a monkey is trying to do because you control every single neuron that's relevant for this thing which you can't do when you're just recording 50 random neurons from cortex and their finding is that basically there's a limit to how much in a single session importantly a monkey can remap what he's doing right so if you ask it to try to do something that's complicated and it's very different from the natural dynamics of the system that ends up being harder for the monkey to learn right so that's the other version of doing experiments sort of on this sort of faster time scale and these are beautiful experiments and really worth reading the the second question had to do with complexity of tasks so can I if I'm allowed to say something extreme I think that's the thing that's holding our field back so I think there's a lot that's interesting to be learned from two alternative force choice but ultimately if you need to do a binary decision you can do that with two neurons you might need 200 in order to I don't know fight signal to noise ratio but that doesn't explain why there are a hundred million neurons there right so I think the fact that that we're looking at these very low dimensional tasks is really impoverishing our view of dynamics in fact it's also impoverishing theorists because you know I know how to generate two fixed points really well people have known for 30 years right so if we want to look at more challenging things theoretically then it will be ideal if the experimental world will also switch to more difficult tasks and the training of animals to do complicated yet controlled things is one of the thing that the fields is not investing it enough crazily I'm thinking of starting to try to do this myself not because it's a good idea theorists you know it could go very wrong when theorists start doing experiments but but I think it's really to the point where this needs to be done and just the sooner the better because I mean I could make random predictions but but it just it so since it hasn't been done at all I think it just needs to be done and then you can look at it okay I think we should stop there so we have a little bit of time at the very end for people who want to come up with some further questions okay we'll go on to the next week great thank you very much


There's not many things that raise your IQ. One thing they know for sure is learning multiple languages makes you smarter. F*** the modern education system. Kids learn a hell of a lot more living in different cultures. I don't know how people are at home school. I can't imagine any other way.  Not what I went through. I wouldn't want that on my kids.  What was the main reason? It's because I'm different, I think.  Entrepreneurial. So you were always like that? Yeah, always trying to make money on my own.


For the module 10 summative assessment, we're going to be working with this very large and complicated model that consists of a number of different parts. You can see down in here, this is the global carbon cycle. And then that includes the percent or the concentration of CO2 in the atmosphere. That concentration of CO2 in the atmosphere then feeds into a climate model that tells us the temperature. And that temperature then goes into determining the climate damages caused by global warming. Those climate damage costs then affect the amount of money that we have left over to invest in the economy. So here's global capital. This is the whole heart of the economic model up in here. So climate damage is coming to play here. Another important cost that comes into play are the abatement costs. These are the costs related to reducing carbon emissions. And so there's something down here called the emissions control rate that we'll fiddle around with that represents essentially different choices we make about how much we're going to try to limit carbon emissions, that limits how much goes in the atmosphere, limits the temperature, and so on. But it costs money. So there's a certain abatement cost per gigaton of carbon that you are not emitting into the atmosphere.  The global population here is sort of fixed.  So this is the big model. That's behind the scenes. When you look at the actual model, you'll be seeing something like this, an interface where you're just going to change a few basic things. And for this summit of assessment, we're really just going to change the emissions control rate here, which is a graphical function of time. And in the next video, I'll explain how to make the.


Okay people, so I have a question for you. How long has Iambana been a thing? Also, how long has Iambana seemed like the worst of the worst PvP experience Spongey can conjure up? It's like they purposely leave Iambana to rot while sorting out issues elsewhere, in fact leave PvP to rot, it's just Iambana is the bottom of the list when it comes to PvP fixes to do soon it seems. Today I bring my first Iambana experience since this reset and wow, it's one crazy one. How's it going guys? My name's DP Jen, I'd like to thank you for stopping by and checking out my video. If you enjoy it leaving a like really helps out and if you are new around to and enjoy daily Destiny 2 videos be sure to subscribe. Okay so Iambana has been bad since its life began. They somehow take the sweats from Komp, pair them with the connections of a gypsy kid playing at his two bedroom caravan while using his father's 3G connection as a mobile hotspot and throw it into the game at a few cheeky rewards and expect everyone to play and be happy. Nope, not me. Iambana is so bad now it's unreal. The live gameplay you were about to witness, although I wasn't recording with my camera so you won't see my tears, I'm pretty sure you will hear the bitching I normally do when playing PvP.  Now have you guys ever experienced this? How's Iambana for you? Let me know down below. But on that note guys I'm out, enjoy the rest of the gameplay and hopefully I will see you on that next. One. Bitch! Sit down! I heard that from here. I heard it just brrrrr. Sounded crazy. I think they are, I think they said they are patching it too. Before this is the last Iambana where it would be good, I'm going to die yet. It's quite heavy. Did he? Where'd he go? Okay, Doug, you've just gone invisible. I don't even know what's going on now. Bodied. Bodied. I can't snipe anymore. What? Yeah. Last time we did the quest there were no three characters, there were no quests to do. That was clearly a headshot game.  I'm just filling the air around him. Oh my dear. How did I kill you? I don't even know what's going on. No! No! I'm dead. Like a line of thralls. That was all on traps. I don't know where that hunter figure is going. Okay, he's running away. Yeah, that's in him but it's too far away from the tip. Ah, Chris, that was an amazing blade. Garage. Almost used my sniping shot. No, no, no, no, no. Garage, I'll pop the map.  Ain't happening. I'd rather you'll pop the map. That's what you get. That is what you get. No shame here. No shame here. Okay, so that went through him. I'm telling you something not right here.  What are you gonna do? What are you gonna do? You're doing nothing. Oh, oh!  That was a save. Oh my Jesus. I just got bodied with a bit of love. He must have been low health.  Show them what that means. Five minutes remain. Wait, is that special? We must do enough to map. Oh, ouch. Don't give visible again. I don't know. This game's bust. This game's so bust. Okay, I've got an invisible. Okay. No, he's still invisible. He's still invisible. What is going on? This game is so low. How can I and Banna be in bad? What? This game is broke. I literally all I can see on the death cam is his gun on his back and his hand can and his hand wobbling along with it. What's there? This is bullshit, man. Look at this. Bitch, that's what you get. I don't know how I just killed him then, but look. Bitch, that's what you get. It's just invisible people everywhere. I'm invisible now. I'm invisible now. Wow. Wow. This is the brokeest thing I've ever experienced. Everybody's invisible. Apart from you, Chris, but your gun's invisible.  It's just, oh, I shot him back. I'm back. I just don't know who to shoot or what to shoot. And I just, I can just see a name shooting at me.    It's the worst. It's the worst. I don't even know what's going on. It's gonna be a quick night for me if it stays like this.  You just can't see nobody. You just aim it for the name. I don't know how I just did that. I don't know how he just did that.  I can't blame him though. Can't blame him. Can I blame him? I can only play him bunchy. Now my gun's invisible, but my hands ain't. This is just, I don't even know how it can be this bad. Did it happen, bitch? Did it happen with, what should be the whole net drop? I'm sure, yeah, I'm sure I stopped complaining about it. An invisible dude shooting rocket. It's just, it is.  There's loads of them. They're all invisible. I'm dead. I'm running. Get out of there. Even my pulse sounds weird when I shoot it. It's invisible. It just sounds like a this is the brookiest thing I've ever experienced. I don't know how we won that. Oh my God. I got a cold heart. Wow. That's so broke. That's so broke. Yeah, I might do. I might do. I might do. I might post it. It's just ridiculous, man. Let me play a couple more so it goes.