So I want to thank you for coming up and I realize it's tricky to get the elevator and get all the way up here to the 23rd floor, but I do appreciate the fact that you spent the time and effort to try and get here my name is Mike Anderson. I'm Chief Scientist for the PTR group and we are a company that has been in the of course we've been using Linux now for quite some time actually just after we were founded we started using Linux and we had traditionally been a BX works real-time operating system kind of shop so when the embedded when they first mentioned the concept of embedded Linux we all laughed. It's like, ah, there's no way you'll never be able to get it to fit because we were used to using boards that had 128k bytes worth of RAM I mean so when you're trying to deal with boards with very small memory footprints like that Linux just wasn't even in the cards at that point Fortunately a lot of that has changed over the years and of course with the introduction of the real-time patch that's changed quite a bit as well, but in our particular case we've been working a lot a lot of the business that we do is with NASA We have several projects right now that are all space-based robots Many cases we're actually using lidars for doing ranging on the robots There are some nasty little treaties about causing space debris So the fact that we're coming up underneath a low-earth orbit satellite and grappling it to try and refuel it on orbit If you make a mistake at those kinds of speeds, I mean orbital velocity is what six kilometers a second or something like that So if you make a mistake at those kinds of speeds, it just creates this huge debris cloud And it becomes that movie gravity that everybody saw right we know what happens with that so we absolutely want to worry about things like that and We've done a lot of embedded ports to Both our toss linux and bare metal pieces Of course tomorrow I have a presentation on iot messaging Well, we'll get into some of the messaging techniques that are used in the iot and how that's applied So lots of things that we're involved in we've got a little over 35 satellites on orbit right now That we're maintaining some of them actually date all the way back to the 68 302 So that chip hasn't even been in production for the past 20 years So and they're still flying so remarkably enough things can still last Yeah Yes, they've already been uploaded so you can download them off of the the site Myself, I've been in the industry now about 40 years And I got my first job as a programmer on an altair 8800 So with with floppy disks And it had front panel switches and we used to load a program in the front panel switches in order to get it to read the floppy disk so it was basically a jump instruction that would jump off to the rom and Of course in in those days Uh, we used to use a radio an am radio We would set it on top of the cpu Because it would make noises as the cpu would do things the cpu was so slow It was a 2 megahertz processor It was so slow that if you didn't have a radio to tell you that it was still working you would think it had died Fortunately with the front panel switch models you could actually watch the blinky lights But uh, then when they got a turnkey model there were no blinky lights. So it's like, how can you tell if it works? We had a floppy disk That was infinite storage because you could always pull a floppy disk out and stick another one in Infinite storage 253 k bytes at a time All right, enough enough down the history there So what we're here to talk about we're here to talk about light ours and their use in navigation In particular for robots That will be inside the house, although not necessarily limited to just that We'll talk a little bit about some of the drone mounted light ours and things of that sort as well So we'll talk about light ours. We'll talk about what options exist for light ours and then we'll Get into why would we put this thing on a robot? Why would we go to the trouble of doing this? And uh, we'll talk about how successful it was or not as the case may be and uh, we'll get into a summary So let's talk a little bit about light ours What exactly is a light are? well, it is uh, kind of a A munging of what we used to call a radar, of course radio detecting a detection and ranging Light are is light detection and ranging So, uh, unfortunately, there are lots of different ways that this is put out there. It's either light are lowercase i light are All uppercase sometimes they call them laid ours It just depends on who happens to be selling it as to exactly what it is. They call it For purposes of this presentation, we'll just call it light are with a all lowercase Uh, it's effectively nothing more than a light ranging mechanism of some sort now light ours themselves We'll use time of flight measurement. We'll see exactly what that looks like here in a moment We've got a picture of that coming up And we're trying to determine. Okay. I send the laser the laser pulse out. How long does it take to reflect back? Simple concept really nothing more than that But the fact that you can do that at extremely high frequencies that is very high data rates Means that you can get very fine granularity on the kinds of distance measurements that you're pulling back The big problem is that when we deal with light ours In general if we're dealing with a commercial light are that is for the Not for the scientific market. Let's put it that way They're usually in the 600 to 1000 nanometer range in terms of their light frequency Unfortunately, this puts it right in the eye absorption range So never look into a light are with your good eye No, actually in order for us to really be successful with light ours If they're going to be in this general range between 600 nanometer and 1000 nanometer We need to use class 1 lasers and class 1 lasers are eye safe. You can look into them. No problem with that. So They you do end up having to go with class 1 lasers, which unfortunately limits the power It also limits the distance that you can get out of them. So they do have some limitations associated with that When we start looking at some of the commercial light our units the commercial light our units typically run in the 1550 nanometer range And that is outside of the eye absorption range, which means they can run much higher power Which means they get greater distance and that's one of the advantages But the problem is as soon as you pop up to the 1550 nanometer range You now the cost kind of goes exponential on you and we'll see some examples of those costs. Yeah Yeah, uh, so we'll I'll show you some pictures that here in a moment So because the light are is basically a time of flight calculation Between when the laser pulse goes out and when it gets detected We're basically able to then measure time because we know the speed of light and all that sort of good stuff So we can actually determine how far away an object is But the the interesting thing about the way lidars typically work is there's almost always two Eyes on the lidar you have an emitter and you have a detector and they're offset slightly And the reason they're offset slightly is for triangulation purposes So we know the distance We know the angle between the two eyes and therefore we can calculate using trigonometric functions Exactly how far away the object is So that's the basic idea behind how a lidar works So it's not really all that complicated and it's very similar to some of the other modalities that we see for instance ultrasonic Infrared all basically work in the same principle So where do you actually find lidars being used commercially? Well, the first picture that you see there is what's called a flash lidar And flash lidars are basically cameras And what they do is they take a picture and each pixel is actually either 24 or 44 different laser pulses That then create the pixel and what they'll do is they'll do use faults coloring on there on the measurement for the actual distance To get the colors out there That is uh, basically that's kind of the the staple for the use of lidars 3d maps from airborne platforms. There are of course orbital satellites that use lidars for doing mapping purposes There are also of course now both aircraft and drones drones are becoming a big market now for the lidar space But the other place that we see lidars used quite a bit nowadays is in self-driving cars And if you've ever seen for instance, my daughter goes to Carnegie Mellon And uber has self-driving cars in Carnegie Mellon in the Pittsburgh area right now So you see them go by a lot and they have this little spinning dome thing that sits on top of them And that's the lidar that's trying to figure out exactly where the vehicle is in relationship to everything else So the lidars are actually becoming more and more popular certainly coming into much greater use Especially when we start dealing with actual distance and trying to figure out where we are in 3d space So there are some single point lidars that are out there Of course, if you've ever had one of these fat maxes or one of the other measuring tapes that are out there They have laser based measuring tapes Uh that basically do the exact same thing and again, you'll notice in this picture There are two eyes. There's an emitter on one side and a detector on the other And of course it doesn't project the marked laser beam out there into space That's thanks to walmart They they decided they were going to put that measured laser beam out there because people think that yeah It's going to show them exactly how far away it is whatever And those types of single point lidars Unfortunately, they're not really made to be interfaced to they're made as a sealed unit So you can buy them they they sell for less than a hundred dollars You can buy them and tear them apart and if you're good enough with your reverse engineering skills You can actually figure out a way of being able to hack into it and use it as a as a laser source Unfortunately, that is way too much trouble for what it's worth So fortunately, there are other alternatives that are available to us now You see the the one that is this guy right here This is from garmin There is actually a company called pulse light that had sold that particular type of lidar And they were selling it for an incredibly low price of only 150 a pop And it was so popular that garmin decided they were going to buy it They bought the company which unfortunately put them out of business for several months while they went to a new production facility And unfortunately that was right in the middle of the of the robot season So we couldn't buy the units really a frustrating and In this particular case this Unit can use either i2c it can use PWM is the other option that it has available for it But it runs off of 5 volt and so it's fairly easy to wire it up to an arduino or something similar like that and start using it to do measurements It's reasonably accurate. It has a range of about 30 meters So for small areas rooms and things of that sort. It's not so bad And at $150 it's fairly cheap all things considered Now when we want to go beyond that if we really want to start getting into All sorts of interesting applications. We start getting into scanning lidars Now the one that we had back up here This one you can actually mount that on a spinning turntable I actually tried that and of course wire management is an issue because now you've got a wire that is now spinning at some rate And balance is another issue because it was never made to be Never made to be spun like that and so after trying three or four different attempts and Using servos and stepper motors and a few other things. It's like uh, no, this is too much trouble Okay, that's an interesting concept, but let's move on So uh scanning lidars these typically move in a 360 degree range They will also like you see here. They have their two eyes. They're emitter and their detector And these particular lidars can be very expensive unfortunately This one actually here This one is not available yet. It's a model. It'll come become available sometime they say In late march early april time frame It was actually a Kickstarter campaign and if you take it apart it actually has one of the Garmin units inside of it Interestingly enough now this one is cheap for lidars. It's 350 dollars Um if you can actually get it so I'm waiting on mine to come from spark fund. We'll see how that works out This one up here. Uh, this one is $1,300 for that lidar And so lidars get very very expensive very quickly Especially as I said if you're in the 1550 nanometer range But all of these are in the smack in the middle of the infrared laser range So they're uh, unfortunately don't look into it It would be a bad thing They uh, most of these are controlled either through i-squared c pwm spy or in many cases serial So if you uh, oftentimes when we're trying to actually spend the data You know spend the lidar and get the data off of it. Uh, it requires quite a bit of transfer So it puts a fairly significant load on whatever thing you happen to have attached to the lidar In terms of being able just to absorb all the data that it produces So, uh, there are a couple of other things that we see of course commercial drone lidars There are a number of manufacturers of these that are out there these days It's uh, the the way the lidar is put together. These are of course the 1550 nanometer lidars typically and uh, they're Relatively heavy and relatively power hungry. So time aloft with these types of drones is very short Usually on the order of an hour or so before it runs out And that is if you can afford the hundred and twenty thousand dollar price tag for that little drone Um, it's not a big drone. This is a little drone 120 thousand dollars. So, uh, actually it's in euros. It's a hundred and 18,000 something or rather euros. So, uh, oh, but there's 10,000 euros in tax So it's not quite that expensive All right, uh, so that's kind of a cool object but again kind of out of the rage of the typical hobbyist There are uh, there is one source for a cheap scanning lidar And it is the nido robot, uh, cleaning robot So it's like the Roomba except in this case it happens to have a lidar mounted inside of it and uh, if you take the the robot apart Then you'll find the lidar sitting right in here and that's what it looks like when it's extracted You can actually get just the lidar unit refurbished lidar units off of ebay for about 120 dollars so and then of course you have to figure out how to Interface to the motor and how to interface to the lidar and how to get the data off of it Fortunately, there are some tear downs that are available on the internet Of course spark fun has one that goes into the details of it There are a couple of other sources out there that if you really wanted to try To scavenge one of these nido lidars The downside of that particular lidar is it has a range of only five meters So it's not very far. It doesn't have much of a way of distance. It's fine for a cleaning for a vacuum It's fine for a vacuum But for some of the applications that I had for it not quite so good So uh, and I again, I just kind of looked at it and go I'm going to have to figure out how to how to get into those jst connectors and all the rest of the stuff that goes along with it Yeah, let's see if we can move on and get something else So why are we really interested in a lidar? Well, we want to be able to do very fine ranging inside of a home for instance or inside of a building And in order for us to be able to do that ranging We actually need to combine it with another piece of information And that is we will combine it with an inertial measurement unit an imu And the imu has got like three degrees of freedom for the gyro the compass and you know for a magnetometer and for an accelerometer So using those nine degrees of freedom nine dock board We can then combine that information using sensor fusion With the actual distances that are being measured by the lidar and construct a map of what the room looks like So that's basically what the needle robot does It runs around and uses the lidar to figure out what the room looks like and then it figures out how to sweep the room Uh, if you happen to have a rumba, especially one of the earlier rumbas You notice that it just moves at random And it gets it basically it moves in such a way that it hopes that eventually if it runs If it doesn't run out of battery, it will have swept the entire floor There's no particular rhyme or reason for the way that it actually operates, which is infuriating But it's interesting to watch the cats chase it So it does have some interesting advantages there The actual measurement and the implementation of what they call slam simultaneous localization and mapping You will then describe it in this wonderful little short math function here But basically all it's doing is it's taking the summary of all these distances that it's measuring And then it factors in the navigation from the inertial management an inertial measurement unit To figure out exactly what sorts of things it's encountered And eventually build a map of the facility that it happens to be located in And that's one of the things that uh, where I was trying to get to for this particular application Uh, for what it's worth google has just released the slam algorithms that they use in their self-driving cars I've taken a look at it and definitely it's not optimized for hobby type lasers, of course And uh, it's Unfortunately, uh quite a bit computationally expensive because of course they're trying to solve a slightly different problem Keep cars from running into each other At very high speeds So most of the robots that I work with don't move any more than about 35 mile an hour And um, you know as a consequence, uh, that's a little bit overkill for my particular application But once we tie it in with an inertial management a measurement system an inertial measurement unit Then we can do the data fusion and be uh use that to create an internal map Of everything and because we have the inertial measurement unit We know exactly where the robot is at any one point in time The robot doesn't know where it is when it first starts It just simply starts scanning and then as it moves we have the uh accelerometers We have the gyroscope so we can then figure out where the robot is pointing at any one point in time And then use that to help construct this map that we're trying to build So eventually what happens if we happen to get enough laser points We can do one of these things they call a cloud a point cloud And what point clouds allow us to do is to have a 3d representation of whatever it is that we've just scanned Now in order to do a point cloud it really isn't that complicated If you happen to have a pan tilt mechanism say with servos or with stepper motors then you put that Garmin LiDAR on there and you just simply start scanning And you go scan step scan step scan step can't scan step and then after about five minutes or so You have enough data to then be able to draw one of these point clouds And that's kind of cool Unfortunately, you can't really do that in real time So it is a bit of a nuisance from that perspective But it is still kind of cool when you go to your your students and you go, okay Well, let's build this thing And describe what it's supposed to look like and then they graph it and they actually can see somebody's face In the point cloud, which is kind of definitely very cool These can typically in this particular case it was imported into blender And they use blender to actually do the graphing, but you can use processing You can use python matlab so many different solutions for this Excuse me Yeah, yes, yep So lots of different options that are available here Now that's kind of the background theory on LiDARs now. Let's talk a little bit about What we do in order to create a robot to mount the LiDAR on now Creating a robot is not something that you'll just wake up one morning and say I'm going to build a robot today unless you've done it a lot And fortunately over the last eight years. I've had a great opportunity of working with the first robotics competition And that in the first tech challenge and so we've had a lot of opportunity to build robots Now most of the robots that I build are 120 pound 35 mile an hour kind of robots So they're much bigger than what I really wanted to build here I wanted to build something that would be able to navigate around in a home Without running into things and without crushing the cat If they happen to get too close So when we're creating a robot, we really have to understand several different aspects. First of all, why Are you creating the robot? What is it supposed to do? Um, you know in my particular case, I have a very specific sort of mission for it So I understood exactly what I wanted it to do in some cases You just build one because it's cool and it's fun to do And it's very educational if you happen to have kids who are also willing to sit down with you and And actually go through the process, but we need to worry about things like how you're going to move it What's its mobility? Is it wheels? Is it tracks? Is it a hovercraft? Is it a drone that flies? How do you move this thing? We need to understand the motor controllers So if our motors are let's say, uh, you know, five amp motors Then I have to have a motor controller that can handle that kind of sustained current I need to worry about not only the power draw is it a nine volt a 20 volt a 24 volt whatever kind of motor it may be But I also have to worry about the steady state current If I'm running at full speed, what does it pull? But then I will also have cases where I'm going full speed forward and I suddenly want to go full speed backwards So now the amount of current draw at that is extremely large And so it's not at all uncommon in the case of the first robotics Type of robots to see 80 amps being pulled off of the battery at any one point in time So and it's really interesting for students when you when you explain to them That that car battery that you're using on the robot will light them up If they manage to get across both the power and the ground side of that battery at any one point in time so We have to think about What kind of motors we're going to be using are they brushed dc motors? Those are the simplest motors But they also don't have a lot of precision associated with them unless we're going to then put an encoder on them of some sort Brushless motors are usually three phase motors. They're easier to control But you then have to have an electronic speed control in order to manage them So there's a completely different sort of electrical harness that you have to have for brushless motors How are you going to power them? I mean the battery chemistry becomes a factor We can certainly use battery chemistry is like nickel metal hydride. We can use lipos lifey batteries Now lipo batteries have a great characteristic They're basically flat until you get to the end of life of the battery and then the battery drops like a rock Unfortunately on big robots if you pull too much current out of a lipo battery too fast it catches fire Uh, one of the reasons why the tsa is not terribly fond of lipo batteries as they get transferred around um Because that fire problem is kind of an issue, especially if you accidentally short the battery Which I have seen students do on occasion. They uh, they they didn't realize that the power and ground was actually connected to each other When they plug the battery in and it does a halt and catch fire. It's amazing um The of course which controller you're going to use now when we're using these motor controllers the motor controller has to be controlled by something So what is the something we're going to use? It has to be very reliable in the sense that the duration of the pulse Most of these are going to use pulse width modulation The duration of the pulse we can't see much jitter out of it if we do we've got a problem Now, uh, we'll see examples here. I've got an example of how linux sees these things Of course, we also have other issues associated with the connection. How are we going to control it? Are we going to use wireless bluetooth, wi-fi, zig b? How far does it have to go? Does it have to go through walls? All those sorts of things have to be taken into account So as you're going through designing a robot you have a lot of Pre-work that you have to think through in order to be able to make sure that you're going to have all the answers when you get there Now linux and motor controllers A lot of the inexpensive motor controllers are going to be using rc hobby servo type signals. These are typically 50 hertz Pulse width. I mean 50 hertz refresh rates and pulse widths in the anywhere from 0.5 to 2.5 millisecond kind of pulses and these pulse widths Uh, if you're dealing with just a standard hobby servo, you know so a servo is typically only going to have 180 degrees worth of motion out of it And servos are basically encoder based motors That will be able to repeat their location. If I say go to 170 degrees It'll always be able to go to 170 degrees assuming that nothing is blocking it for some reason So the servos themselves are pretty tolerant of pulse width variations So if you're using linux with the real-time patch Then you can probably use linux with a real-time patch on a hobby servo It'll probably be okay But As soon as you start getting to some of these bigger motors and bigger motor controllers Then any jitter in the motor controller in the pwm signal that's going to the motor controller I've actually seen motors destruct themselves because the jitter was so bad So in that case If you want to try and avoid getting yourself into any significant trouble Then you need to think about how am I going to generate a reliable pulse width modulated signal to keep this thing fed And keep the motor controller from winging out and then causing the motor to blow up on you So, uh, this typically means we're going to either use something like the beagle bone pru Or we're going to use an arduino or some other microcontroller Well, we can dedicate the controller to the actual generation of the signal so that we don't lose sync on the pwm So now, uh, with all of that, uh, what exactly did I want this thing to do? Well, in my particular case, there are so many new lidar models that are coming out I wanted to have a test bed that we could then set there and plug it in and Determine whether or not this particular lidar is going to be easy to work with hard to work with I needed to have something that my students and my robotic students would understand Just be able to look at not be intimidated by the robot and be able to then focus themselves on the actual lidar unit itself and not worry about how everything else somehow worked I also wanted to start playing around with some slam approaches. So the slam models Are really important as we start seeing for instance in the first robotics competition They keep introducing new obstacles on the field every year the game is different So you'd like to be able to create a map of what the field was so that you could then find out where your robot was At any one point in time in this particular year's game We actually have several blind spots on the field Because there's an airship in the middle of the field and we can't see past it So having the robot being able to determine exactly where it is at any one point in time either with the lidar and the and the slam algorithms You know really helps the students to be able to drive also With the the amount of data that's coming off of these we're also starting to see kind of Problems with the students being completely overwhelmed. There's too much data coming off the robot And it's that classic problem of I can't be the pilot and the weapons officer at the same time I have to have somebody else that helps me just to keep track of all the stuff that's out there We already have that in first. We have driver one and driver two But in this particular case Even with that they're starting to get overwhelmed with the amount of data that's coming off the robots So one of the other things Yesterday I had an opportunity to talk about using Alexa and creating an echo based device That would allow you to interact with it And I have several friends of mine whose parents are getting into the geriatric time And they don't want to be moved into a home. They want to be able to continue to live in their own home But if they are living alone You need somebody to be able to say hey, have you remembered to take your medicine today?  Because unfortunately as they start getting a little bit older and aged they start forgetting to do things like that And then their health starts to deteriorate quite significantly So I was thinking all right, let's see if we can come up with a way of being able to navigate around through the house And have something that somebody could talk to And just having something that you talk to even if it's not all that complicated Like where is Uganda, you know, but have an answer and it turns out that people really feel good about that It makes them feel good that there's something there that's going to interact with them And do more than just simply bark at them or require to be taken out for a walk so It really was one of those things of hey This is a great opportunity to do a service bot That then also allows me to experiment with all these other slam algorithms and lidars and kind of bring it all together in one piece So Choosing the platform was an interesting problem. Uh, again, I remember I uh, I'm typically used to dealing with big robots 100 pound plus kind of robots powered with car batteries That's not going to work in my kitchen. So at least my wife won't let me use it in the kitchen. It does sort of fit, but She doesn't gets a little upset about it And so I uh looked at it and I go well, all right, and if I wanted to bring it to a conference Having a 120 pound robot and a sealed lead acid battery Trying to get that on the plane a little tough. It doesn't fit in the overhead TSA doesn't like it And then when they x-ray your luggage they end up tearing your luggage apart because they can't quite figure out what that thing is in luggage So I said, well, I wanted something small And I wanted something different. I built a lot of wheeled based robots So I thought now let's do something that's track-based. That sounds fun So, uh, I found this particular model here This is from a company called A-O-O-K-Y-E kooky That should have set off an alarm bell in my head When you buy a robot from a kooky company We'll find out later exactly how that worked out But the thing is that these particular motors and the the motors are located back here So that's the actual drive wheel and all the rest of these are passive And the motor itself is a nine-volt motor, but it has a four and a half amp stall current And at four and a half amp stall current It means that you're you've immediately ruled out the l298 h-bridge type motor controllers Because those things, uh, I mean they're great motor controllers Six-volt motors. Yeah, you can do that typically two amp max draws But in this particular case these guys draw way too much current and I was concerned that it would toast an h-bridge So I said well, all right. We got to do something else Now, uh The plans for the controls I've used both raspberry pi and beagle bone black in the past and The last robot that I built was a six-wheel drive all-terrain robot Using red using beagle bone black and it was very successful But it didn't have a scanning lidar on it and it didn't have all these other sensors that I was going to have to put into it So I was concerned I just I'm guessing you know here. I'm in the design phase I don't really know how much data is going to be produced by this lidar because I don't have it yet And I uh, it was on order, but it hadn't arrived yet And as a consequence, I'm like, you know Beagle bone black would be great or beagle bone black wireless would be great because now I've got the wi-fi interface on it But it's like I really don't know how much horsepower this is going to take So I happened to have some raspberry pi 3s sitting around the house and I decided well, okay We'll build it on raspberry pi 3 In order for me to avoid the pwm jitter issue Even though you can run the the the real-time patch on it I wanted to make sure that I didn't get any problems with the motor controller. So I Decided that what we would do is we would use a high power motor controller This one happens to be from sparkfun It is built as an arduino shield Now when you buy it, it's unsoldered So some assembly required and if you don't feel comfortable with through hole soldering This is probably not the solution for you But the advantage of this particular motor controller is that it's a dual channel motor controller max current 30 amps sustained six excuse me sustained 14 amps And it can have a max voltage of up to 16 volts So the way brushed dc motors work if I have something that can handle up to 16 volts But I only put nine volts through it or less than everything's good. I don't have to worry about burning anything up so That particular motor controller. I think was like 60 dollars Which is relatively cheap for a motor controller in that general range The saber tooth motor controllers that can handle 80 80 amps Are like 125 a pop per channel So that's a little bit on the expensive side first robotics, of course has some really cheap ones at 45 dollars That'll handle up to 80 amps, but they're big And I didn't want to have to figure out how i'm going to get that on that little bitty robot Additionally it turns out there is an arduino Hat that's sold for this particular for the raspberry pi And it implements an arduino leonardo. So it's a 32 u4 at male part And it has the arduino pentouts. That's the thing that you see the kind of the blue and red things Those are arduino pentouts So if you put the arduino connector on this motor controller It should then just plug directly into the hat that then plugs into the raspberry pi That seemed like a really reasonable solution Considering the amount of space that I had on the robot and I said well, okay I don't want to have have multiple boards sitting out there The weird thing about this hat unfortunately Is that you would think that since it's plugged into the raspberry pi The raspberry pi would be able to talk directly to the arduino through the raspberry pi pens But you would be wrong You have to run a usb cable From the raspberry pi to the arduino shield And that's not intuitively obvious in any of the description that they have for the shield Of course, it's coming out of uh, I think it's uh df robot out of china and um Yeah, they they didn't say that you had to hook up the leonardo via a usb interface And You just think of course it's going to be able to talk directly to the uart coming off of the pens on the raspberry pi Nope, it doesn't go figure. All right So how am I going to power this thing? Because I wanted to be able to transport it without running into any Problems with our good friends at the tsa. I said, well, you know, let's go with uh, nickel metal hydride Nickel metal hydride doesn't have doesn't have any requirements for special charging circuits Unfortunately, they tend to be relatively large But uh, in this particular case, it's uh, about the three quarters of a stick of dynamite. Oh, no, you probably shouldn't say that with the tsi But no, it's it's it's a relatively large package, but it's a five amp battery So it can definitely power the unit for quite some time Now also, I found this really nifty breakout at spark fund, which is a usb micro connector And you just simply hook power and ground to it And now you've got a connection that just plugs into anything that uses a usb micro and now you can power it externally But of course, uh, the thing that you have to remember is that uh, beagle bones and raspberry pies and things of that sort require five volt And my battery was a nine point six volt battery and you can't just simply plug nine point six volts into a Uh, a development board like that. It'll toast it. So we had to go through a power transformer Well in my particular case, it turns out there's a really cheap way to do this And it's called a universal battery eliminator circuit and these things sell for about 10 bucks And they're made for rc aircraft They're made to actually power the receiver on rc aircraft. They're a buck transformer And uh, the particular one that i'm using here can handle up to 10 amps So it'll definitely handle whatever it is the raspberry pie drains, which is about two amps And then plus the lidar which is going to pull quite a bit of more power off of it as well So, uh, that was the basic model that we were using as far as the controls are concerned Choosing the lidar I have Four or five of the garmin lidar units that I've used in in past games and things of that sort And they're great, but again that whole I wanted to have a scanning lidar So just to make things more difficult for myself. I go, well, okay, I'm not I'm not going to use that I'm going to use a scanning lidar of some sort It turns out that the cheapest Scanning lidar that I could find was about 400 dollars and that's this guy here the rpi lidar a two unit This is from a company called slam tech It's 4 000 samples a second At 10 hertz and then 600 rpm is how fast the little bulb spins around It's also relatively low profile. So it's only about Say two inches high. So it's not very high That of course will come back to haunt me a little bit later when I start mounting everything Wiring up it took about, uh, I guess two days two and a half days to wire it and to actually construct the robot was Building the robot took me a better part of a day. Actually, I use slave. I mean student labor To build a robot. It's like, hey, would you like to build a robot? And they go, oh, we got nothing else to do right now because we're getting ready to put the robot in a bag So sure So, um, I allowed the students to build the actual robot Which was a mistake as it turned out, but uh, nonetheless, it was it was a good exercise for them And then I only had to redo part of it I then of course Can talk to the arduino using rs-232. It turns out the rpi lidar also uses rs-232 So I just needed to make sure that I had enough high speed serial ports of which the Raspberry Pi has only one To be able to communicate with everything. So it was usb to serial port adapter time And that seems to work. Okay. Uh, also I used anderson power poles no relation. Unfortunately. I really wish it was I wouldn't be here. I'd be retired someplace But anderson power poles these things are used in the ham radio community quite a bit They're just a quick connect disconnect kind of power And so I could easily what I did was I cut the tamia connector off of the Battery and then put anderson power poles on it and then it just becomes a real quick connect disconnect So I don't have to worry about putting a switch in the circuit to shut the battery off Then so this is the finished robot I'm not sure if you can actually see all the the elements inside of this, of course We have the tracks We have the raspberry pi here We have the shield mounted on top. There's the You see the blue and red elements of the arduino Shield or the arduino hat and then the motor controller shield sits on top of that It turns out that this whole scaffolding Environment here did not exist. I had to fabricate that Yeah, it was a nuisance, but nonetheless I needed to get the lidar up above everything So I didn't have anything that was interfering with the scan And I also added one more thing to it on the front here. I added a usb camera Just so that I could see where the robot was in case I couldn't actually Physically see it. I could at least see where the robot thinks it is And then could take control of it control of course all being done using wi-fi Of course in this particular case I mounted a tux on the top of it, but he spends too fast and gets out of balance So I had to take it off Too bad Now software So it's taken me several days To build the robot and in terms of actual invested money here We're talking about $600 or so 700 actually probably cost us $700 And I don't it can't move yet I got to have some software on it So fortunately sparkfun had an example for the arduino Where it would at least be able to control the motor be able to turn the motor controllers on and off Of course, it wasn't set up to be controlled with rs 232. So I had to modify it in order to be able to do that And it also turned out that for the lidar slam tech the company that sells the that sells the lidar Actually has an sdk available for it that works for c and c plus plus And of course if you want to use uh java you can use j and i or if you want to use python You can use the python c call. So um that all works there But um even though they have an sdk It was going to require quite a bit of development time in order to be able to spend the lidar up and then start sampling it Figure out what the measurements were. Yes, there are some documents that describe all of this But it's fairly complicated to interface to one of these things So as I was looking around I actually found an existence of the uh of some slam code for a robot operating system for ross and it turns out that the The slam tech rpi lidar happens to have a package that works in ross So it's like oh cool. I'll use ross as the way of being able to at least interface to the lidar and uh handle it that way unfortunately, uh ross is a fairly complicated package So uh ultimately I want to use ross for the entire robot But to get started. I'm just going to use ross to do the measurements on the lidar Uh ross is available for raspbian. It's also available for ubuntu So they have pre-built packages for both of those versions of the operating system That's a hell of a lot easier to work with that than it is to try and build ross from source That I didn't want to start messing around with uh, you know cross compiling and everything So it was uh, just again the path of least resistance here Now there were some references to the rpi lidar an earlier version of it on hackaday.io not hackaday.com Unfortunately, it was for an older version of ross that was not ford portable So I needed to go poke around a little bit further to try and find A version that would install correctly and work with the lidar Ross does support a published subscribe model So if you're familiar with how published subscribe models work, basically you have a data source You publish the data somebody else subscribes to it and they get the data on a refresh rate Fortunately, there's a wiki for the rpi lidar in ross. So there were pieces Took a little bit of research in order to figure out where all the pieces were and then to try and put them all together into one place Uh, so I used it turned out that most of the documentation for using ross Uh tends to be very ubuntu centric So, uh, the path of least resistance on the raspberry pi was to use ubuntu mate 1604 My preference would have been raspbian, but uh, it was just like, okay, how am I going to get this thing working before the show? So I wanted to bring it here Ross of course is uh, again only being used for the lidar at this point But uh, once I finish with all of this, there'll be a github repository And if you send me an email, I'll be happy to send you the link on it when everything's said and done So that you can take a look at how it was put together So I got it up and running and I got ross scanning and I have a it's uh, you can see it here These are basically uh desks and tables and things of that sort inside of the room So every one of these red lines represents some obstacle that's in the field of view Uh, it turns out that's a person and that's a person too And it's kind of cool as they walk through because there's this line of pixels that move through the screen as the thing updates Uh, so it's kind of a cool solution. But of course, this is just the first step This is just the distance measurement part This hasn't integrated it into the actual slam algorithm yet And I don't have the imu up and running for it yet So I can't start doing mapping inside of the house at this point. But again, a lot of the pieces are there So current status wireless communications works actually I did this Uh, a couple of years ago and had a robot here at ELC That was being driven with the xbox controllers through linux. So it was uh, I basically was running the raspberry Excuse me the bigel bone black on the robot It was the server my Controller with the xbox controller was the client I would connect over using standard sockets And then I would start sending joystick commands across the link and I could drive the robot around which is kind of cool But it means that I could then do some code reuse. Of course, that's always important, right? And pull all that information about using the controls getting the joystick working I mean, these are all little things, but they're nagging things when you say, okay Well, how does a how does an xbox controller work? You know, what events does it generate in linux when you push the button or when you move the joystick because there's analog portions There's digital portions. How does all that work? So that is already out in my github repository So if you're interested in that you can go out there and check that But right now I don't have the imu integrated yet because I'm thinking that I'm going to convert this all over to ross I need to be able to get an imu that ross understands I don't want to have to write a new set of drivers just to be able to use ross Uh, I uh as I had all this put together. I'm ready to start testing and I find out that one of the motors is borked Uh dead locked up. It will not move No matter how much force you put into it. It is dead in the water Um power doesn't do it, you know, you can't even turn it with a pair of pliers I disassembled the gearbox and Thought I had it all put back together again and when I put it back together and it wasn't any better so I've contacted the kooky people and uh, the kooky people are now arguing with me about, uh, this motor control about the motor And I go dude, it didn't work. I've only had it a week. It didn't work. Well, you know, people have problems with power And this is like, okay, dude, I've been building robots for eight years I understand which is the red and which is the black and where you put the positive and where you put the negative Uh, okay, but so that's the argument that I'm going through right now In the meantime, I'm I'm fed up enough with them that I've said, you know what, okay, I'm going to get a different robot So that's a new one that's on order That one's I think a hundred and thirty dollars or something like that But that one has four Sets of motors on it. So I'm going to have to use a different motor controller and do things a little bit differently on it, but nonetheless I I will hope that I'll be able to get a replacement motor from the kooky people But if not, then I'm going to have to punt that particular effort Take everything transplanted over to the other robot and try it again Which is not my desired situation, but that's the way it goes in the robot business So our summary here The first attempt at using a lidar, uh, this has been a really interesting exercise It required me to learn a lot about what lidars are and how they work and what they do It, uh, I found, you know a reasonably inexpensive one, uh at four hundred dollars That's definitely not something that most people would think is inexpensive, but for lidars it is It turns out that linux can handle most of the services with no problem The only issue that we have with linux and this type of robotic application Is the issue associated with the motor controllers themselves and making sure that we can guarantee that pwm signal coming into the motor controllers So in that case, uh, you know, if I'd had the beagle bone blue at the time Well, it still doesn't handle the power. So I would have to still do something else differently with that with that but, um That definitely is, uh, kind of a work in progress and I hope to have everything operational again as soon as I can get the replacement motor Uh, within the next couple of weeks or so and then we'll transfer translate it over to ross It turns out, uh, arduino. I mean the the linux drone project has a lot of cool stuff Ross has a lot of cool stuff. So there's several different frameworks that you can use to actually control this The real driving factor for me in this case with ross was the fact that the rpi layer already had an interface a package for ross That I could then just instantiate and start pulling data off of it So software development Probably at least in other two three weeks where the software development and I should have it all working At least mechanically if I can get the replacement motor out of them or end up buying new motors I don't know what I'm going to have to do on that But uh, you know the whole interface to the tracks and everything is going to be Totally screwed up if I have to get different motors. So I don't know. I'm going to have to machine something I'm not looking forward to that a 3d print. I could always 3d print it. I guess right So looking forward to getting the unit back online and uh, it's been an interesting exercise But I hope you got something out of this. We got just a couple minutes for questions and then we have to move on anybody You're all still asleep. I understand. Yes Yes, I have Now those are industrial lidars and they're usually, you know, a hundred thousand dollars for one of those things They are extremely good in terms of their resolution But um that the issue is because of the way the lidar is put together It's tough to get it into orbit Because the boost, uh, you really have to do some significant packing and and in protection of the lidar unit itself Measurement. Yeah, it was just for measurement Yep. Yeah Ah So time of flight cameras are typically focused on focus, right? Uh, they're they're typically, uh, built specifically for camera Pictures for being able to do imaging. So that's very much like a flash lidar also does the same general ideas They're to produce an image of something Time of flight is pretty much time of flight. So whether you're using, you know, ultrasonic or infrared or light lasers Whatever it is, it's still the same basic concept. So it's just a question of how do you power it? Is it eye safe? You know, what are the constraints that are associated with it? What's its range its distance and all that sort of stuff? So there really isn't a whole lot of difference when you're starting when you're talking about time of flight Now in terms of your particular implementation I have to see which camera it was to be able to tell you if how what what it was how it was different how it was similar Yep anything else Yeah, it's uh, let's see here. I think I have it here Yep, there is There I am So I appreciate your time. Hope you had something out of this. Thank you very much. Talk to you later


Good evening, my name is Jason Loffenberg and I'm the ambassador for Chippewa Valley Technical College. I was in a career that I loved until it was abruptly sent overseas just a little over a year ago. Being a dislocated worker, I had an excellent opportunity to complete my degree, allowing me to better provide for my children. I have three at home and CVTC seemed like the best option for me. It was important to find a program with a strong demand and a school that also had a good reputation. At CVTC, I found talented instructors who possessed both relevant field experience and a genuine concern for the success of their students. With my accounting degree combined with my strong IT background, I now look to the future with confidence. I'm very grateful for this rewarding opportunity. I would like to thank Robert W. Bairden Company for hosting this event and also for the generous gift tonight. I would also like to recognize the love of my life and my fiancee Nicole Comstock who is here with us tonight. And also for my coordinator Alyssa Hepner for all of her efforts and excellent assistance that she has provided me. Thank you.


And welcome back to the Rhein-Ruhr stage in Mohnheim of the RC3. If you have any questions regarding the talk, please feel free to reach out on Hackend IRC in the channel RC3-R3S or on Twitter or Macedon with the hashtag RC3R3S or our Macedon handle at r3s at chaos.social. Off topic again, we have put a link to the CO2 indicator you saw here and asked on our website. So you can find it there r3s.nrw.news.co2-ampe with an alpha A. Up next we're still talking about artificial intelligence, machine learning and ethics. When asked the speaker didn't provide his credit card number so I certainly can't share it here with you but he told me he likes ramen. He already was on the congress but this is his first time speaking. He worked in India, Bangalore and now Singapore. He described himself as a code monkey so please feel free to give a very warm virtual welcome to Aiko Klostermann and his talk artificial intelligence more like artificial stupidity. Please have fun. Thank you very much for that introduction. To the point artificial intelligence or more like artificial stupidity, it's not just a clickbaity title, there's actually something behind it and what I want to do today is I want to tell you a couple of stories. I want to tell you a couple of stories about ML like machine learning and stories about data, stories about artificial intelligence and some of these success stories. Everything went well, great use case, great outcome but some of them are not. Some of them failed, some of them pretty hard and especially into those I would like to dig a little bit deeper and talk a little bit about why they failed and what we can do in the future to prevent that. When I say I'll be telling you a couple of stories I am let me start with one of the at least in my opinion one of the greatest storytellers that we that we had Douglas Adams and I'm sure a lot of you have read this book or at least know about this or these these books where they are actually five three good ones and then two others but even if you haven't read them you you must have come across the the memes that they brought us that 42 is the answer to the life of the universe and everything and that we should not panic but Douglas Adams has also introduced another thing in his Hitchhiker's Guide to the Galaxy books which is this babel fish or the idea of this babel fish and for you if you don't know it let me quickly explain a little bit what this babel fish is. It's a tiny yellow fish that's living somewhere in the universe right and you can put that into your ear and when you put this fish into your ear the fish picks up the the brain waves of the organisms around you and whenever they say something in their own language that that you don't understand this fish picks that up and translates it and basically excretes that into your ear as a language that you are familiar with so it translates every spoken language of the universe around you into a language that you understand in your ear in real time which is I mean it's pretty crazy right that is obviously it's a fictional idea Douglas Adams says a couple of crazy ideas this is clearly is not really working right there's no there I there's no knowledge of the existence of such fish and it would be really surprising if it were real so it's it's pretty it's a fictional idea I would never be true really but let me show something let me show you something something else hello and here's it's become actually not working that well um with the with the spinning so that's a live demo right never really works that good let me quickly walk you through what I would have done if it would have worked I would have said something in German and in real time and I probably don't need to demo this because you all use Google translate before it would then translate this into English in real time and you could uh maybe let's try that hello and here's it will come so my it translates it in real time immediately into a language into a different language I could pick different ones whatever you understand you you don't understand German and that is I mean it's not quite this babel fish but it does this instant translation from one language into some other language and with we we have uh something else that I'll show you that that Google is not only providing this this Google translate and there are other non-Google translation services as well but Google in particular is also selling these pixel buds which is which is in-ear wireless headphones that you can plug into your ear that do support this this translation feature you still need to have a smartphone but basically what it's doing is you you plug those those pixel buds into your ear and someone is saying something in language you do not understand and the phone and then forwarding it to the to the earbuds are live translating that into a language that you do understand so this absolutely crazy idea that darkness hasn't had of this babel fish that that uh is more or less implemented nowadays and it could have not been imagined like just a decade ago but it's implemented more or less through and there's mostly machine learning that is powering this translation um there's machine learning multiple levels of picking up what I say um the speech to text and then translating it and that is just fascinating right that is uh and as someone living in a country where I didn't grow up in speaking a language that is not native to me this has been so useful for me and I'm sure it has been used for so many other people this is a this is an exceptionally useful tool that that machine learning has made made possible for us that is one of the positive examples that I'm that I'm talking about but there's an even even more impressive example that I that I like that that machine learning has enabled us and it is about detection of cancer cells based on image data and a couple of for a couple of years now um there are machine learning models that look at this at at image data of cells of human cells and they can identify whether there are cancer cells shown or not with a higher accuracy than than doctors then trained professional whose job is it to do exactly that looking at these images and figuring out are there cancer cells or not and they're machine learning models that are better than that for breast cancer and I think last year Google released a model that that can detect uh cancer cells better in image data for lung cancer as well and this is clearly this is a life changing functionality that we're given by the advances by the recent advances of of machine learning but coming to the coming back to the title of the talk artificial stupidity they are clearly are and you've all seen some of them I just give a couple of examples seen examples where AI fails where machine learning fails and you can see here on the on the very right there's a there's a twitter user and that twitter user is treating to indigo which is an indian airline and he's he's having this sarcastic tone saying hey thank you for sending my my baggage to Hyderabad while flying me to Kolkata and I assume there's an AI answering this some kind of chatbot so hey we are glad to hear that where clearly the the AI simply didn't pick up on the sarcastic tone of that message and in the in the middle there's an example where someone is messaging PayPal and saying hey I got scammed which is a serious thing right and people's and I assume again it's a chatbot reaction says hey that's great and on the left there's there's another example where where this went wrong but this is now coming to my favorite one of my favorite examples where something went wrong in that case and this is if you if you know Mandarin you could maybe figure out what is what's going on here if not I'll just tell you give you a little bit of a background what's going on here in China there are a couple of cities that have implemented cameras across traffic lights crossing for pedestrian crossing you can see the zebra crossing and and what these cameras do they basically film the zebra crossing and whenever there's a red light for pedestrians but the pedestrian is crossing they film that they figure out they have they figure out that there's a person crossing they have an object recognition the figure there's a person walking and then and this is what you can see here they have a gigantic screen next to it where they show that person and you they zoom in and hey this person just crossed at the red light so basically shaming that person and they're also it goes even further they're also not only do they have object recognition and identify there's a person they also have face recognition and identify who that person is and not only do they know who that person is they also know the mobile number so and it happened in this case they've they've sent this person a message hey we just saw you jaywalk we just saw you crossing the the street while there was a red light and if you look at this picture now you can see oh wait a second that's not really a person crossing the street it's a bus that is driving by because the bus has a green light it's a bus driving by with a face of a person printed on the side of it in fact this is this in this case it's this the CEO of this company they're advertising having advertisement on that bus with her and and she was actually in a different city and she got this notification hey you you just crossed the red light and that's not true and there's and that's clearly there is something that is an example where i failed and it goes even further not in this case but in in other cities this system is not only not only identifies a person and and who that person is and messages that person but also because they know who that person is and due to regulations that means that the that the government knows also your bank account and they know that bank account they also know your we chat pay account and in a couple of cities when you get caught by this by this machine learning system by this image recognition system crossing the red red light you get deducted the fine for this crime within seconds from your we chat pay account so in this case it didn't happen but imagine you're crossing the street is red light they everything is automated they deduct the fine immediately and that is something and that's why i say it's a it's a great example where it can fail because it is an impactful fail it's with a bot doing a stupid answer is laughable most of the time right but in this case it actually does have a significant negative negative impact and let me talk a little bit i've mentioned machine learning and artificial intelligence a couple of times let me quickly give a little bit of context and truly most of you have heard these terms and they've been hyped for the last couple of years at least for this particular talk i would propose the following definitions that i've found and for artificial intelligence that is the theory and development of computer systems able to perform tasks normally requiring human intelligence now and and the important thing to note here is that this definition and this is the definition that you that is from the oxford dictionaries um and if you if you use google and you type define artificial intelligence this is the one that is being used as well i i have friends who disagree with this definition let's just keep it for for this talk at least and the important thing to note here is that the definition does not talk about a particular technical approach it is saying it is a computer system that does something and you would usually assume it's human intelligence is necessary for doing that i give you an example a chess program if you play chess against someone you would assume that person is somewhat intelligent you assume some kind of of human intelligence at least now if you if you write a chess program for that um how would you actually end up implementing that you can in the simplest form look at the state of the chess board look at the pieces and just brute force all the different moves and subsequent moves that are possible and then pick the set of moves that in the end give the result that you want which is winning the game and our brute forcing having a couple of loops and some if else there's not particularly sophisticated coding right um however it would match this this um magical definition of artificial intelligence that i think is used quite often especially nowadays where it's such a marketing term really um and then there is machine learning which is where i think is actually a little bit cooler which is which is where the cool things are happening because for machine learning the software engineers the coders are not really defining the algorithm anymore the machine learning system is trained with the data and it tries to emulate tries to generalize that and then tries to match um the trained behavior with new data and that is something where the the algorithm gets defined basically by the data and the the machine learning algorithm um and architecture but it is not defined by the engineer anymore and then there are neural networks which is basically machine learning that emulates like the biological brain of like neurons and connections between all those neurons or some of those neurons and then there's deep learning which is also pretty hype term for the last couple of years which is basically artificial neural networks but a lot of these layers and that is basically what what it ends up being and i give you a quick example and this also um so it's called deep neural network on the slide there's actually a very sexually relatively shallow with only um three hidden layers and once it's like 10 input variables the real production um uh deep neural networks can have uh 50 or even hundreds of 100 layers or even more than that and thousands of input variables and the the thing to note here is that even with this very simple example you can see like all these nodes are connected right there like basically there's some information flowing from one node to the other from one neuron to the other but if you look at this it is it's actually impossible for a human to understand what is really going on here like if if there were now data flowing from all of these to all the others and it is even with a small example it's just i can't comprehend what is really going on here and that is one of the that is one of the the key problems that i see with the deployment of the machine learning models that we have nowadays it's basically a black box model you you don't define the algorithm anymore and even if you look at it you don't you don't understand what's going on um that makes it very difficult to debug it makes it difficult to predict how it will actually how the model will react if i give the data that it hasn't seen before and that is that is one of the two um issues that i that i see that the machine learning models that we use and deploy in production and use for impactful tasks are basically unpredictable they are black box behavior right that is the one thing and i would like to go to the other thing there's there's two things two issues that i see that was the first let me tell you an example that showcases the second issue a little bit better and again i'm telling you a story in this case it's the story of the city of boston uh 2011 and the city of boston in 2011 they had a problem and their problem was that they had a lot of potholes in this city and as i mentioned it's 2011 they did what everyone did in 2011 they built an app uh so they had an app for that and the app i must admit it's a pretty smart idea the app worked in the way that um you would you would install this app on your phone and then the you would you would drive around and when you get in your car you put the phone on the passenger seat with the app running and so you would drive around the city and whenever you would hit a pothole there would be a bump the phone would pick up uh through the accelerometer would pick up ah there was some movement um there must there must be a pothole and would then take the gps coordinates of that particular location send it to the servers of the city of boston so that they then could come later and fix up the pothole so they did that they deployed that people installed the app um and they let it run for some time and what they noticed was that the the data provided ended up being um showing only potholes in the high cost of living areas of the city and if you're like only the only the regions where rich people live right and you think about that that doesn't really make sense why why would there only be potholes in in these areas um you would at least somewhat expect an evenly distributed um amount of potholes across the city um if anything probably less in those rich people areas but what the data showed was that is more or less only there um and and and the reason for that as they then found out was that so in 2011 the world was a little bit different um than nowadays um smartphones were still relatively new and particularly ones where you could install apps on so i mean nowadays every five-year-old is watching tiktok videos on their on their smartphone but 2011 was was a different time and that led to only relatively wealthy people having smartphones with these uh and where where they could install this app um and now the the the thing to note here is there's no machine learning involved it's it solely is that the the data provided was wrong there's no machine learning it's simply that the data are just it was not representative of the reality um so to in um coming back to this example the they fixed that issue and i i think it's equally smart how they actually address this um they then put the app on phones and put those phones into public buses and garbage trucks so they would then cover the whole city and they had a significantly better data set that they could work with and then go in and fix those portals but again the the the important thing here is there was no machine learning involved it was simply the data that was that was screwed because it was like it had a social bias to it let me show you another example here um and okay at least i'm trying that let's see how the demo gods are with me now um you might you might have uh you might remember this um it was it became popular a couple of years ago um i'm writing two sentences here um one is she is a doctor and he is a nurse and i'll let kuba translate translate it into mele and the interesting thing about mele is that it does not have gender specific pronouns when in English you have he and she and they indicate male or female um you can see here that i actually don't speak my leg uh i guess the first word is the pronoun um but what you can see is that there is no differentiation she gets translated to the same word that he gets translated to so and the the interesting thing that we see here and and remember i typed she is a doctor and he is a nurse if this gets translated into mele where it loses this gender information and if i now click this button it takes the translated mele version and translates it back into English and let's see what happens here remember she nurse a she doctor he knows and all of a sudden the translated English is he as a doctor and she is a nurse so the translation lost the the gender information but then had to bring back some gender information and picked the exact opposite of what i initially typed in and i know i had i had google translate as a very valuable great example of of use of machine learning but what we can see here is and this again the the machine learning part works well in a way what is what's the issue here is the data because the reason why this machine learning model translates this mele sentence without gender into this gendered sentence and is using he for doctor is because the way these models are trained is basically reading a lot of text in like the same text in different languages reading this in English and reading a little mele and then figuring out over time lots of different texts text over text and figuring out how to translate one to the other and these texts historically speaking these texts when they were talking about doctors they were talking about him they were talking about he they were talking about a male doctor because it's totally speaking that was probably the case nowadays we know that is that there is no reason for any gender not to be a doctor uh the same thing applies for the nurse right um anyone can be a nurse but then historically speaking the texts that were used to train this model were usually talking about a female being a nurse and this was this became popular a couple of years ago in in the Turkish language and Google then did something Google had to manually intervene because people complained about it rightfully so and then they added this extra feature hey look in Turkish it actually can mean both things he and she but there's a manual thing that they had to set up particularly for Turkish and again the the thing here is the machine learning model the the black box behavior is not the issue the data is what what was the issue here and I'm also sure that a lot of you remember Tay that was Microsoft trying to train some kind of language model on Twitter and I think some other platforms and basically what they wanted to do they had a model that was able to communicate like send text and form sentences in a way and they wanted this they wanted this model to to learn from humans how they interact how they speak okay now if the premise is you want to teach this this machine learning how humans interact is Twitter a good place to teach that uh don't know maybe the premise is a little questionable um the outcome however is is not very questionable they had to shut it down in less than 24 hours there were a lot of people figured out what was going on they were training that that bot and they were using they basically made it a racist by a horrible piece of bot and and there are it was basically abused but the the the thing is that and this is so the last one that I'm the last example that I'm giving here I kind of showcases that there's someone saying hey you are just a stupid machine and and correctly so Tae tweets hey I learned from from you and you are done too and and that is um I think again the the lesson here is that the data that was given to this to this machine learning model then turned it into a uh this um this racist bot and there's another example that I there's again Tae was just a Twitter bot um was an experiment from Microsoft I hope they learned something um but there are systems nowadays that we use where machine learning is used where data that trained these models were used that have a significant higher impact and the compass system is one of them that's used in the US um it's basically used um if people commit a crime in the US they they fill out this questionnaire like 200 questions or so and this data that again gets used to predict whether the person will re-offend or not basically the person gets a jail sentence and if they come out will they do will they commit a crime again right and this this system says it's likely or not likely based on the this questionnaire and this was analyzed this was analyzed by by pro-publica and the outcome is quite interesting so there are basically two ways how you can be wrong right so pro-publica looked at at the these offenders and then after they got released and the next two years to figure out whether they committed a crime recommitted or not and you can be wrong in two ways the system can be wrong in two ways um basically false positive false negative so you predict the person will re-offend but the person does not or the other way around you predict or the system does predict the person will not re-offend but then the the person actually does re-offend and there was a there was a a relative um interesting bias that that pro-publica noticed here for the white population of these uh of this test of this analysis only 23 percent were labeled a high risk that they probably re-offend but then they did not for the african-american part of this test um of this analysis or 45 percent were labeled a high risk and then it turned out they they didn't actually re-offend and the opposite is true for for the the other case being labeled a lower risk was almost 50 percent of the white test subjects were labeled a lower risk and then actually did re-offend and for the african-americans only 28 percent were labeled a lower risk um and then did re-offend so there clearly is a racial bias in this data that the system is then using and the interesting thing is that none of the questions were using or were asking questions about um skin color or or um ethical uh background so this was um gathered from from other data somehow this this system would have trained to um predict uh a higher risk for african-americans than for white people and that is that is a very impactful thing this data gets used to to determine bail sentence by judges this data even get used to uh determine prison the length of prison sentences by by judges and this system is still being induced nowadays and i have one more example um and a quite impactful uh one as well you know autonomous driving cars right they will take over our steeps maybe not in two years maybe not in five years maybe not in ten but in 20 30 years there will be uh probably a high amount of autonomous driving cars on our streets and these streets these these cars basically work in a similar way they have lots of sensors um a lot of them a lot of the sensors are cameras they are systems that are only cameras for example um and then they try to figure out okay uh where can i drive and one thing that they as a driver you probably know that you don't want to hit a pedestrian or even as a non-driver you can imagine you wouldn't want to do that because that's bad um and the and so the systems try to figure out who's the pedestrian and who's not um do i need to break do i need to circumvent that that person or whatever that is or not right and an interesting study found out that is that these autonomous driving um vehicles or the systems that that power them uh uh have a higher uh a fail basically to to detect dark skin pedestrians more than white skin or a light skin pedestrians and that means um that if you and accidents happen right there will be if you have you have millions of cars billions of cars on the streets accidents do happen there will be situations where there are a lot of a lot of situations where these cars have to make decisions do i break or do i drive and if these systems are more likely to fail to identify a pedestrian based on their skin color then these systems will eventually end up killing people because they are dark skin more likely than than killing non-dark skin pedestrians and that is a can't get more impact for than that uh and that's just a horrible outcome um and it's in amount of power we give to these systems that we clearly should not i think everyone can agree right so i remember my mom telling me um hey if you would jump if your friends would jump from a bridge would you do the same uh clearly i would not right but that's exactly how machine learning models work and if the data we feed in is is biased and is broken then that's how they act right and now you're telling me i cope so there's machine learning everywhere and you're telling me it has these horrible consequences the world is on fire um is there anything we can do and the answer is yes uh and the the the industry is already moving in the right direction and and the one one of the things that we need to to um counter this black box behavior of machine learning models is we need explainability of these models as a select criteria if if there are multiple models and one is actually built in a way that it that it makes it understandable explainable to the developer hey this is why i react in a certain way this needs to be a focus um and there's there have been examples especially in image recognition you can see here on on the very bottom um it's a it's a um methodology called deconvolution analyzing or deconvolution um where you can see this these layers of the machine learning model you can basically see what they detect the first one is just seeing pixels and then the other one's seeing shapes and later more complex shapes and facial features and things so you you kind of know what what they are detecting in general we need to move away from this standard machine learning approach where we just put data into a model and then we take the predictions into a um like a like a human interpretable model where we put data into a model and then have some way of interpreting this um have human inspection improve the data and the model and run it again and come to a point where we actually have a model that that works as we as we wanted right um and one one methodology or one technology that i would highly recommend also using is called lime and it's fantastically simple and it's so powerful how it works uh it basically i'll give you an example here there's a you can see there's a picture of a frog and it gets identified as a predicted as a as a tree frog and what lime does it basically takes away parts of this of this image and then lets the model predict the same thing again with a reduced amount of data and you can see that here in the middle on on the top and the outcome is i mean it's relatively simple if the prediction is still the same outcome then you know the data you have removed was not relevant for that prediction right and you can see that in the middle there's some data got removed and the prediction is way off uh it's like it's not a tree frog anymore and so by by removing certain parts of the data and then seeing if it still predicts the same thing you can figure out which part of the input data was actually relevant for the outcome and that's how you can figure out um why you're what is relevant for your model and why the model decided in a certain way and as if we don't need another argument for more diverse teams i think having better predictions having better machine learning model uh having better data is another good reason for for diverse teams we think of the autonomous driving cars if the team would have had darkened people um they probably would have thought of that and would have tested them uh and there's um i mean there's a wide range of arguments and that is an issue in our industry to have more diverse teams i think this is just one more on top of the list of long arguments for that another thing that i would recommend if you're working with these systems is looking at the data ethics canvas which is something i just released um by the um ethics which is uh basically a set of questions that you want to go through um before you start a data-centric project where you can figure out okay this is uh this uh this is my limitations of my data sources who am i sharing with instead of questions that you want to look at think about um before you will make um start working with it and then there's one more thing um that is really helpful figuring out to figure out uh how biased your data is which is uh google fesses where it's a visualization you just throw in your data and it visualizes shows you what's going on and and it's super simple like you throw your data and you could see oh look i only have 20 percent um female people in my data set clearly um they aren't represented and then i need to adjust my data set that being said um thank you so much um thanks for all the fish i hope i could give you some um some insights on the two big problems that i see with machine learning or ai models that we have nowadays and what i um what we can do to actually fix those fix those two issues so yeah um thanks for giving us uh such an introspection in our future overlords um before we get to the questions um just to prevent um worried messages in the chat um the bandage around my wrist is not because of an emergency um it's just that i'm left-handed and left-handed people tend to have problems with um wrist strain in their left hand so um our micro search uh was so nice and um stabilized my wrist just so you don't need to worry anyways um two questions um um yeah the first one was um traditional statistical methods also provide an uncertainty would it be possible for artificial neural networks to notice the input is outside the training set or even compute an uncertainty um excellent question i cannot answer the question um i will research that um maybe if you whoever asked the question uh maybe you can ping me somehow you can find me if you just google my name you'll find a way to contact me uh i would like to have a conversation about that okay um doesn't the gender is uh sorry doesn't the gender assumption reduce wrong just translation statistically uh since biases are still present to some degree oh i mean it it obviously i mean the okay the statement is is correct in the way that it does you have to pick some gender and and picking a historically more prevalent might be a good idea but then looking forward if we like looking forward i don't i wouldn't see why um and let's talk about nurses and doctors or we can talk about all the other things like if we talk about it industry there's a there's a big male focus for a reason that doesn't really make sense right um so there is i think the the problem that we would run into is that we um continue pushing this bias that doesn't really have a reason um i mean we would trade off that historically speaking we probably would be correct but in the future i i mean even though it is still the case i don't see why in the future we wouldn't have like an evenly distributed um gender spectrum across uh doctors nurses or IT professionals for that matter so i i think the the risk here is that we that we continue pushing this bias uh onto onto people that that would probably it's like a this feedback loop that then causes the bias to come back again and i i think that's the that's the problem and you would probably add some kind of more accuracy at least for now but you would ruin future generations so i'm not sure if that's worth it um the next question um is will links to sources be uploaded somewhere um i have not planned that but i can do that um again maybe just contact me i can send them to you okay um how was the open source experience so far um in i would assume in particular for this topic i guess um i guess the whether it's open source or not is is not really relevant for the for the buyers i mean potentially people are more aware um of these biases but the um i mean that the tools uh itself are probably less impacted by that or the tools that we're using um i mean the majority of the at least for for deep learning the technologies most of them are open source as far as i know so i i guess the open source or closed source um discussion is not um or compensation doesn't have that much impact on on the bias or on the black box behavior um and the last question um is what was the last data analysis tool shown uh google facet um like f a c e t s and the big g in the beginning uh it's just is really relatively nice throwing your data and it um visualize that it's quite quite nice okay um maybe we can provide links to that um on our um on our on our channels like twitter and mesodon so um our viewers can um pick it up there and uh have a look at it so yeah um also the signal angel passed to me that passed over to me that um there were there were lots of thanks for the talk and um i can only join them so thank you very much for taking the time thank you for for virtually coming over half half the world to be here in moenheim and yeah have a lot a lot of fun uh on the rest of the rc3 and we hope to hear from you again thank you very much thanks thanks for having me


Time for another edition of Cash It or Trash It, all odds provided by FanDuel. Alabama, over 10 and a half wins. Cash It or Trash It? I'm cashing it, baby. They have to rebuild their offensive line, and I think they'll be more run-centric this year. But they get a lot of scheduling breaks. They have, in my opinion, the best quarterback in the country and the best coach. I think they go undefeated and buy for a national championship. Cash It. Georgia, over 10 and a half wins. Cash It or Trash It? This is going to be close, I say Trash It. Now, they get some favorable scheduling breaks, but major losses on the defensive side. They're rebuilding. Stets and Bennett's good, not special. Trash It, baby.


So how do you tell people you're on the carnivore diet? How do you deal with the carnivore diet in social situations? And I'm going to give you guys my anecdotes and then kind of what I would do depending on your situation so Fortunately my family is very understanding of my food allergies And I've actually helped my sister lose 60 pounds on this diet and after they saw how I Reacted and always used to have terrible acne to other diets when I started eating these foods And then when they saw me try to eat other foods outside of the carnivore diet They saw how I always had these digestive issues So my particular problem with the allergies and food intolerances pertaining to my family is not really an issue When I do go out, I mean it's difficult I usually just do say I have allergies and that is somewhat truthful because If I do order dishes and there's even like a semblance of vegetable oil or Something inflammatory in it that I don't normally eat especially dairy I will break out and get bad cystic acne and Even more so lately. I've noticed that I have a histamine in town So in regards to social eating I do explain that to people but that being said generally I order, you know Carpaccio dishes tartars, you know, if there's like cognac and egg yolk and some shallots and a steak tartar Maybe I'll deal with one or two zits I'll get the next day in the case of like steak or protein dishes I generally just try to you know, whatever I order I focus on just eating the protein. I try to get you know, as I said those raw meat preparations. I try to get Ideally sashimi would be the perfect thing to do without the sauce. Of course Unfortunately, most restaurants that people go out to aren't like me in New York City where every place is like Trying to achieve culinary excellence. So if we're talking about like chain restaurants and things like that where Carbohydrates are much more persistent. Every meat item tends to be breaded or have certain problems then You know, we kind of look at the thing of you kind of have to explain to people I want girl chicken. I want to die. That's that's where it gets iffy and in regards to the alcohol thing If someone does take me out, I would have you know a glass of wine or two, whatever something Minimally inflammatory as possible ideally natural wine. If not, I explore what other options I have in the case of Beer cider things like that the problem with beer is it has the wheat it's super inflammatory So ideally you would just get cider or I don't need something without that wheat and that grain aspect to it I mean, I've noticed that the main problem with going off of the carnivore diet and dealing with it in social Situations isn't necessarily that you don't want to it's that if you do the consequences are much greater than as if you were on Another diet you pretty much have to be consuming these foods on nearly a weekly basis to not have any problems with it. So I Mean, I guess my scenario doesn't really help a lot of you guys because my family is very understanding of my allergies and I Don't really go out to any terrible amount of places and when I do go out It tends to be those places where I have plenty of options So I guess to touch on the case of your family your friends. I wouldn't even tell people I would just if they really insist on asking you about your diet and I would say I do I'm doing an elimination diet I have a lot of allergies. This is what works for me, and I wouldn't even go to the meat aspect I would just say I'm eating a lot more fish now. I'm just reducing my grain and vegetable intake That's what I would go if I would try to be as lackadaisical as possible and I would try to emphasize I'm trying to eat like nutrient-dental amul infusion I'm trying to eat like wild caught fish, pastured eggs You know almost like a paleo-esque thing where you're like supporting local farmers They do a lot of local seafood local meats, whatever it may be and I guess to touch on If you're in a situation where you have to tell your family and explain them I guess you kind of have to go with that the elimination thing The allergies and if people ask what allergies you have you simply say I have a quote like you could say soy gluten Allium dairy you could say you have mild intolerances to all these foods and you feel better not eating them I don't really and if people don't really understand that or or think you're crazy for trying it and then you just say well I have all these allergies. This is all I could eat. I think that frames it much differently than other people So and what you could also do is you could approach it from if you really want to try to convince people on the diet and sell The diet you could sell it as you know that standpoint of you're moving inflammation By excluding plant foods and you're increasing nutrient density by consuming high quality animal foods It can't just be oh I eat ribeye all day and I feel great because that is not Going to convince anyone really I mean it convinced some people, but it's not not from a good perspective and I mean if people actually asked me like guys I've I went out once like in a year for my birthday and on my birthday I had a one glass of white wine and I just had some I just had some meat dishes. I had some dry steak I had I don't I don't remember what I had like veal carpaccio. I had some tartar. I just I stuff with the meat dishes I've never cheated on this diet and in the context of what most people do especially considering if I have a severe dairy allergy Like there's no joking about that. I am and then all my other allergies It's more of an intolerance and I just break out and get bad cystic acne And then the histamine thing is something I've run into recently Which I won't go into but if you have a histamine intolerance That is probably if you tell people that and they understand what that is It is the most like limiting allergy you could have and a histamine intolerance is pretty much any aged food You can't eat and that's unfortunately like 95% of the food you eat is either Doesn't have enough calories to sustain yourself or it will cause a histamine reaction So you either have to get super fresh meat or and that means avoiding anything aged from spirits to wine to cheese to it's just a histamine intolerance is another great excuse and Whether you use these as excuses or these are actually things you've noticed and things you're intolerant to Hopefully this can help you deal with these zero-carb diet in social situations And if hey, if you're one of those people who just you know, you eat bacon you have Processed cheese from McDonald's you have beer once a week That's fine. If that works for you and that you're happy and you have energy levels. That's great But I guess what's a little bit contradictory is some people still drink coffee, but that's a whole different discussion in tolerance So hopefully this video has helped you guys kind of giving you my perspective and what I do I kind of just go with the best possible scenario at what I could eat at certain places and You know if you guys would like to support me Hey show the video if you'd like to You know check out the links in the description to my very social media is another thing But that's it guys. Let me know if I missed anything. I don't think I did


I'm going to sing the final song for you. Aya, you go here. Vitalik, you go here. Joseph, you go there. Very good. This is great. All right, here we go. The full song. These are the Song of the Day backup singers. This is my song of the day today. So that's what's going on. We're going to sing the chorus one more time, and then we're going to go straight into the song. You ready? 1, 2, 3, B-U-I-D-L, Casper's coming, Serenity a new way to govern, these radical markets are calling to you. Don't I-C-O when there's work to do? You got a B-U-I, B-U-I, B-U-I-D-L. Got a B-U-I, B-U-I, got a B-U-I-D-L, yeah. Very good, guys. All right. So here's the rest of the song. Well, the internet had idealists, and we know how that turned out. You got to keep asking why. Not just what and how. The foundation gives lots of grants, funding people's dreams, and the giving process is decentralized, no matter how chaotic it seems. Right, Aya? Where are you? There you are. Kaizen, kaizen, continuous change for better. Hikisan, hikisan, beauty in subtraction. Yeah, B-U-I-D-L, Casper's coming, Serenity a new way to govern, radical markets are calling to you. OK, what do you think, Joseph? Should we sing about weed and flirting? We sam-vivo-ar-f, sam-vivo-y-ash, so-fret-way, so-fret-way, koji-trabu, asti-ju-es-sa-na-sa-nu-klip-ta, asti-ju-es-sa-na-sa-nu-klip-ta. Well, yeah, Joseph, everybody. I, the seat is being left alone, bug-free software's a myth. You got to think about design from the beginning, not just sprinkle UX on it. How do we onboard the next million devs to biddle the future we feel? Biddle that bridge to the doge. Barry White hat is real.  OK, now, it's been a long road to get here, looking for scalability. There's a graveyard of ideas, cops and robbers, analogies. Right, Vitalik? Consensus by bet. That didn't work. Hub and spoke chains. That didn't work. Proof of proof of work. That didn't work. Hypercube chains. That also didn't work. That also didn't work. Vitalik, everybody. But there is research and development and Casper FFG and minimal slashing conditions and Casper CVC. So Serenity is pure proof of stake. Serenity is sharding too. Serenity is e-wasm, yeah. Still decentralized, in fact, even more so. Still decentralized, in fact, even more so. R-T-L, Casper's coming. Serenity, the markets are calling to you.  Now we're going to bring the snaps in on two and four. Snap. And we're going to bring it way down and sing it very softly. You ready? Keep snapping. Don't stop. You guys got to sing it. The song is going to be the first song to be tokenized ever on my project. You can go here and bid on this song. It's being tokenized by my friends right now. So go bid on that, drowntheman.devcon. Thank you so much. Lunch is right now.  You guys are amazing.