Well, thank you for coming My name is Brian Borum. I work for a company called WeaveWorks So let me just check who's in the room Who had heard of WeaveNet before? Before they read the schedule about half half the room. That's okay and who who identifies as like a kernel developer or a device driver or DPTK developer or They've all left. Okay. That's great. So so we can because I I put on this slide. I am not a networking expert. I'm I'm like a programmer and I've been looking after this project for for five years It's been downloaded 250 million times. It's Starred 5,000 times. There's There's a certain things to be proud of but It's yeah, fundamentally, I don't think I did anything clever so so I'm kind of glad that all those all the clever people have left the room and I Put my smiling face up Somewhat because I'm gonna put a bunch of people up You know, this is a talk somewhat about the technology and somewhat about the people and the history of this project So hopefully that's interesting So What is we've net it's a container network and I'll talk about that in a minute The Primary thing we were aiming for is that it's easy to install it just works It runs anywhere and there's a little asterisk because we mean anywhere that is Linux But Windows like runs Linux now We have tried this you can actually run weave net on WSL so Nearly everywhere And we never it's open source. It's Apache licensed. We never made an enterprise version So enjoy Yeah, what is a container network Well, we had we had one definition from Justin Garrison works for Disney I'm an environment of his work and generally the work of Disney. It's good stuff But he said there's no such thing as container networking So that was a bummer because I've been working on it for five years But actually turns out He uses we've net and it just works So yes, so So more seriously, you know, what is a container network? So the point of containers is is is or one point at least is isolation That through through namespaces and kernel features One thing is sort of believes it's completely separate from another thing Including network. They have separate network namespaces. So now how do these things talk to each other? Well, whatever the answer to that question is that's a container network. That's that's the definition. I'm gonna work with Okay, let's go back Okay, let's go here Conceptually, what does this mean? So I'm gonna draw I'm gonna put up a large number of Diagrams that look a little bit like this So the the meaning of the shapes the big darker blue shape Is is a machine whether it's bare metal or VM or something like that That's that's your that's your kind of node in the network machine and the light blue blobs are containers Okay So sometimes containers are talking to each other on the same machine sometimes they're talking to each other on different machines and By and large we're gonna have like lots of these things And they're all gonna be talking at once in in different amounts Okay, so that's the kind of theoretical high-level model that we want to that's what we want to do Go back. Let's go back five years five and a half years this smiling face Mr. Sackman wrote the first version of what became weave net and He was Out of rabbit MQ in fact the the founders of we've works all came from rabbit MQ So here's a Erlang programmer and the the code a Lot of the code has quite a strong. It's all written in go But it's a strong kind of Erlang flavor to it Which is kind of cool if if you want to think about that 3400 lines is the first commit and Spoiler we're at like 30,000 lines now, so it's it's it's rolled a bit Anyway, so basically what we do That version Fundamentally what we do is we we put a bridge a Linux bridge on each Machine we connect all the containers on one machine to the bridge and then we Tunnel the packets from one bridge to another bridge So that's the kind of the the the conceptual model taken to a natural implementation. Let's take that down a layer further So specifically we set up a bridge for each container we set up a veth virtual ethernet device One end is inside the container namespace one side is attached to the bridge in the host and we Listen on that bridge using pcap The room went quiet No, seriously, we tested three different ways to do it like tap devices and whatever and as that five years ago in go as it Stood pick up work best so So there it is So if you've got If you've got two containers on the same host, they're both attached to the same bridge They'll just talk to each other other over the bridge and and and we've net doesn't get involved If you have two containers on different machines Then we're going to pick up those packets. We're going to put them in Another UDP packet it's kind of homegrown encapsulation and we're going to send that over the network and and Deliver it to the bridge again via pcap packet injection on the other side deliver it to the bridge and That's going to end up at its destination This is I like to call it at least a distributed ethernet switch it is we've net implements an ethernet switch It's a layer to network It works purely in terms of MAC addresses It does what a what a dumb ethernet switch does it learns MAC addresses by seeing a packet come in And it and it observes the source address that it came from and when it later on it has a packet to send to that destination it uses what it learned And it delivers the The packets that way so so physical Ethernet switch is going to deliver on different cables. We're in software. We're going to send to different hosts It's exactly the same concept We also have the same felt fallback behavior if we don't know where it's supposed to go send it everywhere that behavior will come in useful later This is a real website Somebody was kind I didn't put their smiling face up Somebody was kind enough to make this website I'm pointing out something that we did actually know But if I if I step back Can I step back? Yeah, okay, so yeah, the reason it's kind of slow is We start off in user space here in the program that's trying to get some work done We go down into the kernel here. We go back up into user space through p-cap We put it in a UDP packet. We go down into the kernel again across the physical network And then do the same thing again.  It's terribly so it adds We used to measure it like five years ago used to measure at 300 microseconds per packet extra latency and Now, you know, you have to set this aside What are you actually going to do with those packets if the next thing that happens is they get delivered to a massive heap of PHP code then 300 microseconds is not your problem But whatever the the yeah, it's kind of slow. Okay, so next next step in the evaluation We implemented What we call the fast data path because we have no imagination when thinking of names So kind of similar picture The packet starts off in a container again, we've attached a v-th The other end of the v-th now is in a in a different device, which is a v-switch data path So this is implemented by a kernel module from the open v-switch project This is the only piece of the open v-switch project that we use So we're in these kind of these demon processes are basically implementing our control plane independently of open v-switch, but we're using their kernel module So it takes the place of the bridge and at least in this version of the code and we add we add a few kind of bridge-like behaviors to it to get everything we need out of it, but once a Source destination Mac pair has been seen to be talking on the container network We we set up a VXLan tunnel and that goes kernel to kernel So the the packets don't do this up down up down up down thing. They they are encapsulated which costs you a little bit But we used to measure this like on a on a 10 gigabit network, which we thought was fast in 2015 On a 10 gigabit network, we'd measure 8 gigabits of throughput, you know, so it wasn't that bad And it's doing encapsulation, but it's it's kernel to kernel and it's delivered To its tonight destination pretty fast So the person that did this was mr. Rag dr. Rag, I should say almost everybody that worked on we've net has a PHP PhD except me Sorry Yeah, so Like I say, I like to put up the smiling faces So that's the the fast data path Fixed our main kind of Obstacle that we had in the in the marketplace, which was that it was kind of slow Let's talk a bit about how we set all these things up So, you know right from the very beginning. We need a bridge. We need vets. We need to step into network namespaces step out again We need to Set up some IP tables rules. We need to do Set up some siskals, you know a bunch of things we need to do So how do we do all that? in a shell script we Borrowed Liberally From this project called pipe work by Jerome Petazzoni Who was a docker at the time I think and This project is a shell script. It turns out. It's actually really Concise to do the kind of IP net and s blah blah blah stuff so So we had our own shell script, which is called weave and It started off 350 lines and it it has these commands like we've launched and we've attached and so on at Peak it got up to two thousand two and a half thousand lines It's not a very nice place to be to maintain a two and a half thousand line shell script So I I sat down and re-implemented a lot of it into go So it's it's currently I mean this is the latest commit. It's at 1600 lines or so. It's The features keep getting bigger on the option. It works in all kinds of different modes and so on so that bloats it but But one thing is when recoding from these things in shell script to go It the code gets like 50 times bigger because you know go is notoriously verbose, but But there we are So what else we do encryption We do that Both ways with the fast data path and the the slow data path We renamed sleeve because slow data path didn't seem like a good a good branding position Corporately You know sleeve is a thing that encapsulates something anyway, so So cunning metaphor here in in user space we use the NACL library NACL sodium chloride salt. Yeah, okay To do our encryption in Kernel space we use the XF RM framework and there's a wonderful Explanation on that link at the bottom at all the minute details of how we do this One interesting tweak We couldn't get this to work at all for months Essentially because the the the open v-switch data path I Doesn't Provide any way to to drive the the packets through the exo-firm frame framework we can't set a policy to say everything on this data path go through here and Eventually the idea how to fix this we stole from Docker We we put all the packets through an IP tables rule which marks them and then set a policy on that mark So we have an IP tables rule whose only function is to glue these two bits of software together that otherwise Inside the kernel that otherwise don't play and that's kind of a Lot of the history of this project has been sort of fighting with things that didn't quite want to do what we wanted them to do That's You know the history is there in the code and some of it I can remember but anyway, so we we We encrypt the packets We we're doing key management up here We did not roll our own crypto and Yeah, some people like that feature. It's it's encrypted on this side It's encrypted when it hits the underlying network. It's it's not encrypted here You know so if you have if you've managed to get on to the machine and you can you can sniff this Veeth then you'll see the plane Traffic there But I always reckon if you're if you've got that much access to be on the machine and sniffing a veeth Then you probably lost the game already so Who knows What else? Oh, yeah, Martinez wrote this Martinez did all the Gluing things together at the XF RM level he now works on Celium at I surveillance which is which is that vendor So that's Martinez Okay, so change tack again We've not is a is a peer-to-peer network with it. I mean that The title of this talk is is like no central point of control and it's a pun on the management style and the technology we We wanted it to just be install and run whether you're running on your laptop or in the cloud or on a hundred hosts or whatever and What most people Did to put together a container network was was they they rely on something like xcd To to be a thing a central consistent store of what's going on to to hold all the container information or the roots or the whatever And we didn't do that So we've net is is completely peer-to-peer you can you can start with one peer and you can start adding more peers They talk to each other via gossip So I've given each one a little flag Each peer has an identity on the network and that that peer can be Present on the network or it can go away, you know You can close your laptop and take it on a plane and open it up again, and it it'll still work on the network So the way we do that is all the shared data structures are implemented as as CRDT's as as a eventual consistency Data structures there. They're specially designed so that we can do that like somebody can be absent for any number of hours and come back again and and the the data Reconciles it all fits together That is incredibly hard work so Anyway, it has this property that you don't have to set up at CD before you get started, but But it is very hard work We do this for IP address management as well we do it for several things but but one of them so we basically take an IP address space and map it on to a to a ring Data like a distributed hash table type ring the idea and then spread that across the network and And gossip updates to that ring. That's how we why we do IP address management Yeah Okay, I wanted to talk about the community a little bit So as I yeah, I think I have a chart here, so this is the When it says installs, this is what we get account from Docker of the Docker pool operations So it's running well over a million Yeah, so I mean it was up at two million. This is the one year the last year two million a week Down to about one and a half million a week We we see the software fire up a lot we don't as an open source project We don't have a very good idea of who's using it Right. We you know, we don't People people write in when they have a problem sometimes But they generally don't tell us just that they're using it and they're happy with it So this is one of the few bits of evidence we have the thing gets gets fired up in some sense a Million or two times a week Compared to that we get a we get very few PRs We we get lots of people coming along and saying things like, you know This is just one I picked on because it came up recently And this is over a period of a year and a half people complaining about a setting and saying why isn't that why don't you change the default? It's one line send a PR People don't know how maybe yeah So most of the work has been done by by people being paid by we've works This is the the github contributors list Fun fun statistic after being the lead on this project for five years on the second highest contributor Matthias Radistock Who is also ex-rabbit MQ he was co-founder of we've works He's still the number one contributor, but all yeah all these people work for we've works This this is the Mike Bryant is the biggest contributor who doesn't work for we've works Oh, we don't have a kind of long tail of people who did manage to come up with one or two PRs Which is great, and I I would like to encourage that, but It is a it is a little bit Dispyriting when People just want to kind of complain About about the software and demand that it does something else Okay, Kubernetes this is what you were promised right this is the theme of this day So So we've now is quite popular With with Kubernetes. I thought I just kind of run through what is what does that mean exactly and and what is it doing there? And how does it work? So Kubernetes Doesn't just talk about containers talks about pods a pod is a collection of containers on the same machine and So so in in Kubernetes world conceptually the the blue blobs are pods and but the same same stuff is going on They're talking to each other and and Kubernetes has a very small set of rules One of which is that any pod can talk to any other pod without going through NAT And Funnily enough The rules the the sort of model the networking model of Kubernetes matches very well to Google's network So I don't know if we'll ever figure out how that happened But you know What if you run it to GK, you know Google's commercial Kubernetes then this thing with the bridges They just they just have roots like IP root layer three roots from machine to machine So they have the same thing with the bridges, but they don't have they don't have anything else other than the Google network to Transmit packets between machines They just use Linux routing and let the Let the underlying network deliver the packets to a bridge at the other side and that just works if you Google It pretty much doesn't just just work anywhere else So there is a need for for something to take that place and and we've net is one of the things that people sometimes choose to take that place So back around the time this was getting popular which is about four years ago now The project rocket which came out of CoroS, which was kind of a competitor to Docker They had They had this very simple model for network interfaces where they would exec a process that Would add a network interface So that became CNI essentially Essentially some people including weave workers got in a room and and said yeah, let that should work and And it got it got named and it got turned into a project And I am I'm a maintainer of the CNI project But it the CNI is supposed to be really really thin I just thought I walk walk through what exactly that is So CNI is is not coupled to kubernetes it is like I said came from rocket It's completely independent of network and what we call a runtime So kubernetes is a is in the place of a thing we call the runtime and CNI speak And physically it's the the bit of kubernetes called the kubelet, which is the bit that runs on on each node So the kubelet calls a CNI plug-in When it calls it right now the interface is is it exact it execs a process in the in the host host namespace not in a container It supplies a JSON Config which lists a few things out like maybe which subnet you're supposed to be using something like that the Plug-in then talks I mean conceptually Conceptually you've got a network some you know somebody showed up with a network you bought one from Juniper or you installed weave net or you're using psyllium or whatever Somebody's got a network. So the job of the plug-in is just to be that little bit of glue in between to interpret this JSON spec and to cause the network to attach itself to a container So that's That's the idea of the CNI project And I think it's it's It's worked fairly well in in its goal of being agnostic and kind of staying out the way of people and I do quite often hear complaints that the CNI doesn't do this and CNI doesn't do that and The unfortunate news is it's it's never going to do those things because it's trying to be the thinnest possible Layer that could work for everybody This is Jason if you want to say extra things in Jason just add them just like party on just add fields Okay, that's CNI How do we get weave net installed? So I just mentioned The plug-in runs in the host as a process on the host And and everything we're talking about is is containers You know which are isolated So we get that we get round that by devious trickery we mount a directory off the host and Start when we've net starts up it copies the file into the host directory. So now it's in the host I As far as I know I invented this trick, but everyone does this now. So maybe I copied it from someone else Tell me at the end if it was your idea Yeah, so so kubernetes has this concept of a demon set Which basically means run the same thing on one on every node? And restart it if it dies that kind of thing So that that's how we fire up. We just we arrange for someone to ask for that demon set That that fires up a copy of our software on every node we do this trick with Copying a file on to the host And now we're away. So, you know kubernetes now gonna gonna call the plug-in plug-ins gonna call back up into the demon and That's how that all works I observed about Not having any kind of central consistent idea of what's going on and and of course in In kubernetes you have exactly that you have a the central what's called the API server does know everything that's going on in a kubernetes cluster And so a few times we thought about about abandoning the eventually consistent stuff and just rely on On what kubernetes is telling us which is what everyone else does And never quite got around to doing that Anyway, it's an idea if you want to submit a PR that'd be great We do we implement kubernetes Network policy which which was mentioned if you were in a couple of the previous talks So like saying who's allowed to talk to who? We do that by relying on what kubernetes tells us because you know, it's the only thing that knows all the labels on different things And and somewhat excitingly the the network is implemented at layer two and the network policy is implemented at layer three And they they essentially have no connection between them. We just run them as two separate processes in the in the same pod Anywho is that all I wanted to say I just skip over that. Yeah, that's that's Pretty much what I wanted to say. Does anyone have any questions? Yes Okay, so well, that's not a question. That's an observation, but the the observation was the I Okay, do we have any plans to support IPv6? So we've met has has no support for IPv6 in two ways It doesn't support IPv6 inside the overlay and it doesn't support IPv6 as a target on the underlying network So which of those two did you want? You wanted both of them? And may may I ask a question? I mean the whole point of Overlay networks generally is that you have some problem that stops you just rooting across From one container to another and that problem is very often an addressing problem in IPv4 So do you know what what problem in IPv6 you're solving by having a Like why can't you just root between the two containers? Okay, so you said your answer your point was that you need some pods Now my suggestion is that all pods can have globally reachable IPv6 addresses So you so you don't need anything else you don't need me to write any code because IPv6 will solve your problem All right, I think we have to take that that offline Yeah, I mean, you know bottom bottom line Nobody did why doesn't it support IPv6 because nobody did the work It's an open source project We've worked as a company found something much more exciting to do which is called gate ops And you should all buy that We never managed to Monetize weave net we never made an enterprise version and we never We never found anyone that was for instance willing to pay us enough money to do an IPv6 implementation Now thank you for the question any more One over here question is what was the contract race condition? So I should put Martinez's smiling face up Oh, I'm pressing on the wrong button to put Martinez up So in particular that that link at the bottom is not the right one to look for I Should have changed that sorry Okay, the I'm just trying to see if I can give you a short explanation of this basically it shows up when doing DNS requests Particularly on Kubernetes particularly using the muzzle C library And what happens is it it does At two requests at exactly the same time for the a record and the for a record Notwithstanding the fact that we don't support IPv6 the the The two DNS requests go out from the same source address to the same destination address so same source port same destination port and They hit a race condition in contract and one of them gets dropped. No, it's fixed in the Linux kernel Yeah, I like I say we spent most of our time not not on our own software we spent most of our time fighting other people's software including Linux and and And in some cases fixing it so that yeah, so Martinez wrote two patches He found three three race conditions. He wrote two patches to fix two of the race conditions So this is a this is a you can Google like Why why do I see a mysterious five second delay in my kubernetes? system So this is not the only reason why people see mysterious five second delays But it's it's certainly a very popular one The the the nature of the the requests They are from the same address source address same destination address same source port same destination port and Contract does not know how to deal with that Yeah, well, I'm times up. Thank you


Um, it sucks, but, um, we don't really have time to dwell on it, so we just got to get back on the horse tomorrow and try and get a W. You mentioned the fact that you do not have time to dwell on it. I believe 15 games left this season. The wild card race obviously very tight. Can you just describe the weight that this team is feeling right now? Um, I wouldn't say weight. I think we're always trying to win every game. So, um, we're just going to keep going out there and trying, trying to, uh, trying to win. Another question for Jordan. Pete, how they are? Go ahead. Did you know you were very high and strike out so that game, and how did you feel six that? Um, yeah, kind of recognize I was punching a lot of tickets last tonight. I felt good in the six. Really? Just my foot kept slipping. slip in, mound was in pretty horrendous shape and do a 2-0 competitive heater to Mount Castle and he did probably exactly what he was supposed to with it. I just didn't want to fall behind anymore and just wanted to try and get back into count. But I kind of just kind of lost my groove. Go next Eric Bowlin. Jordan, I played a couple of wild pitches in the ninth inning. As a pitcher, can you describe the challenge of keeping your hand and the ball dry on a night like tonight? It's tough.  I think he had pitched in in previous two or you're wiping your pants, wiping your hat, wiping your jersey and everything's wet. It's really kind of hard to kind of it's just it's hard to grit the ball.  Go next to Paige Leicke with MLB.com. Hi, Jordan. What exactly were you feeling with your Arsenal tonight? I know you liked pretty heavily on your off-season pitches and more heavily on your foreseamer. What was what was your feel with your command? Really didn't try and overthrow anything tonight. I was pretty happy really just with the smoothness. I kind of found a good tempo and tunneled my pitchers pretty well today. I set up my heater up and in and kind of kept them away from looking out looking out over at my curve ball and change up and then used my best pitches to put them away. So I'm pretty happy with how I threw the ball today.


It's like two thirty a little past two thirty Richie and Ron already left though Scorpio out here solo trying to find me. What's his next pray? German TNT out here man Congratulations, I guess we made it Hey, you can tell I'm already late man. They got the Henny right now, dog Let me any that's a cigarette. You hear me? This is Bobby trying to see what's good without you


I hope you all enjoyed your lunch and I'm happy to introduce Nathan Wells talking about Automata Flames. Thanks. So now that we're officially started, I can tell you that the program describes this title for the talk. But since we're all here, I can tell you the real title of the talk, which is slightly different. Fun with sharp edges, corrosives, and fire, which is to say that when you deal with Linux and free software in the garage, you're talking about where you physically have to smack things around and wedge them into place, which is not always the case for free software people. And it's different because you literally have to worry about flying rocks in a car and salt water and solvents and things, which is not a normal operating procedure for a Linux box, at least not a Linux box that I've ever used. So like a brief background stuff. I'm specifically talking about DIY car hacking because there's less of that. What that means though is that I'm an amateur, I don't actually code in any of the projects that I mentioned. My day job is I write for LWN.net, which is a Linux news site, which means that I write about you people, and it means that if you stop doing interesting things, I will starve to death. So please don't do that. Also sort of background-wise, I'm from the US, I'm from Texas, which is a place much like New Zealand, except for the being an island and not being completely flat aspects of it, but it is a place where people drive everywhere. And so if I was giving a talk in London or Gothenburg where people take public transit, there might not be the same interest, but I think at some level, it's all related because vehicles of any kind need software these days. This is a side project of mine. It's my second LCA the last time I was here. I was in Canberra, not here, but I talked about open source fonts, which is my other side project. And open source fonts are doing really well today. DIY Automotive Linux. It's a little bit more uphill, but that's what makes it worth talking about. I'm skipping to the end just in case you want to take a nap. Here's what we're going to discuss. Automotive Linux is kind of a broad term, so I'll define a few things and if you get bored during this part, just wave and nod off. Then I'm going to talk about what I, my own personal build, which I did in my own car over the past year or so. And I'm going to tell you you can do the same thing. I'm going to tell you there's parts of it that won't work, but you should still do it. And my advice these days is to maybe build a dual boot system so that you can run Tizen IVI, which is sort of the only automotive-centric Linux distribution and then also run something else that you're more comfortable with that maybe you can play around with more easily and have software packaged for more easily, that sort of thing. I'm also going to do a little call to arms at the end because the application level is sort of where the big funded projects with the car makers have sort of stopped working. I think there's some movement where they're going to maybe do some more development there, but I really think we're at the point where the community has to be writing software or else the car makers are happy to hand that job off to the proprietary software people who are eager to be in another app store on another kind of screen. So that's something you should worry about and make life better for everyone by helping out. Okay, so generally speaking, automotive software, two separate domains. They don't meet one of them we call either ECU or ECM and the other is IVI. Specifically, ECM, they're the electronic control modules, little microcontrollers that handle all the things that make the vehicle do the vehicle-ing stuff like keeping the engine timing correctly and reading sensors from the engine and the cooling system, making the brakes work, the ABS correctly, that sort of thing. IVI is infotainment, which is a horrible, horrible word, but it just means the stuff that interacts with the driver and with the passengers. And if there are passengers in the back seat who are children, they maybe have different requirements than the driver does, as far as the driver should not be playing any kind of games, whereas the people in the back seat don't really need to be connecting on the diagnostic port and things. So yeah, to sum that up, it's either the vehicle or it's the driver to look at it another way. The electronic control stuff is the dangerous stuff that you should worry about and the IVI stuff is the non-dangerous stuff that you should also worry about. It's tempting to say that the electronic control module stuff is the stuff that will kill you and the IVI is the stuff that will distract you. That's a little too simple because these things are separate and they don't interact except that they interact with each other all the time and they overlap all the time. For example, your curb sensor that tells you you're about to run over a kitten has to chime in with an alert that overpowers your music just playing on your IVI system even if you're getting to the really good part of the song. So that means the software has to respond to events in these two separate domains actually crossover. Luckily, the people who worry most about that are car makers and so the DIY person, you get a little bit more flexibility as far as it's your own fault if you mess up and hit the kitten. As far as the ECM side of Automotive Linux, we can rush through that really quickly. Automotive Grade Linux, which is a workgroup at the Linux Foundation, there's some interest there I think in moving some of the microprocessor things onto Linux boxes, embedded Linux systems and be containerizing them just because more and more complex cars have more and more of these microprocessors doing traction and stability and things. So it's easier for them, but right now there's not a lot of that that you can do yourself. There are some people who do their own electronic fuel injection systems. I would see the DIYEFI.org site for that. The thing is that's very engine specific, not a lot of reasonable code unless you're talking about a really common engine like there's a Mazda engine that there's a lot of people doing DIY stuff with. But if you want to follow up on that, I suggest that you have already thought about it. So if you weren't planning to replace the fuel injection controller in your car before you came in today, you don't leave here thinking that's really something I need to do because that's a different kind of domain expertise and you'll know if you're the person who needs to be doing that, I think. As far as the landscape of Automotive Linux stuff, these are the main groups. I've talked about this little project of mine like three times and the first time I spent a lot of the hour talking about these different projects and how they relate to each other. I've kind of condensed that down to like a minute and a half now. But feel free to chime in with questions if some of this doesn't make sense. Genevieve is probably the most widely known of these efforts and that's a car industry consortium basically started by BMW and friends of BMW like suppliers that sell parts to BMW. If like 10 years ago BMW contracted out some model of car like the next 10 years the IVI system was going to be built by this supplier and what they got back was made with a month of this to Linux. And then two years later they had another car that they were updating and putting the IVI system in. They contracted out to a supplier that it should be the same as the first one and they got back something based on Windows Embedded. And then the third time they did that they got something based on QNX and they're like you know what we need to stop doing it this way and just establish a standard that all our suppliers can stick to. Genevieve is that their main product is the specification and the compliance tests but along the way they haven't been writing pieces of software that they need for the functionality that they want. You can tell by the name they really just care about the IVI side. Automotive Grade Linux or AGL is a working group which technically means it's just people who are interested in the topic. It was sort of started by Toyota I believe and like I said they're a little bit more nebulous. They're producing some code. They're a lot newer than Genevieve is. The other interesting overlap there is that Tizen IVI which is part of the Tizen project which is the Intel Samsung consumer electronics Linux thing. Automotive Grade Linux is using Tizen IVI to do all of its stuff. So they're working on some applications and things integrating some donations of code by their members and they're doing that on Tizen. You may have heard of Android Auto which is something that was announced at CES Consumer Electronics Show like last year and it's still kind of not clear if what that really is is an Android program that Google is trying to rope in car makers so they don't, you know, diverge too far from Android or if they're doing something of their own that's different. At this point you don't really have to care. There's only a few people Renault or Renault if you will is using Android and its IVI units and I think maybe Google just wants to make sure that they don't get too off base and cause them incompatibilities. The last thing I've listed there is called OpenXC which is a Ford only thing that is... I'm not sure how you describe it. They want to have an API for people to write their own applications and they've developed this specification for it but it requires a specialty hardware dongle that you plug into the diagnostic port on your car and it produces a JSON output. JSON in particular, I don't know. It's a little harder to tie into everything else. Really what it is though is it's a vendor specific program. The other car makers are rolling out their own vendor specific programs as well. OpenXC at least I care about because I have a Ford car and they are also pretty good in the sense that they publish a lot of stuff. You don't have to sign up NDA to see the OpenXC message codes for your Ford cars and things. So that's nice. I think I just actually covered all of that. This is what you actually get from these projects. There's some demos from AGL. There's an actual software from Geneva. Tizen sort of bundles it up. OpenXC, there's two flavors of that. One is Python libraries and one is Android stuff. Let's see. Moving on to my own build experience. Maybe clear, restaying is not the name of anything. That's just the title of the slide. I started planning in 2013, like in the end of the year when I discovered I was still a growing boy at the age of 38 and I'd outgrown the Miata that I'd been driving and didn't like trying to climb into it all the time. So I needed something bigger and I decided to really get something I can test all these automotive Linux things I'm hearing so much about in. And what I bought was this car, which that's the day that I bought it. That's like 20 minutes after I got home and it's not looked that clean for a moment ever since. It's a 2005 Mustang GT. If you want to sound like a car person, you'd say it's a 4.6 liter engine, 300 horsepower or five speed, other things that I'm not as in tune with there. The nice thing is it's new enough to use the standard protocols and things that modern cars do, even though it's nearly 10 years old. The actual build. Though I'm not a very good maker, I don't have a very good build log on my blog or anything, but the general look out, what I put into the computer side is a micro-ATX motherboard with an Intel processor on it. One SSD, which is pretty resistant so far to the bumps in the road. It has integrated GPU, which you kind of need and I'll get to why that is. It also comes with onboard Wi-Fi and Bluetooth, which is kind of problematic because integrated Wi-Fi and Bluetooth does not always have drivers in the kernel. We'll get to that. Everything else is USB, meaning USB sound card, USB GPS receiver. Let's see, what am I leaving out there? The keyboard is attached over USB, so that means you have a lot of cables to manage, which is a recurring theme. In other words, to sum up that previous slide, this is as PC-like as I could create the thing, which sounded nice at the beginning because what I wanted to do was to install Tizen IVI, and I basically only messed with PCs. I haven't really done embedded things or ARM stuff. The Tizen IVI builds or releases raw images that you're just supposed to copy onto the disk and install, and they make exactly two of those. There's one for a real specific ARM system on ship and there's one for an X86 reference unit. Not having done any ARM stuff, I just felt like I should stick with the PC-like X86 side on that. This is another misleading shot because that is sort of the basics of the system installed. You can see the USB sound card. There's the red bit. There's also a lot of cables not connected there, so it looks cleaner than it normally would. In fact, most of the time it looks more like this, which is to say it comes in and out of the car a lot and I constantly have little scrapes over my hands from putting it in and out of the car. I should learn to get a bigger car or a smaller computer, I think. But I want to give a general overview of the approach they used. This is an image from somebody's... what's the name of it? It's like the SketchUp library online, where you can put your SketchUp models of things together. Someone did a model of that generation in Mustang, so I'm going to use that as the base for my diagram. This is sort of how everything is wired together. The battery is the red block there. I made the mistake early on in deciding I should put the PC stuff itself in the back where I can get to it through the trunk and from behind the seats, which means I have to run power to it. And then I also have to put the screen in front, so I have to connect the screen back to very close to where the battery is. And then another thing I didn't necessarily realize initially was that when I was putting the screen in where the factory radio was, I meant I had to get an amplifier from somewhere else. So that's the big purple block there, and then I had to run the wires from that back to the harness for the radio. So you spend a lot of time buying wires and buying that wire loom plastic stuff that you punch it into. That's something to be aware of. You also see there's a green line running from close to the battery back to the PC block, which is the gray block there. That's the ignition sense wire. This is something that exists in all cars because of stereo amplifiers, and you need to wait until the car started to turn on, or else there's a big pop from the audio system. So that's what the power supply of a computer that you can put in your car needs. It needs to know when the ignition is turned on, then it powers up the motherboard. It consumes a trickle of power otherwise, but it's controlling start-ups so that you don't just drain the battery while it's sitting there. There's also a yellow wire, which is the 12-volt wire from the ATX power supply to the monitor. That's because the battery power is dominantly 12 volts, but it's really anything from 9 volts to 13 volts. If you get it off of the ATX power supply instead, yet it's another pair of wires to string, but at least it's clean power. There's something else. I guess the orange line there is probably just all the other cables and things like the HDMI cable that connects to the video. There's a lot of places to route cable in a car this size, which is certainly handy. The other thing, I think, from that whole wiring diagram, I'd say the notion of being PC-like doesn't particularly last very long. The difficult parts of the build process are mostly on the car side, which is to say everything before the OS starts booting up, because managing the power and making sure that it comes on at the right time is pretty car-specific, and there aren't a lot of tools for doing that on PCs. After the OS boots, it's pretty simple to get around, at least in Tizen IVI. It's pretty similar to a desktop Linux system, which is actually what the project wants. The other thing to note is that when I showed you that wiring diagram and said I made a mistake of trying to put the PC where I could get to it, that's because the system planning stuff has trade-offs that start at step zero, and you have to make several of them at once, so you can either put the PC in the dash where the radio would go, and it's closer and you have less wires to worry about, but then it's harder to get it back out if you have to change something or make a modification. So you do one or the other, and whichever choice you make, you're kind of locked into that unless you're willing to start over. That can be a source of frustration, so go into that from the beginning. You don't get to relocate the battery of your car most of the time, so that determines some things. Yeah, so that's the, again, the diagram illustrating that. Yeah, I just said that. Power is non-negotiable, and again, this is the radio pulled out of the dash on the car, and you can see there's a wiring harness in there that you'll get a closer look at later on. This is the screen I have stuffed into the dash at the moment. You'll note it barely fits in there, and in fact it's conveniently almost exactly the size of the double radio slot, which is, I think, coincidence, but it has sort of created a cottage industry for the two people that make monitors that size. So, okay, I mentioned the radio and the amp thing. I had to buy an amp. I got the cheapest one I could find. Heavy, no matter what you get, that is also another set of power you have to wire, and more cables to run, and the cables are pretty thick and pretty expensive. They're, like, that thick gray stuff there is something like five dollars a foot. It's really worth it, though, because it's really thick, but you can, like, tie it in knots. It's very, very flexible. It's all silicone. I also discovered that pulling the radio out, I had to, if I wanted to listen to FM radio somewhere, I had to find a tuner for that, and I eventually just went with a cheapo RTL tuner, which I sort of managed to get working. It's sort of a, one of the sob story aspects of this is that the big automotive projects don't really care about FM radio anymore. They care about FM, sorry, about MP3s and about syncing with phones and streaming and stuff like that. A few other details worth noting, the case that I chose was this thing. It's made by a company called Minibox. It's a mini ITX case. It has the peculiar problem of expecting a DC, like an external DC power connector, like you would use on a laptop, which I didn't have. I have something like this, which is an ATX style power supply. It doesn't fit in here, so it has to go in a separate box. The thing is, though, that I could have chosen maybe another ITX case. There's really only one company that makes these, though. There's sort of two options. There's the one that takes battery power and produces standard ATX connector rails on the other side, and then there's the thing that is more like a UPS, and the UPS just has a little reserve of power and then you plug in another existing computer's power supply to that. Again, that's why I have to run multiple wires back and forth is because you don't have a lot of options on the power supply side. You do have a lot to worry about, though. This is right off of the battery. That extra thick wire is being split into two wires, one that goes to the amp and one that goes to the computer. Both of those have to have these big expensive fuses on them so that when something melts down it doesn't bring the car to a screeching halt. Here's a shot of pulling the door frame, door stills off, so you can run all those wires. That's at the early stages of the project by the time I decided I had to route some of them somewhere else. There were just too many to fit into the gap between the plastic molding and where the plastic clips fit on the molding. This is an earlier image of when I was hooking everything up. The black box there is like a radio shack plastic case that I have the ATX power supply thing, and it doesn't work very well in the sense that I had to like carve with a dremel to get the connector to fit into it. I actually had to cut a hole in the ATX case in order to get that ATX power connector into the case because there is literally no room in there to connect it otherwise. Maybe that's not the best possible option but again when there's only one company that makes the power supply that doesn't have a lot of choices. That's the shot of the replacement case that I put it in. It's metal, this ATX style case and when the top is on there's a fan so it'll keep a little cooler. I haven't had overheating troubles but I haven't driven it for long journeys in Texas in August yet so that could still come up. So back to the things you learn. The road is unforgiving which means that it's going to get around a lot so I had things bounce loose the first few times.  That's an MDF board sort of base plane for everything and then there's like two threaded T-nuts that back into each other and a little brass thing on the outside of them that keep them lined up and a lot of time was spent trying to keep bolts and things from shaking loose and snapping off. It also doesn't look great. I eventually coated it with the stuff that they put on PC speaker boxes which makes it look nicer even if it still has clearly got Velcro straps sticking out of it which are homemade. As far as the other big pieces there I don't think you have to be too picky about GPS these days and on the other hand if you want to connect the diagnostic port on your car you don't have a lot of choices. That's the second item there OBD link MX is the diagnostic tool. I've got a picture of it here. It plugs into the port that they use at the garage to tell it what's wrong with your car and it listens to all the codes. Some of those come over something called CAN bus which is the controller area network bus that has been around for ages but any particular car maker is also going to have other kinds of codes. Some of the people that make this know what the other codes are and can decode those as well. That's helpful. Another thing to consider there is I eventually found a capacitive touch screen to fit in the dash. Most of the ones you see in that form factor are resistive which means there's a film that makes it a little harder to see and a little harder to punch on there. Again it's because there's only a couple of makers Xenark and Liliput that make that size. You don't have a lot of choices. There's no resolution. They tend to be 800x480. I'm not sure about this but I think that that size was originally built for video cameras for shooting digital video in the field. It's sort of the monitor you attach to a movie camera. They are aware that there's a market for this automotive stuff now though so they seem to be improving on their offerings. Last but not least when it comes to I.O. situation is going to be complicated.  It's pretty small so at least you can tuck it in between the seat and the car but you kind of have to do that or do USB and then have another cord hanging around because you generally, at least when you're building you need to have keyboard access when you're booting up and you're at the BIOS screen. You have a lot of RF interference in a car at times so you may not get great performance out of that. Maybe you'd prefer to find a good USB keyboard and use that instead. Anyway, back to this the OBD link model that I have is Bluetooth connected it's, I think $89 or $99. They also make a USB one which is a little cheaper but again then you have a cable to run from wherever the diagnostic port is to wherever you're connecting it so it's one more cable to run to trade off. I know that's a on the giant screen it's a little hard to tell so just to give you a sense of scale I took a picture of the OBD link next to a giant novelty quarter that I got off at eBay and that's the novelty quarter next to the regular quarter so you're on your own to extrapolate back to the size of it. For other reference the thing on the left in this image is the OpenXC connector for the Ford specific JSON formatted stuff. It's bigger because an Arduino type thing on the inside that you have to flash with firmware specific to your car you can hook both of those up the CAN bus is a bus so as long as you don't put too long of a cable on it you can get a wide splitter for the diagnostic port and hook both of those up if you want to use both of them. This is the keyboard I mentioned earlier that's literally a thousand times bigger than it is in real life it's smaller than my hand together. I don't recommend it but I don't have a good option to recommend it in its place. The good news is that eventually you can get all of this side of the build process done and then you start to worry about the software side and that is also it's not vehicle specific it's driver specific so how and when you drive determines what you need and it's not going to be the same for everybody personally I work from home and I live in a place where I've lived for 14-15 years so I know my way around and so I don't depend on navigation on a daily basis like some people would if you work where you have to go to customers, locations and give support or if you're a real estate agent you may care a lot more about navigation and a lot less about diagnostic codes or MP3 support. A little more background on the actual build stuff Early 2014 what I first got running on the car was the Tizen 2.0 series which they have since replaced with Tizen IVI 3.0 which makes some significant changes most significantly they replaced the Web runtime which had been built on top of WebKit with this new project called Crosswalk which is derived from Blink and does more stuff but it means they sort of cut loose several applications and things that were available in the IVI 2.0 series one of those is Navit which is the open source routing software so currently 3.0 doesn't have a good navigation engine which is another reason why I suggested you would consider dual booting between Tizen IVI and something else 3.0 also doesn't have a pure browser component yet or at least it doesn't have one based on Crosswalk the Web runtime it does include the links browser which can be fun for a lot of things it's not really helpful with Google Maps or OpenStreetMap though let's see what we're seeing here is me booting up Tizen IVI on the bench how long was that anybody know five, six seconds something like that the people who work on this full time have hard targets it has to boot up in one second or produce camera output in a second and a half something like that you're not really going to get that with the home built system because you're at the mercy of motherboard BIOS and things like that also the ATX power supply that you have only one manufacturer to choose from the way it works is when it sees the ignition since there's a five second delay and then it provides power to the motherboard so if there's a built in five second delay you have no hope of getting everything up and running in under two seconds, three seconds, whatever it is so you have to live with that limitation I think that it's probably not as big of a practical problem as it is if you're a car maker wanting to advertise how fast it works what I actually do with this, mostly I have the audio player working and I log some data I haven't really messed with like, souping up the car in general tuning in or anything but I think I might do that at some point and it would be interesting to compare the data that I can log now with what I log then and I've been testing out other things hoping to find ways to get radio navigation projects working there's still a lot for me to do, the next big thing for me to tackle is sensors the display that I have has a built in light sensor but it's only affecting the brightness of the screen it's a monitor control mode I can't like sense the ambient light in the car and then adjust like the theme that would be something you'd need to do in software have access to light sensor data maybe inertial measurement unit digital compass data that kind of thing so that's that's still on my list of to-dos but it comes to the stuff that I recommend for other people to try if you find this interesting you start with figuring out what you want out of the car and then you're going to proceed to decide whether you boot right into Tizen IVI or you boot into something else that you can run nav it on for example it's a personal decision I'm advising that you dual-boot something the goal there is that I really would personally find it more helpful if a lot of other people were testing that Tizen IVI stuff there's about a dozen people I know of who have attempted to put it into their car and that's not very many as far as getting it to work so it would be really great if more people tested it and reported what works and what doesn't work but you should also certainly have something that you can use for whatever your key use cases are a few planning notes like I mentioned the Tizen IVI builds are released for x86 or for this particular arm board which means you probably unless you have one of those you should go with the x86 build your life is just going to be easier in a number of ways there that said the x86 support is pretty broad initially they had a very specific sort of vetted device manufacturer produced an IVI thing that you couldn't even buy unless you were a company you were ordering hundreds of them this past July officially added support for those Intel Nux which are the little usually called them a net top I don't know what the current name is but the low power devices officially this particular model which is pretty specific is the one that they support but I didn't have any trouble bringing up Tizen IVI on a generic mini ITX board with a chip that I bought off of someone on Amazon so don't be afraid to just find some Intel hardware and see if you can get it working on the other hand you probably do need to stick with Intel GPUs because getting the display stuff working on Nvidia is definitely going to be drastically harder and probably not worth your time most of the work on this is done in Intel and they have good quality open source drivers which you can't say about just any GPU manufacturer you pick again like I mentioned you don't have a lot of choices for the power supply and for the IO stuff this is the name of the company that makes those ATX power supplies I mentioned and they also make something that's a lot smaller and it's really more like a UPS that powers up an external power supply two companies that do touch screen displays that you use from all of which are sort of the same size and then weird keyboards the final thing I've noted though is that you don't really need to worry about CAN bus it's sort of it's known as the automotive bus that they use for diagnostic messages and things but it's not it's not the kind of thing that you're going to do new stuff with it's a legacy support issue and if you get one of those adapters like I showed that interprets the messages and sends them to the PC over Bluetooth you're kind of done I searched long and hard trying to find other devices that you could add that would speak CAN bus and I only found two one of them is a really high-end radar detector which you can control over CAN bus but you can also control over Ethernet which is a lot better option and then you can also get like a car alarm system which you can control over CAN bus but again you have other options there too so don't necessarily worry about doing CAN bus stuff another note your interior trim you have to pull that up put it down a lot it's not technically a PC building issue this is stuff that's not designed to be removed and put back in a lot it's designed to be removed when there's a problem so if you're running cables and putting it back together a lot little things are likely to break off as far as actually installing Tizen the official procedure is these raw images that you copy to disk and it overwrites everything if that works for you great not really necessary to do it that way what I eventually did because the motherboard that I had gets really confused when I try and use the GPT partition tables and eventually I realize you can just install something else and then just leave a partition just copy the Tizen stuff from the raw disk image onto it set up the boot loader on your main Linux OS to launch into the Tizen partition and it'll run pretty much just fine and then I also recommend in this model though one for your other OS one for Tizen, several one for data because the Tizen stuff the new release will also be right over the whole disk and you don't want to lose your data and you want it to be accessible to both operating systems pick something you're comfortable with for your other OS if you're not sure what that should be there are some valid reasons to say Ubuntu that touch support actually works which it doesn't work as well on some other distros probably more specifically Genevieve and some of the other projects build and test their stuff on Ubuntu and Tizen IVI SDK is released for Ubuntu first it's not a huge difference whatever you're most comfortable working in is going to be fine so like I said it's not hard to just take the boot loader entry Tizen IVI image put it into your other bootloader you do need to make sure you watch where you point to with UUIDs which will not necessarily be the same I recommend you remove the quiet kernel boot option because you want to see what's going wrong and the releases from Tizen IVI are designed to be as fast as possible I had trouble with TSC as the clock source which was the default in Tizen maybe you won't have trouble with my hardware but you can switch, use the different clock source parameter and maybe have more joy so my example grub menu entry sets the root it's exactly the Linux and rootfs and it already commands that were in the Tizen bootloader as it was delivered on the raw image don't have to change anything there what you get when you boot it up is this there's more there than maybe meets the eye because that's a desktop Linux system and that's sort of what ought to be like the big selling point of Tizen is that you get the standard kernel and I mean standard like all the normal modules why I first hooked up I couldn't find a mouse so I hooked up one of those touch sensitive graphics pen tablets and the Wacom driver loaded so if you want a touch sensitive pen tablet in your car you can do that with Tizen API Erron's Weston you get standard libraries and things like gstreamer and pulsio if you were in a Yachto built system you would have just the modules for the hardware that you needed and you in android are going to have the whole android stack instead so this is easier to develop for you get the actual developer tools get the C and things like that access to python QT which is actually contributed by Digia not by Tizen employees and there are these sample apps that we'll look at you don't get good meta packages like build essential you might have to install make and auto make separately you don't have python packages that you can just pull in with pip so a little more manual stuff on that like I mentioned the GUI browser and navet are gone at the moments also at the moment GTK stuff is not in Tizen although I think I've heard there's some interest in changing that from from people in Intel so maybe that'll be better you have zipper for package management you can just grab the kernel source so if you have an out-of-tree driver which I did for that built-in Wi-Fi card you can build that module and it works you should also probably do some basic system maintenance when you're setting this up the route password in my car is probably still Tizen I need to change that and the system descripts you can dig into it and I had to change the order of some things to get the Bluetooth firmware loading correctly on this built-in card but it's not hard to do you can customize your file system the way it's set up so that your extra partition with your data on it is the way you like it the Tizen media player app which does actually work wants files to be in opt user media which is not a real standard place to put that so you'll have to do a little bit of work to configure that stuff but not real painful you also will probably need to get some development packages to pull things down and the two things under configuration tools are very important because currently you have to configure Wi-Fi and Bluetooth on the command line with Bluetooth control, command control it's standard operating procedure for how to do that but you have to install those packages yourself they're not in the default I found a couple other things that are useful auxiliary programs, M4 API there is a third party program to modify the settings on those ATX power supplies I showed I haven't tried that out but it's interesting and then I had to get this hex to HD HDD thing to convert Bluetooth firmware to the format needed from the Windows driver you don't have to go through that applications this is where it gets a little bit of pressing as far as like right now Tizen Avi 3 there are two separate frameworks Modelo and ICO Modelo is newer, it's also incomplete whoops this is an image of the basic like home screen app there do you notice anything unusual about that compared to the screen that you can buy it's not the right orientation Modelo is portrait only they're intent on keeping it that way I've asked back in August and September I tried hacking through some of the themes to get it to look correctly sideways it didn't come together real well because there's a lot of stuff that just assumes portrait orientation I haven't really said I'm not sure if it's really worth doing some of the apps will launch and you only see the first half and you can scroll up and down on the screen others will get off to one side and it's kind of a mess the other apps you can use this is an older ICO app called Ghost Cluster this reads diagnostic data and presents it like skeuomorphic dials that again it's a little bit off on the size but that's because the resolution of these seven inch screens is slowly changing you can fix some of this by rotating the screen 90 degrees is that worth it? it's okay it's not ideal there's actually more you can do that you can read about on the Automotive Linux Wiki site there on the plus side the big thing that is really nice is Automotive Message Broker which is a multiplexer for vehicle data sensor data, diagnostics, GPS it will display on Ghost Cluster and on the Modelo like heads up app you can also log it which is what I do database, configuration is pretty simple you tell it your Bluetooth or USB device that has your OBD2 sensor on it I've also managed to get some open XC demo things to run and I actually got RTL FM to run for FM radio tuning that doesn't that's not how you're supposed to do that but somebody who's at the open radio comp really needs to write a good SDR radio app for the car smart device link is a thing for connecting your phone in theory that works I don't have a new phone, I have an old Mego phone but again if you want navigation you're going to have to boot into your other operating system and probably use Navit at least for the time being Geneva has several proof of concept things like a browser and a nav system that they're restarting developing on so that could still develop into usable apps there also a few third party things some of which come from books and from demos there's a good Google one that I've kind of got working I think that the API has changed on Google's down there is that kind of a weak application story? Yeah it is and that's sort of why this is something that the community needs to like take a closer look at because the car makers aren't evil but when it comes down to it they're going to ship a car if there's an open source component for what they need or not I would like to see more stuff that is better than demo quality but the community will probably have to write that because the car makers are done when they have something they can sell the frameworks I think are decent they're pretty good they need a lot more testing and so the choices sort of they're starting off on this industry with a free operating system is it going to be good enough if it's all free up to the application level and the stuff you interact with is proprietary I don't think so the other option is that those of us who are writing applications can write baseline apps that are good enough that the car makers don't want to replace them with other things so we don't have to settle for demo quality there I got some examples I could media players and things open street map why do we have to stick to Google Maps there's a lot that's not real insightful on my part I don't think let's see this is just a summary please give it a try if you are interested in this you can dual boot Tizen IVI and something else you get somebody to compile or something you're working on try writing an app tell me what I should do with modell landscape mode a lot of options but it does require that those of you who are in the audience actually give it a try that's all for me thanks so sadly there's not really time for questions maybe one or two if there's burning questions burning questions I'm kind of done kind of done I've kind of done one of these well it's a bit of a work in progress with an Android tablet which I think has advantages and disadvantages compared to compared to the approach that you've taken do you have any have you had a think about that trade-off yourself I haven't attempted it myself which is I feel like most of the trade-offs are going to be real practical there like I don't know how you get power to the tablet you can in some devices including the next sevens you can there's a kind of a kernel hack that will let you supply power to it while it powers also while it also does USB host mode you can even use it up okay so it would be that kind of thing I would think would be the bigger issue once you get it running you're kind of in Android space it depends on how you feel about Android I guess to know if that's better or worse you get some really useful yes there are open source Android apps that do OBD diagnostic reporting and things like that so that's definitely true you can come see me exactly so again thank you very much we have an acknowledgement for you on that here


Hello guys, thank you for coming and today I will be sharing about how I found 19 NPM malicious package and first thing about me, my name is Kwan, I'm a final year's CEG student. My interest lies in computer security, IoT and guitar music as well. In the summer I interned with Veracode and this is the project that I have done with Veracode. There are actually two employees at the back who are from Veracode and yeah, they are hiring. Okay, so malicious package as you can see here, this is an NPM advisory, some of you might be familiar with it. So I found 19 malicious packages and I reported to them. So at the end of the talk, you might discover and you might want to discover and then have the name in the hall of fame in the NPM. So let's start. So the contents I will cover will be static analysis, what is that and train tracking, what is that. Then I move on about how I do my scan for eval function which is kind of a dangerous function in JavaScript and also my scan on malicious packages. So let's start with static analysis. Static analysis method of debugging based on only the code and static analysis can actually be used to detect bugs and vulnerabilities in the program. So an example of a static analysis program is ESLIN, yeah, very familiar. So ESLIN used static analysis to identify rules that you have already mentioned and if you think about it, ESLIN can be used to detect vulnerability as well because there are actually rules in ESLIN which cover security. This one of such example, this allow eval. So eval is a very dangerous function in JavaScript. It basically lets you execute a piece of code and in this example, it will execute object.x which will result in full. So ESLIN is able to detect this method alone but is it able to detect that this is a vulnerability? Well this is not a vulnerability because key is literal x so it's basically calling object.x. That's not a vulnerability but if this key is an user input then that might potentially lead to a vulnerability. So in order to find out whether or not key is a user input, we use 10 analysis. So there are two main concepts in 10 analysis. One is a source, another one is a sync. A source is basically where the data come in and a sync is where the data come out. So in this example, there are five lines, six lines and this is the example of a code that is vulnerable to cross-size scripting. That means you can inject this script and then you will be able to execute the script. So let's go through this example. So a is request.get parameter. It's basically try to retrieve the get parameter full and this is a sync because that is where, this is a source because that is where the attacker or the user can input his input in. So through a bunch of assignments, for example b equals to a, c equals to b, something, d equals to c and then e is equal to something d and then you move on to the sync. So the sync here basically just write down e in the browser or whatever is running this program. So that is a sync because that is where the program produce the output. So the 10 analysis try to evaluate if this e, this variable e is tainted by anything that can be input by the user. So the way that it does is that e is tainted by d because of this method and then it trace back d is tainted by c and so on and so forth and gets back to a. So the input of the user, which is a, would directly affect the output, which is e and that is how the cross-size scripting is enabled in this case. So there are many complications in 10 analysis.  There might be multi-parameters in this case. There are multi-parameters in a big program. There might be different data structures that affect the 10 analysis. In this case is the hash map, which might affect the 10 analysis.  So is the input by the user sanitize anywhere between the sync and the source. Yeah. So that is basically what you need to know about static analysis and 10 analysis. And to do this, I use an algorithm for JavaScript to build a 10 analysis. It's called approximate call graph or you can call ACG. This algorithm has been reviewed by papers to have high precision and it's most importantly is very efficient. So I will explain how the algorithm works. So given a simple piece of code on the left, x equal to a plus b to the a of x, you have a representation on the right side, which is called ASD graph. And for static analysis, people usually use this graph to do the analysis on the source code. So the ACG, basically what it does is that it will traverse this graph and then while it's traversed this graph, it will be the tainted graph. So I will explain. So when it traversed down to this assignment, on the left, you have x on the right side, you have an arithmetic operations which include a and b. That means x is tainted by a or b and b. Also means a and b taints x. And after that, it travels up and then move on to the next statement and so on. And that is how the algorithm basically works. It's basically traversing the graph. And what I did to this algorithm is that I changed it up a bit so that it's more suitable for the application of finding security bugs. So security bugs, especially dangerous functions like evaluate does not frequently occur in the source code. So it will be inefficient if you scan all the libraries and then build the core graph or the tainted graph for every single library. You only want to build the tainted graph for the libraries that have dangerous function. And you only want to build the part of the graph that is related to that parameter, that variable alone. So instead of doing from top down, that means from the program and then traverse down the normal way, I will do it from the bottom up. That means I identify whether or not the dangerous function is there. And then from the dangerous function, I trace back to the top of the tree. So let me explain again. So it's the same piece of code. But now you know that the dangerous function is here, eva, and the variable is a. So what my algorithm do instead of going from program down, it will just trace from a here. So trace up to assignment operation. And then you know that a is tainted by a only. And that's all it's going to do. So basically it will only traverse this part of the graph instead of the whole program. Yes. So in the top view of what the algorithm do, it will first trace all the variables in the nodes. That means how to find this eva, and then trace whatever inside this eva function. It will try to trace back of what is the source of this a. How does it do that? By adding intra procedure edges, adding inter procedure edges, and then go back to tracing variables in the node. Basically it's a loop. And it will stop until all the notes that all the relevant notes are being inspected. I will explain what at intra procedures and inter procedure edges mean. So in intra procedures edges basically the operation that is only happening in that particular procedure yourself. So some of the example is this. So if you have an assignment like a equals to b, that means a is tainted by b or b tints a. And some of the other example includes a equals to b or c. That means b or c or tints a. So as I traverse the tree and then arrive at some node, I will inspect what kind of node that is. And then based on the type of node on these procedures, I will add the relevant edges to the tainted graph. And on the third column is a score. The score here is a score of confidence. That means if the score is zero, I have utmost confidence that the edge is the correct edge. And if the score is higher, then I have less confidence of that edge being correct. Right? So a good example is a equals to b or c. So in where in runtime environment, you will not know that a will take the value of b or c. So when you do static analysis, I would just assign a to be tainted by b and a to be tainted by c at the same time. So this is an overestimation because in runtime, it can be tainted by only one variable. And that's why I increase the score. And the purpose of the score is that I want to keep the analysis as precise as possible so that I can achieve a more efficient code. And so each procedure is quite simple to think about. What's more important, what's more complicated is the intra procedures edges. That means all the edges are added across the program. That means there must be some kind of function cost or something. So in the case of a equals to fb or new fb, that means a is equal to some function call with the parameter of b. There will be edges from a. That means the variable to b is the input to the functions. And similarly, a will be tainted by the return value of f. So you can kind of see this kind of more complicated because you have to draw an edge from the declaration of the function, which is stored somewhere in the code, and then the actual call of the function of a variable somewhere else in the code. Yeah. So these are the only operations that is needed in the algorithm. And after the algorithm is run, then what you have is a tainted tree. So using the same example, the tainted tree will be x tainted by a and b or a, b tends x, y is tainted by a, b is because of this. And then a tainted itself because of this. Okay. So the tainted tree is already obtained. And I would use this tainted tree to find the vulnerabilities. So I want to find all the vulnerabilities related to the eval functions. So using static analysis, I try to find all the occurrences of the eval call functions inside the code using this and identify what is the argument of this eval call function and find the 10 graph of this argument. Yeah. And this will be done in when I traverse the whole ASD tree of the program. After finding the 10 tree that I would take, I would use this 10 tree to apply the filters on it. That means is there anywhere in between the tree that there is a sanitization or is there anywhere inside the tree that one variable has a source of a parameter? I will explain this more. So sanitization filter, the purpose of this is to reduce the false positive of finding vulnerabilities. For example, in this case, we have the eval payload. And this payload is tainted by the variable request. And assume that the request comes from the user. So it's easy to see that this can be a vulnerability, because payload is tainted by the user input. However, in between, there is a function called remove blacklist, which can act as sanitization. That means the payload here will be safe to eval. And those are the cases that I want to find with sanitization filter. The way that I use to filter this is by heuristic. That means any method that contains some kind of keyword, like sanitize, blacklist, whitelist or replace, I just consider it as a sanitization method. Yeah. Source type filter. So in this example, the eval function is here, which is called an R0. And R0 is something that is literal and it's not input by the user. And in this case, I would say that it is not a vulnerability because, well, it's not input by the user. Yeah. So using these two types of filters and putting the tree as an input to these filters, I will able to scan all the results. And these are the results on a set of the top, I think top 1000 or something libraries in NPM that start with J. And the most important thing to take away here is that with no filters, that means with no 10 analysis of all, then there are about 180 libraries that are identified. But using the 10 analysis and all the filters, you can reduce it by three quarters. That means there's only over just over a few 45 libraries left. Yeah. And yeah, using this, I will able to find some exploitation for some of the libraries, which I will introduce here. Okay. So this library, what it simply does is that it converts the data to a table. So it provides a convenient method to just input the data in this field. And then you have a table in HTML. And one of the way that you can, so using the static analysis, I was able to find that there is a way to execute code in this data field. You just escape it without using this. And then you can basically execute any kind of code here. So here for demonstration purpose, I just alert something like malicious. Yeah. Another example is here. So this library, J2N, it converts a string to an object. So again, using static analysis, I was able to construct a payload that will execute this piece of code. So for demonstration purpose, I just num equals to 666 and then console. And then you can see in the execution, yeah, the console was executed. Yep. And that is for the scanning for eval dangerous methods. Next, I want to scan for malicious packages in NPM. And if you don't know, malicious package has been very popular this day. It's one of the easier way for the attackers to run codes in the user website or whatever the user used in the applications. An example is like this library, when it's installed, it will run executable targeting windows and upload information to a remote server. So I want to find all these packages on NPM. There's a pattern that I recognize with these packages. And I would use these patterns in my static analysis scan. So I will list five patterns here. These malicious packages are often obfuscated with Bay 64 so that the people who read the code cannot understand what it really does. It must be sending a request to a server or something because that is what the attackers really want to do. Sometimes it attempts to read sensitive files like a password file, SS shared file, and they usually run script in the post install and the pre-install field in the package.json. So the code that is written in this field, post install and pre-install, will automatically run in your environment when you install the NPM packages. And these packages are often typosquad as well. That means they have like an extra X or MongoDB without the correct... Yeah. So I did a scan using the same method, using the same static analysis, but without the 10 analysis because 10 analysis is not really effective here.  And this malicious code basically send the information of the host name and whatever type of OS that the environment is running the program to a server online. And it's a Chinese server. So yeah. And these are the libraries. And I will go through the ways that these libraries execute the malicious code. So there are four ways. The first way is that they can require the file in the index.js. So when you require the library from another program, then that library will automatically execute the .test malicious code. Another way is to include the code inside the package.json and the fields to include them, the install field. Oops. Install. It's pretty small. But yeah, I just tell you the field is installed or post-installed or pre-installed. These will automatically run the malicious code. Yeah. And yeah. In conclusion, over the course of three months, I found 90 malicious packages and four libraries with a dangerous use of the eva function. And I think that any one of you can do the same thing. And I really think that the security in open source libraries is very important. And you know, for bug bounties programs, you usually need the skills of manual inspections. But for open source libraries, there's no way that you can inspect because there's too many open source libraries. So static code analysis is probably the best way to find the vulnerabilities and keep the open source library safe. Yes. And that's all. Yeah. If you have any questions about the project or about Veracode, you can ask the two guys. Can you raise your hands? Going here is manual, right? Or do you plan to make this? Well, the static analysis is not really manual. I mean, the manual part is to write the code. After the static analysis is run, it will produce kind of all the libraries that might potentially have the vulnerabilities. And then I will look at these libraries and then do a manual inspection because you can never trust it to the 100% that there will be a vulnerability there. Yeah. And most importantly, when you try to submit these things to any kind of advisory, you need to come up with an exploit. If there's no exploit, they are going to ignore your post. I guess what you mean for automation is like you have your github has, for example, a git secure or something which runs across different repositories. Oh, oh, yeah. Oops. Yeah. So there's actually API on NPM. So I use the infrastructure built by Veracode to scan through all the libraries automatically. But if you don't have that access, you can use the API from NPM and then retrieve the library names and then download it to somewhere in S3 or your computer, scan through a library, delete it, and then do that again. Yeah. I just looked through the most popular one. How would you tell that it is a whitelist or a whitelist? Well, I don't need to tell whether or not it's a whitelist or it's a blacklist method. How do you tell that it's a sanitization method? Yeah. So I use heuristic because there's no way that you can static scan a method and then know that the method is a sanitizing method. So I just base on the name of the method. Usually people would not use a user defined method to sanitize the output. They would usually use some kind of sanitization library from NPM to sanitize the input for them. Yeah. I think that you showed, right? How does it work? How would the graph for a recursive function look like? How does the graph for the recursive function look like? Hmm. So I cannot draw it here. But so when you traverse the tree, it just deficits and then you found something. And then while I was transfers the tree, I take note of all the locations that is related to the point down here. So when I found the variable here, I looked through all the references of that variable and then used the same method to walk up the tree till the point that I don't want to walk anymore. Yeah. I mean, if you want to know the details, you can ask me later. We all had to be compromised if you install a malicious package at stand 30, but it didn't import it. Wow. Because of typo and install a malicious package. But like we just use it. We all think we're compromised. But it depends on how the malicious package is designed.  Yeah. And the only way you can reverse is just to reinstall your whole environment. Yeah. But if the attackers is more kind, then you will just execute the malicious code when you call some kind of functions or when you require the library. By remaking your whole environment, do you mean like just NPR install or like you have to recompete the whole thing? Well, the attackers can basically because the NPM package manager, they have the rights of the user who is using that. So basically the attacker can install any backdoor anywhere in the computer. So yeah, with my environment, that means the whole machine. Yeah, unless you put it in SamOp, that's correct. So if you have like any more questions about details and stuff, you can approach me later. Yes. Thank you.


Hi everyone! Thanks for tuning in. This is five reasons for using YouTube for education. This little video is just gonna highlight some reasons and things to think about when you want to use YouTube to enhance your face-to-face courses or your online courses. So the first reason we're looking at is using YouTube you can develop and post mini lectures for your online or face-to-face courses. Now why would you want to do that? One reason is for emphasis. Providing a video in which you're using voice and images can really identify the key points or really connect the key points. You know, providing notes is great in a course, but a video where you're really getting into the material and using your voice and tone is going to communicate well to the students. What are the really important concepts and how do they relate together? Repetition. We have a variety of students that would appreciate getting to hear a lecture again. Some of those students might have hearing issues, some of those students might have English as a second language. It's not their primary language. In some students they'll either miss a class or they just they need to hear it again. And so having a video they can go online and rewatch, pause, rewind. Really useful for them. It also provides visuals and we know that when content is communicated to students, when it plays upon more than one of their senses, their ability to recall and recollect is that much stronger. And then finally it helps to avoid class cancellations. If there is a snow emergency, if there is some kind of problem, you can't make it, you have the ability to have this material and say to the students, okay class is canceled, but here watch these videos and either participate in this discussion or we'll follow up with talking about these videos the next class. Another reason is that you can provide a video introductions to your students. Now this works particularly in an online class but it could also work in a face-to-face class. Often face-to-face classes you don't really have time to share a lot about yourself. So here's an opportunity in which you can humanize yourself. You can say, hey look at me I'm a person and I'm not a robot and I just don't only think about this subject matter but I have these other things. I'm a fully formed human. I'm not just this subject. In a lot of ways particularly for online it shows the person behind the curtain. It animates them. It allows the students to know there's a real person there and that it's not just a machine. You'd be surprised how many students often have to some degree a perception about the lack of humanity for instructors, particularly in online courses.  Students time and again will say that it's the instructor's enthusiasm that makes a subject more interesting and so this is that great opportunity for you to talk about whatever it is you love to teach and really share that with your students. Another great thing you can do with with YouTube is you can create playlists and this is great because you can really curate a variety of materials to enhance your class and again this works for face-to-face or online. So what can you do? You could curate course materials that is the mini lectures that you create or videos that you create or think are useful. You can pull those together and create a playlist that you can share out with students. You can also curate a range of videos as recommended resources. So things that are relevant to the class but you may not be able to get to you can say hey here's this great playlist of videos with lectures from all over you know from people from all over the world. Feel free to check it out or feel free to look at this if subject matter X was of interest. And then finally you can also use it to curate course artifacts that if you encourage your students or if you start to develop projects where they're going out and they're creating videos on YouTube talking about their experience in the course or talking about research that they've done doing video presentations you can also create a playlist so you can pull that all together so students can see one another's work. Another thing you can do using YouTube is create a course trailer really kind of a you know a sneak preview of what your course is about. One of the reasons you would do this is to share with potential students. You sometimes get inquiries from students hey you know what's this course about or can I get more details. Well you can really create a video to say here's what my course is about a two to three minute video could really pull in a student to get a full sense of what your course is especially if it's a course that generally is taught in a lot of sections how do you distinguish your own course from the 15 other course sections of that subject matter. It also allows you to provide an engaging course introduction right when students sign up for a class that really all they have to go on is 500 characters of a very generic description well here's your opportunity to really pitch and provide a you know a sense of how this course is set up what students will learn how they'll learn it a really good clear sense about the course which of course prepares students for when they get into the course they know what to expect and again going back to share your enthusiasm for the topic this is an opportunity for you really to show why this course is great and why you enjoy teaching it and provide that to the potential students or the students that have enrolled in your course. In the the final of these five reasons there's plenty of other reasons but these are just the top five that I came up with is you can track your content usage so this is in this is really good feedback so what you could do is you can see how often your videos have been watched and that's really useful because of course if you have a certain amount of students in the the amount of times it's been watched is above or below that number that tells you what's you know what may or may not be going on you get a certain amount of analytics around the video usage which will be really useful for figuring out did they look at it how long did they look at it you also can find out what the average duration your video has been watched so if you have a 10 minute video and the average duration is you know six minutes then you know there might be a lot of people that didn't watch the full video again it gives you feedback it gives you an opportunity to figure out what was done with the material we don't really get this when we are dealing with books or articles or things that we hand out to the students we don't even know if they've opened the book you know there's ways we can assess this but here we can know well if I've got 10 students and the average you know the average duration was six minutes then they're definitely students who didn't watch this and then finally you can find out other details about how people are interacting with your content so again they might be able you might be able to get feedback from comments or likes or other ways in which people are looking at that content that'll help you think about its usefulness or help you learn about how students are interacting with it so thank you for watching by all means you know please let me know how are you using YouTube for your courses and you can definitely let me know by providing comments below or tweeting them to atletent01 or make your own video response and share it with us so thanks again and hopefully you found this video useful


What's up everybody, it's Jason here with jazmuskatesbricks.com We just quickly sold out a 1920 pinini prism one-pack cello break number 38 random teams And this is to help give away and win spots in our 10 box mixer We're gonna give it with eight spots in the mixer each spot is valued at 120 a spot So eight spots at the end of the break. We randomize those customer names only And top eight get a spot in the mixer each We have number 39 in the store, which is our last one that sells out the mixer And then we can break that open a little later tonight. So let's quickly do the pack break randomize those customer names and teams Let's roll it. We got a five and a four nine times one Two three four five six seven eight and nine Fred Williams then Alexander our nine times here one two three four five six seven eight and nine nine times Magic out of the heat Yeah, and he's close enough for us to do that too guys. I'd say we're gonna do the mixer tonight We might even do the alright alright alright mixer Love the name of that mixer to Fred with the magic title with the war is Daniel That's Maverick of the Rockets Johnny with the Calibri's mark with the wizards Robbie of the pistons Michael with thunder Rob With the Celtics and Hornets Fred with the 76ers Edwin with the Hawks Fred with the Kings Kevin with the Nuggets TJ With the next Troy, you have the Pelicans Fred with the Pacers Chris. You have the Grizzlies Kevin with the Bulls Chicago Bulls with Kevin with Chicago Bulls Mark with the Cavs day with the T-Wills Jason with the Blazers Mike with the Raptors Maverick with the Bucks as well as the jazz That's KS. You have the Lakers Matthew with the Sun's last bubble Joe Fred with the Mavericks Robert the Spurs Alexander with the heat So there you go quick 30 seconds guys if now we're gonna close a trade window. It's just a one quick pack celebrate All right guys. Good luck trade when I close Let's do it and again. We will Randomize those customers are now After this quick sell a pack. All right. Good luck. There's Garland play Thompson green Steph K. I like to bar Parker There you go How's quick these switch scenes again customers in the top eight? Will win a spot in the mixer after? 602 eight times. Good luck one two three four five Six and it was eight times, right? seven and eight in Valentine Boom, are we got Rob at number eight Johnny at seven tj at six Daniel at five Jason at four Rob at three Fred at two and at one remember top eight Yes, so Fred Fred Rob Jason Daniel tj Johnny and Rob. There you go In the top eight Congratulations, you guys are now in the mixer We have one more break and it's already at 25 that unlocks the mixer that we can break later tonight. Jasper's case breaks.com