Hello everyone, welcome to Canalized Demonstration at DefCon Car Hacking Village 2022. A little bit of quick acknowledgement before we go into the talk. Many of the images used here are not created by me and definitely the credit goes to the owners of those images. And all these views or opinions are from my learnings of car hacking and canalized team learnings of car hacking. We apologize if we miscommunicate any information. And also using this tool, you can only control certain functions of the vehicle depending upon the hardware you're using and how those functions are implemented in the vehicle, not the entire vehicle. This is not like a Swiss Army tool of doing everything in one click. No, it's not nothing like that. And a little bit of shameless promotion about me. My name is Karthik Lade and I'm currently working as an IoT security researcher at Pyattu. I love automotive and IoT security, breaking things and trying to do as much as I can. And I've been a speaker at conferences such as black at Asia Arsenal last year in Asia and the cocoon as a truck cocoon is a conference which happens in India, as I said, as I said track in that and seaside conference and besides really car hacking village. And apart from this work stuff I'm an absolute community lover and volunteer I work with the seaside community which is a completely open source community and free conference for anyone who is interested to learn. And I'm an open source contributor along with this tools, whatever I make the content which I make it on blogs and the tools and all are completely open source. And yeah, I love watching Durham one. Yeah, some people say I'm still a kid. Yeah, I'm a kid if you have to remember please give it to me. I would love to see the episodes which I don't know. And generally I go to health stations or something to do this thing. Currently I'm working on a new as I want to secure to audit platform called I would be auditor from my company. So that's about me. And I want to introduce my friend Rahul who unfortunately due to circumstances he wouldn't be here for the recording and he's currently working as a digital specialist engineer at imposes. He is the code guy. So generally it happens is like I coded and I give it to him he cleans the code and he scolds me like writing a bad code, but yeah my code works so I still depend myself. He's absolute open source contributor like me he also did level up a lot of libraries. I think I believe he worked on making mood MCU how can it be used in IOT automated projects and all his a lot passionate about robotics and mechatronics, and he thinks a lot a lot of work. So that's about us. So the main point around of this whole doolies on canvas so if we see why the industry needs canvas and why we need a lot of canvas in in vehicle networks or in vehicle in vehicles. If you can see the vehicle here in the top thing. So there are a lot of components. So what happened is car manufacturers or vehicle manufacturers when they started adding more and more actuators or sensors. They want to make a closed loop feedback based centralized network inside it. So each one can talk to each other and make the whole vehicle go forward or make certain functions. They had to make sure that each and every component is connected with each other. So if you keep going like this it literally makes a haywire of wiring and also it creates a lot of drag because of those wise and obviously the fuel efficiency and all go slow. So they had to come up with a new age protocol in 1980s where you can handle all the data in a very good centralized fashion and you don't have to drop the packets also imagine you want to slam the brakes but it is getting dropped because of a lot of traffic and that is not also to happen. So engineers at Bosch came up with canvas where like you see it is a single line. It actually connects all the components in your vehicle issues and all the stuff and well issues basically. And so actuators and sensors will connect with these issues and all. But the main point here is like you just removed all the wiring and going on single bus. Nevertheless, it also came with few drawbacks such as lack of encryption and lack of authentication while sending these things. And it's another way if we see it works more on centralized fashion so everybody is literally trusting every other packet which is flowing in the bus. So this actually opened up to a lot sort of frame attacks and protocol attacks where you actually do this and all we our tool is more focused on the frame attacks. So if you see can also this has carrier sense multiple access with collision avoidance which actually makes sure none of the packets are getting dropped. And obviously there are many other protocols like can FD and the link bus and all flex day on all this automotive internet at all just can is one of them. And yeah, it is one of the efficient ways because it has CSM as last year and also differential signaling basically to resist to the noises and get this thing. The wire which we are seeing in the previous slide it's just like a single wire a twisted pair of where which post of can high and can low and flows around through all the issues in the vehicle and all. Periodically can also can transmit up to one M bit per second but lowest bit cans go up to 125 kilo kbps. So that is there and if you see from an eagle point of view from a top point of view about a can packet you can see something called arbitration ID which actually makes shows which packet has to go print and control field and data field data field which contains the exact data. So, in a very simple piece of cake point of view, if you know what arbitration ID and what data is actually making a certain function to go on. In this case, a certain function can be a indication or turn a steering nowadays many of these things are happening drive by wire instructions. So if you steer your turning it might be happening on a canvas, any canvas it doesn't depend like everybody should be having canvases, because modern architectures have multiple canvas networks also. So basically if you know the exact packet and the data you can do a lot of stuff, but in order to get that exact packet and data and standard can format is 500 kbps so for that you get a lot and lots of packets so manually reversing all of them is very a hectic process. So this is happening for me when I was participating in different CTFs, especially kind of CTFs, I was finding it very hard to go through each packet and see the traditional way which we all know is like having them each half way replay to see if your action is actually happening or not and then do according to that, but I wanted to come up with some kind of tool which will help me, you know, which will automatically say me okay these are the packets which happened in this specific time frame which are not happening in general time frame. That is one thing. And well I was thinking, you know a lot of this canvas attacks and all this after I joined I started my journey or job real corporate job and all. So one of the attack vectors while I was working on security and automotive they are like, you know, more little words hardware based, you have to have a hardware connection with that. So I was thinking, is there any way I can put a remote thing using this tool because this now has in my mind, you know, the automatic filtering thing and also it has. Easily filter the packets and all. So now I wanted to keep it into a remote Raspberry Pi. So this is the initial layout which I was having in the mind, obviously apart from this telegram port. So my idea was to put hardware, whatever hardware I'm available and keep it in the Raspberry Pi can analyze inside Raspberry Pi and actually put it inside a vehicle through wired connection like a hardware backdoor hardware implant backdoor and but later we found a way we came to know that you can actually send to using telegram bots, you can actually send data to telegram and get this thing and all. So that was one thing telegram bots using, you can also do this thing, but also I realized, you know, after this. Well, I was well I was working on few projects of electric vehicle collecting logs of multiple multiple logs every day day in and day out and filtering them and making sure everything is in place is also very hectic. So now I want something, you know, where I can put all my logs like sessions like if you know the Bob suits right, they can actually collect all the sessions all the requests which are going in and out from the service process I was and collect them in one place. So I wanted to have this thing. So this is what canalize was started. So if I say along with my friend we were actually trying to first implement automatic filtering way by comparing with two files that was the previous version. Then we came to know you know we can use something called hash maps, and we can actually make sure that it goes automatically so you can directly get the payload refined payload outside. I was always thinking out with how can we do over the internet, like hardware implant, and how can we have a session based thing where you can collect all the dump and all. So, and also a few assumptions here. If you are using this tool, we assume that you have the required hardware and also you're in the same bus where you can log the data, and log the internet action which you want to find out the packets and make sure your power supply is taken a lot of times I had to redo it testing and all because the power supply which I was giving is not stable. And also if you're using it as a remote hardware implant like shown in the previous slide. Please make sure you have that hardware has a remote connectivity thing and all. So, why can less there are a lot of things happening as Cappy is there as Cappy has a Python can support can support and all and candle is there. But canal is more on Python can library and it is completely available for all the things which we are implemented using an IDE, which is coming. You can see there's three main stuff smart scan ID and connect to telegram basically connect to telegram works on top of ID which is there, passing comments to HPT PIP key, but all the functions which actually this tool works and all, it happened to Python can library so the code is pretty much malleable, you can add your own functions and you can do it and you can be using it remotely also. And there's one more important thing which we added while we realized while testing of this tool that you know there should be some data transfer importing a log file into it and then replaying it and recording it and all is okay. But what if wherever in whichever session you are creating a log file or a basically creating a particular data frame set of data, and you can seamlessly pull it to another session and all that also we want to influence and this happened and it's a command line based tool I personally like command line based tools more than GUI. And yeah, sometimes I hate it but yeah I love it overall. So that is there. So we wanted to get all of this. Now you might be having the questions like you know Catholic whatever you told us nice and okay but how exactly this happens in your tool. So for that, let's see the process flow a little bit. And like I was explaining, let's say the main concept here is like you make two files, which you can compare with each other. And then you actually do the whole use the whole tool to automatically do the stuff. So let's say the first file as source file and imagine you're on a road or you're riding your vehicle. You have some 40 kilometers per hour speed and suddenly do this is not in steady state, even though you are running it 40 kilometers per hour steady state. But there is a lot of data, huge amounts of data has going through in your canvas networks and doing all these things. But you apply a break or you give a signal or you turn the steering or do something which can actually be controlled on by can packets. The use of that packets compared to all the packets which are having on the bus is less than 5%. So what we do here in this tool, particularly while we are in this tool is we create a source file, which actually contains a general and not contains the intended packets. Let's say my intended packets right now is to open the doors and put on light and left indications. So we create a source file first. And then we record an action file, which we create through SmartScan or ID. In SmartScan, it is automatically done.  And then you can actually pass commands to do this stuff. So that also you can do with action. What we basically do is like in source, basically you don't have the intended action in action. You have actually the intended packets are present for what you're trying to do. So then what now. Let's call it as noise. It's a general traffic in the bus. And now you can have action file where you have intended packs of action, definitely with the source. And also I have here noise start. Why noise start is like there might be additional packets which are generated in time of your action happening. It might be affecting it might not be affecting it might be different stuff. For example, if you're slamming the brakes, two packets are being generated one to send it to the actuator break actuator blitz and all to actually slam the break and also same time indicators on your input in my hub, or the dashboard has to show up that you know breaks are slammed and all. So if this might be in your interest, this might not be in your interest. So generally, if we keep only featuring this source and attack file, the action files. They basically have extra packets which you might want don't want to know, but using smart scan and ID you can actually filter into that level also. Don't worry we'll see it in a detailed demonstration and coming forward. Yeah. And there is payload events we payload what happens is like action minus source obviously and noise start you can depend that we will see. So if we put into a when diagram point of view, this is something you want this is the niche ice cream you want, which you are interested, and it might be coming with a noise and it might be having some overlapping with source. So all these things and so now you're saying like okay, we have a source file you can have an action file and you want to create a payload file you're just doing some kind of filtering picking up packets and all. I'm doing this, but what if you have multiple action files you want to record that also you can do with smart scan along with the ID smart scan you just need to do it in a recursive way. Keep telling it now you have to collect the action for the action for it will does all the things and it can analyze between them and create a payload for you. And what if you have multiple source files, since smart scan does the functions recursively, it actually records all the packets recursively without even using it continuously checks the pulse, compares between two files and doesn't think. And you can generate payloads also. But what if you have multiple source files you want to record multiple action files you want to record and you want to create multiple payload files that you can do in ID, what in ID we are using this we are leveraging the power of an ID environment which you get and Python can and also the database for advanced and faster filtering and also SQL commands because a lot of people know SQL nowadays we thought like you know instead of doing it by hand. Why don't we use SQL power of SQL here and basically help out people in making the process easy run SQL command to treat the logs you created as tables and run SQL commands and then get out of this things. Well, if you're interested on how this exact code works, I have a few, you know, main functions of it, the collect signal function. So basically what happens here is like whenever you enter into the smart scan mode, it starts this function collect signal and where it collects all the data which is appearing on the particular communication interface you have mentioned in the settings. For example, if I'm using a simulator in this case then my interface would be watchful can zero and yeah, the socket can and channel is what so I can see when the interface socket can, it simply picks up all the data, and it quickly converts into a hash map, and whenever you go to noise, you can call it as instead of signal you can call it as source. And when you go to noise, it will also create a check with the hash map or whatever the packets it is coming in the noise file, whether it is present in action or not. So here the assumption is like if it is present in both files, probably that is not your intended packet. If it is not present, then yeah that is your intended packet. We are actually checking the whole data also we are not, you know, simply checking the IDs we are also checking the data. So it comes out that way. And yeah, it's enough of what myself, the process flow and the whole, you know, the code flow and all let's actually see the real mate here, the demo.  So, yeah, it's good thing but it's also a bad thing because I was not able to get a real demo to this talk. I still regret that. But yeah, when if you're doing it in your car, then please make sure you don't break your car if you're using your friends cars or your colleagues or some well wish of cars. So please make sure you have the necessary permissions to all this stuff. I'm currently focusing on the IC simulator. So guys, welcome to my VM lab. Here I want to go with the VM for demonstration purposes and I just want to put it with show. I'm using three here IC simulator and can you tell support and the canalized. So once you get to want your versions and you installed all the required things. You can do this CD into the canalized and it should be something like this. So this means like you have all the things I have already listed all the requirements in the thing, whatever we are using inside is. So please run it using the three minus requirements for text and it will do all the stuff for you automatically and then use pseudo I recommend pseudo because we are putting a lot of file saving and all stuff. So you can do pseudo Python three canalized interface top five. So yeah, this is how canalized looks when you open it. So before going to the options I want to quickly show you about the settings. There are three settings here you can set up your communication channel communication interface and API token. Since it is developed on top of Python can impact using Python can. Thanks to all the developers who contributed to Python can who made our lives easy in developing this tool. You can also use all the channels and the process the Python can use supports so I want to check what channel it has. Yes, it has a weekend zero which is I'm running like you guessed is simulator here. It also runs on weekend zero and the interface is socket can. Yeah, so now let me just start with the smart spend where we said, you know, it will automatically do the stuff. Yeah, so here the few options you can play save and you can put from this thing. And also you can press the space bar and start feeling the signals and all. And yeah, if you don't like this key bindings you can change it anytime and completely open source so you can do it on your own. It's a very easy code. So, but till the time we are on a conversation that smart scan collect noise function actually silently recording all the packets which are going back. In the weekend zero. So when I just hit space bar, and I give the right and left signal just for easy purposes, and I was being it directly came with the packets. So yeah, why it immediately get the packets. I'm doing something bad here no no no so just because we are having a converse in a lot of time, it quickly found out you know using hash map it is the only packets which changed while here. And you can actually do one more thing we can go this time faster and do to see how it doesn't live. And I do. Yeah, so if you can see this 244 if you know is simulator 244 is this for signal indication, but because it does a hash map, it is constantly filtering all the stuff automatically. And you can save this logs. It should give back to two packets. For example if you're not getting that filtered thing you can do it because only once again to get the exact same packets and all, you can actually save this file, then press s to save and then let me say this is my village. 2022 block. And it will save in that particular folder in that let me just open my tools and can alize yeah if you can see here it just mentioned the lock same files along with the time stamp and sorry for my UI. You can see it also took the time stamp and all the IDs and all. So that is about the smart scan but what if you don't want to do it automatically, but you actually want to do it in a smart ID way. For that you can use channel is ID so ID it's like any other Python ID. Please edit this out. Yeah, so ID is more like any other Python based ID you here you have like command line interface you can set pass some functions there are set of predefined functions, which we will see now, and you can do that and all. I will just record a few variables here and after that I'll make a new file noise and I'll do use the scan function which is predefined here and I'll mention, you know, collect all the packets from weekend zero channel, and then do it for 60 seconds. And yeah. So what's happening now is the functions which we use to defend the scan function it is by using Python can this collecting all the packets which are happening in the bus and keeping it to a Python data frame, basically, Pandas as pandas data frame, so that we can go forward and use it much easier and all. Let me let it complete for one minute, and then we will do the same thing for action, but that time will open the doors close the doors and give indication in a particular sequence to see whether it is able to collect the same sequence pack and filter it exactly. So let's just wait till the scan completes. Basically, the more you scan for noise now the more a fault reduced your thing because it has a big directory of all the messages, which you're not interested or that is not exactly happening in your intended action time period. It's good to have a good filter so that you get much fine business. Yeah, now it completed. Let's do the same. Let's make another action by and scan it for particular 60 seconds. You can zero and then 60 seconds. But this time at the same time, what we do is we'll open the right indication and open the doors and do the left indication and close all the doors and open all the doors back and close all the doors. So technically speaking, all this package should be inside the action file which we are creating other variable where we are storing all these packets and we should be able to compare the noise and remove the noise from this action. So there are any extra things we can also do that since like I mentioned we are using SQL here.   It is now also like you remember it is considering a lot of noise like noise star and all. Yeah, and let's see how much packets it actually collected in this format, or in this time period. Yeah, it collected around 4643 packets and in action, it collected around 4700 packets. And you can also increase the speed and see you can do this for more just for demonstration purposes here I'm doing it very, making it very simple. So yeah, that is there. And let's just filter them. So filtering is very simple using a skill commands just run an SQL command here, consider these two as tables and I want to select ID, comma, data from, let's say action obviously action and I just don't want which are there in the noise value except, you know, select ID, comma, data from noise. So that's it. And it literally stemmed through all 4700 packets and it defined like reduced it to very less number of packets let's see how many. Actually, 244 should come here also, but since we recorded for a large amount of time it didn't appear here. So yeah, it literally filtered down to six packets. This is good enough if you want to do it manually if you want to send each packet and check and all, but this is not good enough if you want to store this long in a separate way. So for that, this is actually present whatever happening here, these packets and all these are actually present in the files which we created. So we can actually use the SQL to match with the same and to color the exact timestamp and channel, which are corresponding to this ID and data for that, let me just do something like, you know, payload. Let me create a payload, which is SQL of select a dot timestamp from my a dot channel from my a dot ID from my data from action obviously to be only available action action a and I want to use this SQL function in a joint and in a joint to filter. In a joint to filter, let's call it as B on the junction a dot ID is equal to the dot ID and a dot data is equal to be taught data. So this would be the command to actually pack it back into the same. Okay, there's some issue here. Okay, so let's do something called load and actually pass the string into this just copy this. And now we can verify this, but this is the thing. And let's just load it into a payload. It is equal to SQL of just give me a second. I want to see what. Okay, I made a small error. Instead of filter I'm just saying Peter. So let me just run that once again. Let's just see what is on your filter. So what is equal to select a dot timestamp a dot channel comma a dot ID my a dot data action. In a joint filter be on a dot ID is equal to be taught ID and a dot data is equal to be taught data. I don't think this works. Yeah, I had to pass it in a string. Very sorry guys, very sorry killing your time. Okay, now let's see load. Yeah, and now let's do payload is equal to SQL of load. Yeah, now it just packed everything into the one payload five. And a lot of it is so yeah now you can see it packed with all the things and here you can see what order it became the it also packed into those stuff here you might miss the order a little bit because it is on different sorting algorithm. But when you pack it back according to timestamp it will adjust back all the things. And now you can actually play this payload thing which we created, and to see if it is happening on the same. So same thing like the same thing and all scan function like and all so we can zero and just to payload. Please don't make a stringer as like I did. I'm very ashamed of it now. So if you see it actually does in the same order which we recorded which make confirms that this tool will not reverse your can prior conversion and all those things it doesn't do all the kind of things. It actually works on time stamp basis so it does like that same close the doors. Yeah. So it completed playing all these things so this is one thing but we it is a very small thing now right now we just created two actions and one action and one noise and hardly 4,700 packets are there and only six, seven, you know payload packets are there and seven, all these things. And but yeah if you might have a question like you know why it is giving more than what it is supposed to be here. So when it checks this is that noise start I'm saying it checks that and then it sometimes it gives you, you can actually particularly remove something from this thing and you can also do that, for example, I want to remove 190 so I can do something like, you know, payload, it's on filter so let's just do filter one, which is, you know, let's say SQL of select all from filter. In fact, you can run payload also here, you don't need to worry about that, where ID is not equal to 19p and just close this and run it. So if you see filter one will actually give me the packets which you want. So in this way, like was explaining you can create multiple files payload files also if you want and all. But even though it is very small but we created a lot of stuff we ran a lot of commands we did some blunders, I did some blunders. So yeah, what if I want to have this whole session exported in ID you can also do that for example when you're doing different version projects you work you're working on assessing how good the break function is, but you want to extract, export all the things, you can simply put export, and just type the name, I want to do CHV 22, like I did. Let's just add another way we'll talk and yeah, that's it, I just pass it as a string, and I put enter right now it is exporting every single command and everything. It did completed also wave pass. So all the things which we did till now are actually done here. So let me just open that you can see here. There's a CHV 22 top folder here there we have logs, all the logs which we created filter one noise action payload, even though you don't convert into logs, because whenever you play it back actually makes in the log. You can also, for example the filter which we never used it anywhere. So it can also convert all your things into a CSV and it can save it to you. You can also do this in the CSV way, and also here we use some kind of meta language kind of stuff, the dot twice. So you can actually see all the commands which we ran till now, along with all the blunders and stuff and also if you see here, we also have this thing. The data dot class, what data dot class actually does this like I was explaining can analyze ID you can do seamless data transfer. So right now I exported this logs so I have this my payload and filter one and all. So if I want this into any other project, I don't need to copy paste copy paste and all I can simply call or import from this particular directory, this this particular payload prior log but it is that you just import that. So we'll do that I let me just exit from here. And do it into new thing where I want to show, you know, do we have this stuff from now. So let's say, let's try for filter one dot log. So let's see if we have filter one dot log or something, you know, filter one variable at least. It's not different. It's completely remote. But I can simply say, you know, just get filter one from here. In fact, I can give another name CHV is equal to, yeah, let's just filter one you just need to copy paste it and then yeah. Okay, again, blunders, a lot of blunders today. Yeah, filter one you just got the same thing you can actually do the same thing here like we did same ID so you can play it. We can zero comma filter one, and it will just keep the thing signals. So once it is done, yeah, you've got the graph one doing, yeah, left one also both of it's done. So this is how easy it is using canalized ID and recording the projects and doing all the stuff. Yeah, let's also check out the telegram part. So to do that, I'll quickly open my telegram. I have also created a bot known as canalized bot, and I have this API token. I'll just copy it here and I'll go to settings. And I'll put my API token here. That's it. So if I want to check, I can go back and check. Yeah, it's the same one, which I just used and I went back. And I want to go to initialize this bot. This bot is initialized. Let me just connect to Telegram. Let me just start this. Yeah, it gives a high cut means it has binded with your bot. And what happens here is like, we can do all the same stuff which we did till now. And if you remember, we have multiple files here, so we can actually use them and do the same kind of same commands. The same commands and to see, but we'll just restricted to 10 seconds like that. So let's do that. So let me just collect the noise once again for 10 seconds. So whenever telegram while you're using to telegram pot, please make sure your port is initialized control and also when the command execution is completed.  Otherwise it puts you an error. And also let's do one more action by this time. Let me do the action by for 30 seconds. I want to do something. So I want to open only two doors here and then, yeah, let's see how that works for us. I'm opening the first door and doing the right signal and I'm opening the second door and then in the left signal, and I'm literally closing all the doors. So let's see how it happens in this. So same commands like IDG, like I discussed it simply uses the whole commands from IDU to do here filters now. Like we did. Here also you can see it says message received and output is good and all. Yeah. And also you can load it back into a payload. Okay, again the same error because yeah, there's no fighter there's only filter. Yeah, it's done. And you know we can actually do this also. Yeah, it is done and this I'm literally creating the file from the load, and I'm playing back the payload file. Here you can see it is following the same order of opening a door and then being the signal and then opening the other door in a few seconds it will do that. And also here the conditions like, you know, the files which we have created, we should also be having all the packets we wanted that is also a condition. So if it doesn't have because of the time frame used 30 seconds and 10 seconds if it doesn't actually was able to filter it out, then you won't be able to put up but yeah it got here it's very late. So, yeah, that is this thing like there you can also export here also you can just send an export command it will save it to your own thing. So really, I'll just put that once it is done it just gives an export time. So whenever you collect back your device, you can actually see that is also the rate of CHV like the same format all the commands logs we have used in the payloads we created and the filters is everything the same. So this is the demo of the channel as ID Robert please cut it here I will stop sharing my screen and once again my slides. So I hope you love demonstration tool demo and all I did make a lot of blunders in passing the commands and all I hope you can forgive me for that and yeah that is about ID I believe we the team channel is believe you love this and all. And also, since we are saying these things and a lot of stuff which needs improvement at one of the things is the terminal UI, we are planning to improve the US or it makes more user friendly both in ID and smart scan way. And also I know how powerful we know how powerful is and we will are looking ways to implement or like gets happy also in the ID so that it helps you doing a lot of advanced stuff also. So first I'm not sure right now because I want to master it first and then I want to put in this tool. Otherwise it just ends up like any other tool which is available. I want to solve something there. Yeah, so that is my future plans. And like we are discussing this tool is completely open source and made for community people who want to get started with hacking or who wants to make the process more enjoyable and easy. So if you have any idea, please don't forget to contribute to the tool. It's just in the GitHub. It's just a pull request away. So please do try it your thinking stuff and all. And finally, I would like to thank my mentor and my company Prashant KV and my company by two for supporting me in this, and also I would like to thank Rahul in this moment for being with me in all this days, and actually running through all my bad cold fifth runs, by the way, and all those things. And also thanks to whole guy King Willis team who pulled it out. We know how much it is hard to run a conference and interesting. And thanks to DevCon USA. Thank you all. Thank you for listening to our talk and supporting us. Thank you.


All right, time to come to start, sorry. So I realized that I just have 35 minutes for this talk. It's a bit short, but I'm supposed to boot fast anyway. So let's do that efficiently. I hope you have a fast brain as well, at least at this time of the day. So I'm Michael of the Knacker. You can read that. So actually, this presentation has a goal to actually give you some measurements about boot time techniques that can be used to help you guys know in advance whether some technique is worth exploring or not. So I'm going to go quick on the technical explanation, like how this works and things like that, and add more focus on the savings that I achieved on this kind of system. So it's basically a bigger bone black with an LCD cape. Connected to it, there's a standard USB webcam.  So it's going to, on the software side, it's going to play FFMPEG just to show the video stream from the USB camera. And then the initial time for the system was 9.45 seconds, including late the seconds time out in U-boot. So it's easy to optimize, of course. Another thing being very fair here. So again, here the goal is to give you results to use. And in particular, for U-boot's Falcon mode, I had trouble finding some details. Like it seems that not so many people are using it yet for boot time optimization. And it's really worth it, actually. So I'll share some technical details and some results with you. So here, there are great presentations about boot time reduction. They have already been archived from previous ELC conferences. So I'll give links to them. Here, I just want to give you simple principles. The idea is to first focus on optimizing things that won't hurt your ability to further reduce and measure in the next steps of the boot process. So start usually by the end, like user space, tool chain, and things like that. And progressively move into the early boot steps, like the init scripts, and then the kernel compression, and eventually bootloader. But when you optimize the bootloader, you won't touch the kernel again. So that's fine. If you do that the opposite way, you lock down. I mean, you make it much faster. You use the ability to optimize the kernel anymore because you have no tracing, no debugging. So another technique I'm using, too, is keeping quite slower, relatively slow storage, and quite as long as I can so that I can, like it's like a magnifying glass. It makes the times longer and easier to observe, like some deltas in behavior are easier to observe if they are slow enough. This way, I don't use like the fastest compression, for example, I keep the default one, which makes a bigger kernel, which takes more time. And of course, you can have a boot time presentation without this quote from Donald Knuth, that the premature optimization is the root of all evil. So first tool chain optimizations. So I just compared an ARM tool chain with a TAMTO tool chain, and the result was surprisingly good. Like with TAM, I have like the full system size is 18% smaller than with a regular ARM tool chain, so it's really worth it to use it. There's no real performance benefits. It's about the same performance, so I'll buy it. I'll buy that. Smaller in the same performance, I keep this. I also tried to replace UCLib C, well, obviously, GLib C as well, by Masson. And we have a win in terms of size, library size, starting with a file system that has the library inside it. So I'm saving 16% of the library size, so it's worth it. The goal eventually is to have a small system that will fit as an intramfs in the kernel binary to have like just one read from the MMC instead of multiple ones. So one of the goals is to reduce the system size as well to make it faster to load. So we stuck to UCLib C, then optimizing the applications. So the general idea is to compile your application with less features, less dependencies. And yes, so at the end, the program is smaller and you have less libraries that are built. So at the end, what I got with M5MPEG was a reduction of the total file system size as generated by build route from 16 megabytes to three something megabytes, like almost minus 80%, which saves about 150 milliseconds application loading, execution time, because probably it would try less things. Normally it is loading on demand, but probably the code is trying to load a few things and testing things. At the end, the total reduction was about 350 milliseconds, maybe because they might be some fast amount time as well. I expected less, but let's take it. Now let's talk about the init script optimization and also reducing the size of the root fast system as a whole. So there are some techniques that are documented like to analyze the boot process with boot char d, for example, or using some tools associated with system d. You don't mount slash proc slash js, you simplify the busybox configuration and think a few things like that. They are well documented also on the Elinux subiki that is nice chapter about boot time reduction. And switching to static executables. So I'll go through each of them. So yes, effectively a smaller file system is faster to mount, especially as our goal at the end is going to be to load the root file system inside the root file system. So the kernel will be smaller to load from storage if the instrument faces smaller. Kernel decompression will be faster too, normally. Well, I'll explain that later. And effectively the kernel and file system are loaded inside a single read operation from storage which also makes sense rather than multiple small ones which may need time to start and that could be some overhead, the file system layers and things like that. The intramfs is a very efficient technique for accessing files. So a technique to detect unnecessary files is to take advantage of the fact that Linux stores the last access times for files. So don't hesitate to boot your system the first time and then just take your SD card out and just run a fine minus a time on it and you'll find the files which have actually been accessed during the boot process. So that's a way of eliminating the ones that were not accessed or finding out at least and you can decide. What else? We also simplify the busybox configurations which this reduced the size from 600 kilobytes or 700 kilobytes to 86 kilobytes with dynamically compiled against Ucdip C so it's quite nice in terms of size. And at the end, the total file system size was reduced by 34% like down to the 33 megabytes. In terms of boot time difference, it's hardly noticeable because of the on-demand loading, I guess, and also because the init scripts didn't take much time anyway. Here in that system, I had just had two executables, actually busybox and FFMpeg. It really made sense to eliminate the shared libraries because you have a lot of code that you eventually don't use. Effectively, you might copy some code inside two executables, FFMpeg and busybox, but the overhead is still not so much. So here, as you can see, you can see the content of the file system at the end. It's the only thing that's left in the file system after my optimizations. So eventually, yes, I'm down to 158 megabytes of total storage space, as measured by the TAR archive that Biltroot generates. So that's all I need, actually, at the end. Some file system optimizations. Well, we could test various file systems here. The goal is simple. We're just switching to an interim FS. So we did compare with other file systems, but it really made sense to switch to an interim FS in that case. So the root file system is embedded in the kernel image. This will just want access to storage, as I told you. You don't need block storage and file system drivers. So the kernel is gonna be bigger because you include the file system inside, but once you remove the, that's the next slide, actually. Once you remove the block support and MMC support in the kernel, you actually get back the overhead, no, not exactly. The boot time that you lost because the kernel was bigger. The kernel is still bigger with an interim FS than it was before, but you recover the extra time by compressing the kernel and copying it. So there's something you have to remember that people may not realize is that it's very important to not compress the interim FS, which is not the default, at least for the Big Bone Black DevConfig file. So make sure you have an interim FS, sorry. Compression none. Otherwise, the interim FS is compressed twice once before being embedded in the kernel and then the kernel compresses it again. So this actually achieved a reduction of size of 200 kilobytes and saved about 170 milliseconds of boot time, right? So at the end, effectively I got back the same boot time as before, even a little less than before, even though the kernel was bigger. Thanks to having less things to initialize at boot time, like no block support, no MNC support. Now a few words about kernel optimizations. You know there's this in-call debug feature in the kernel that allows you to dump actually some information about the starting time and ending time of function calls during the boot process. So you can use this to dump more information in the kernel log, and then you write your demo stage or you copy the console output to a file and you process it with scripts, boot graph.pl and it generates a graph like this with the biggest consumers of boot time. So one of the first ones was, I don't know, first the technique, you could use, you could actually get the names of the functions here. You can look them up in edX here, typically, edX.bootlin.com that indexes the kernel source code. And then you can try to eliminate, to figure out what this means and whether you need it or not. So just be aware that some functions that are reported by the in-call debug are actually the name of a module and the score in it. They don't correspond to an actual function name in the source code. Then you can use some techniques to try to optimize the existing drivers, like looking for parameters that would impact the behavior of modules and account for the increased delay. So just an example, using this, I realized that before this, there's a tracer init tracerfs. So all the init tracing infrastructure here was taking about like 550 milliseconds to initialize. That was enabled by default, so I removed it and saved a lot of space and time. Others, there's the TTY interface, the serial interface for the OMAP which was taking a huge amount of time. I didn't find an obvious reason in the code. I found some parameters corresponding to the number of TTYs that are initialized, but haven't managed to change that yet. Other things like a network driver that will be disabled anyway, so I took care of it. Some things that are related to the USB initialization of the camera, it cannot be skipped, I have to keep it. And all the other ones were actually not low hanging fruits, they were quite big, quite small, sorry. So I meant, I expected to save the corresponding time by eliminating some features from the kernel by working on the kernel configuration and reducing the number of features. The preset loops per GP is a well-documented technique for reducing the time, so when you boot for the first time, the kernel is going to estimate how fast it goes through the calibration loop that you use in new delay. So it's not necessary to run this loop every time, you just do it once, get the loops per GP value and feed it to the kernel command line. So in the past it was saving more time, now it's saving 82 milliseconds. On the Linux, the Linux wiki, it may be still 200 milliseconds if I record correctly. Or 25 GPs, or 250 GPs, yeah, 25 GPs. Another thing is SMP, so people were saying try SMP, but I have never tried, and effectively on a single core CPU, or even in a multi-core CPU, it could make sense to disable SMP, which makes your system faster, like this, like 126 milliseconds of boot time savings, and noticeable size reduction as well. Compress kernel, so all the time when I mention a kernel size, it's going to be compressed, so it's even bigger if you decompress it. And it also contains the Inetram-FS, so it's not completely fair. I mean, if you remove the Inetram-FS size inside the kernel, it's even better savings. So here, if you have only one CPU core, you could, that's a clear winner. Otherwise, if you have multiple cores, you might try to start with, you need SMP, of course, but you may try with one core and plug in the next ones. I don't know what it's going to give you. Removing kernel module support will save 82 kilobytes of compressed kernel size, 20 milliseconds of boot time. I had a bigger number in mind, but now that's better, probably. I remembered from past experiments that something like 300 kilobytes, but now it's more efficient, I guess. But again, it's the compressed kernel. So be careful when you do this, be cautious, take your time, otherwise you end up like being very feeling lucky and removing lots of things and you have no clue why your kernel doesn't boot anymore. Oh yes, and notice that what I had to do to do this is first remove all modules. Otherwise, if I remove module support, all the modules are turned into, which I don't use in my system, I don't load modules, are turned into static ones, and you end up with features that you don't know whether you can remove them or not. So I'm looking for a way to turn all modules to no in an automatic way. So maybe by just doing a set on the config slides, but it's going to disrupt some dependencies, so I'm not sure exactly what the best way is to turn modules into no. Maybe a new star config makes something, or no modules or something like that would be nice. So remove all modules first, and then you can remove module support and then remove the static ones that you don't need. Another technique is to silence the console with the quiet command line parameter. It says 577 milliseconds, that's good. That's very nice because the console is a slow device, so that's easy to understand why it saves that much time. Then, of course, you can go ahead and beyond and remove complete support for writing messages, like even the messages don't get compiled in, which saves a lot of kernel slides. So disable config, printk, config bug, which saves about 5% of the size. Also removing case old sims, which surprised me because it saved 107 kilobytes for the compressed kernel, so it needs a lot of space. The total savings for both, although these things silencing and turning off the kernel messages saved a lot of time, 700 milliseconds and more than 200 kilobytes in the compressed kernel, so in terms of, before compression, it's probably much more. Using config embedded and config expert allows you to turn your generate kernel, which can run any application into a dedicated one, which can only run the system calls that you need to run. So you can come out, compile out some system calls that you're sure you never use in your particular system. This reduces the size by 51 kilobytes compressed and a boot time by 34 milliseconds, essentially because there's less code to initialize and the kernel is smaller. I tried some to kernel, didn't work this time, so in user space it was a good result, but in kernel space, surprisingly, the same tool chain, I got 40 kilobytes of extra space and a total boost time that was also increased by five milliseconds, so I didn't select that one. Probably need to. Any idea why this is this way? Ah, okay. So I did try, effectively, to disable arm unwind separately, but you advise to do that together. So, I'll try, yeah. Thanks. I'll do that the other way, like first disable the arm unwinder and then thumb. Okay, sounds good. I made some tests with the slab allocators. No change. Slab is still the best. Slab, which is supposed to be much simpler, is actually not so good in terms of performance. It's a disaster, even for, well, I have like 512 megabytes of RAM, so maybe it's too much for slab. So I stuck to slab. So slab apparently makes sense for very small systems with a very small amount of memory. If I had reduced the amount of RAM, maybe, ah, yeah, that's an idea. Like boot my bigger bone blank with just 16 megs of RAM and see if it gets bigger, if it gets better. I forgot about that idea. But, effectively, it's much, much, much, much slower, like more than almost 150 seconds of increased boot time. Kernel compression is interesting too. You have various compression schemes that are available. By default, it's as an MA, at least in bone black. I guess it's more dependent, or maybe it's a default, which makes sense, I guess, for 86. So I made some tests. I updated the test I had before, which were based on a 3.1 kernel. So here, as you can see, LZO wins over LZMA, and even GZIP, which is actually the closest one. So the best contenders are GZIP and LZO, but LZO wins. So LZO is a very fast decompressor. It doesn't compress as well as GZIP can do, like 15% less, something like that, but it's very fast. So you have a bit bigger kernel, but it's much faster to decompress. At that time, it was time to switch to faster storage, because we are close between GZIP and LZO. So I selected some SD cards with better performance. And eventually, what happened, I found a model that was not the highest end model of SD cards, but was the best performance I could get. Even if you take a Sandisk, extreme, extreme, extreme, something, the limitation is eventually what the hardware can do in terms of read performance, I guess. So here, as LZO still wins, even with the differences about the same. But then from now on, I'm sticking on the fastest storage, but I didn't want that to disturb the tests. I tried CC Optimize for size, so compiling the kernel with minus OS instead of minus O2. Actually, the results depends on how fast your CPU is. So on the bigger one, black, the winner is O2. The kernel eventually is, no, OS is a little faster, a little faster, but hardly for many seconds. But if you have a slower CPU, you can really have OS slowing down your machine significantly. And keep in mind that the system calls that you're making will run some kernel routines that will be slower to execute. So that might not always be a good solution to use OS in terms of long-term performance. So here, I chose OS, but it really depends on your platform, on how fast, your CPU can be, or how slow it can be. Other things, removing the proc file system removes about 50 kilobytes of space, but in my case, even though proc was not mounted, I believe, FFmpeg stopped working when I removed support for proc. So maybe FFmpeg mounts proc by itself, if it doesn't find it, that's funny. So at least I could remove some support like proc slash sys, but saved a little bit, not much. Removing ccfs saved 22 kilobytes and 35 milliseconds of boot time, so that's good. So you can do that if that's compatible with what your applications are expecting, of course. So if you're lucky you don't use proc and sys, you're fine, otherwise you can't do that. Removing all the compile time checks, all the compiler options in kernel hacking will save about 40 kilobytes, especially config debug info, which I expected to be bigger. And also, effectively changing the arm and winder technique for nicer stack traces, replacing the EBI stack on winder by the default one, saves, effectively, 24 kilobytes of space. So I'll try with the FFmpeg 2 and see how they interact. So there's a little bit of here increase in terms of booting, but it's almost negligible. So I kept the default and winder, now in the original and winder for arm. Another technique you can use is appending the DB to the kernel. It's just because I observed that, I observed that if you, in your boot, you see the first big loading of the Zimage is fast, because it's big enough, but the DB is smaller, and therefore the performance is not the best one. And therefore, oops, the idea was to use the old technique. When you boot didn't support the DB, you just appended the DB at the end of the kernel, and the kernel binary, and it just works. So I'm just doing this, like cutting them and loading that, I just have one image to load in set of two, and effectively the performance is better, like I saved 20 milliseconds of boot time, like I measure that starting kernel time, user space starting time. It's a little bit, but it can give you the extra milliseconds that you need to achieve your objectives. And to finish, the boot loader optimizations. So there are lots of techniques that are well documented for improving your boot, which is a slow program by default. Here, I just decided to skip UBoot all the way, and using the Falcon mode. So it's just, instead of loading the second stage of UBoot, you just load the Linux right away, taking advantage of the Falcon mode infrastructure of UBoot, which is actually easy to use, and the next thing about it is the same technique for using it, whatever the board that you have provided, it supports the UBoot SPL. So you need the UBoot legacy image, so U image instead of a Z image. So you have to load the loading address of that kernel inside the U image container. You copy that to the SD card. An optimization in the UBoot compile time was disabled support for an environment in the SPL, which apparently is possible, but takes a lot of time. So I saved 250 milliseconds, that's really, the environment was huge, like 128, whatever, maybe, I don't remember the size, but it was unnecessarily big, right? So in case you want to use the Falcon mode, we have full details on our training labs, I'm sorry. But here are the most important ones. So to run the Falcon mode, you just need to first load the U image in UBoot, set the boot tags, if you don't have it yet, you really have it, actually, when you boot in normal mode. Then you simulate the booting the Linux kernel. So it's just like boot Z, or actually boot M with U image, it just loads the kernel headers, prepares the eight tags for passing that information to the kernel and boot it, but it doesn't do that, it just prepares that information from reading the kernel binary and writes, eventually you can write that to a file that's called args on the MMC card, just storing that information so that UBoot doesn't have to compute it every time you boot. Here the size is a bit arbitrary, it can be probably smaller, so that's 16K for that information, that I didn't have time to figure out exactly the exact size of it. So once you have done all these four steps, you just can reset and your boot will, your board will just boot with through the DSPL, the first part of UBoot and directly and straight to the Linux kernel. And in case you need to get back to the previous UBoot, just take your SD card, do a normal UBoot, just take the SD card out, remove the args file, and you're good to go. And that's the normal case with DTB, so that's the same thing, except that you're, instead of exporting eight tags, you export FTD information, flattened device tree. But otherwise here, when you reduce SPL export, we have the address in RAM where the data are, like this, this one, sorry. And the size of it, like the end and the beginning, so you compute the size from both numbers. So the Falcon mode is nice because it saves almost 500 milliseconds of boot time, like not going through UBoot, so it didn't even have to optimize it. And the total boot time at the end here is now 2.5 milliseconds. So here I have something to explain. That's, this time is when I'm in use space, so I'm in use space after 75 milliseconds roughly. And now I have, think because of the fact I'm connected to a USB camera, USB is asynchronous, I have to wait for the camera to be detected. And in my case, it takes 1.2 seconds. I haven't managed to address that yet. So it's a bit frustrating because I'm doing nothing for one or two seconds. I could like display a splash screen and do things, or just instead of using a camera, I probably will switch to just displaying a video which could start immediately. So now you can see after one step video is ready, I asked a run FFMPag, which takes about like 500 milliseconds to run and start showing the first frame on the screen. So I modified FFMPag to actually write a message on its log after decoding the first frame and actually to write a GPIO as well so that I could count the time on Arduino that's connected to the board. So I didn't manage to effectively to reduce that time for USB, any ideas for making USB synchronous? Like in the driver? Okay, cool. I'll try, like the USB core? Ah, okay, thanks. Hey, thanks. Didn't manage to boot yet without the TTY layer but there's probably needed extra work but I expected to save hundreds, a few hundreds of milliseconds. And so if I eliminate the 600 milliseconds that I was found in the boot tracer plus this 1.2 seconds for the USB camera to be enumerated, I'm gonna be, I should be below one second. One second, so that's quite nice. Yeah, the time is limited, I just have two minutes left. So don't miss Chris Simon Simons' presentation tomorrow on optimizations related to system D. You have other presentations from past ELCs which are brilliant. So I put the ones I love best. There's also our boot time training materials and labs which are more extensive slides on the various techniques to be used and advice. So that's a training course we can teach as well. So questions, suggestions, or comments if we have time. And I like to finish with this, like a picture of the various techniques that I used and highlighting the most important ones as hopefully it helps you to explore some techniques which are really worth it in our case at least. Any questions? Yes. One of the suggestions for you, if you change the font size, you take your image from the font. If you make it very small, you can reduce boot time or if you use raw partition instead of font, it would be even faster. Also, you can remove font support from the U-boot which is quite true. That said, I haven't had time to do that but it's the next one. Yes, effectively. Directly read raw data at fixed offsets rather than reading, going through fat. Thanks. I have a question about another site of this optimization. How do you debug such a system without pins, without trace? It's getting worse and worse. Do you enable everything back and debug it or? No, well, I really have to do things in the right order. Debug first and then optimize. But so analyze the performance with things like S-trace and things like that. Do as much tracing as I need to do ahead of time or not too late at least. Otherwise, with this approach, effectively at the last stages I'm stuck, effectively. Actually, one of the things I used to do was I had some macros to write directly to the serial port. And then I could use a serial timing program. Or you can, if you have a logic analyzer, use gpios. Ah, yeah, so it's not like having the full printk but just for your own... Just out of character. All right, good idea. Or a number, or maybe... And by the way, I was using grab serial from you, so thanks. It's worth mentioning. Everything was done with grab serial here. Thanks. My beyonds are to this and I didn't understand it correctly but did you already... Did you at all include init scripts in your example or did you just do init equals ffmpeg? Yes, right, exactly. Yeah, so init equals my play video script that waits for the video device to be ready and then start ffmpeg. So just one... I replaced the init script by just my own script, which is not init anymore. So if you... Effectively, if you exit, there's a kernel panic immediately. Have you tried bypassing the init script altogether, just pointing directly to ffmpeg with a sim link? Ah, right, but I still need to parse the command line for ffmpeg. Well, I have a script that calls ffmpeg almost right away waiting for the device, video device to be ready. But it should work. Effectively, you could have all the mounting and things like that is easy to do from C code. So waiting for the device file to be present, mounting something if you need to mount something can be done in C code. So effectively I could tweak ffmpeg to do that directly. Yeah, true. And it would save the time like executing busybox.sh and things like that. It would save a lot of time too, sometime at least. About the disable tracing, it takes a 550 milliseconds. It sounds a very big part, I think. So why does the disable tracing take so much time? Which one, sorry? Disable tracing. Oh yeah, effectively, that's quite surprising. Well, I guess there's a lot of overhead in that case. I should investigate that probably in deeper detail, but that's what I observed. Yeah, it's surprising that in a production DevConfig, you have this option that really slows down boot time significantly. So that's something worth investigating and maybe change the DevConfig for that CPU family. Unfortunately, I think we're out of time. Yes. Thank you. Clear.


Where's that attendance sheet right now? Right up in front if anybody hasn't signed in, please take care of that. And just like to review the public input guidelines, please limit your suggestions or comments to three minutes or less. Comments and suggestions will need to pertain to the agenda item under discussion. Personal criticism of other individuals is out of order. The Planning Commission will receive citizen input and does not respond or debate. And after the first speaker is at the podium, the next person wishing to speak may sit at the chair next to the podium and be on deck to speak next. And then when you begin, please, your presentation, please state your name and address for the record. Thank you. Next item is 2.1, which is an approval of the Planning Commission minutes from February 24th of 2015. Take a motion to approve. Second. And after that motion in support, is there any discussion on those minutes? Seeing none, all those that are in favor, please signify by saying aye. Aye. Opposed? Motion passes. Next item is a conditional use and variance application by Wells Fargo to install a new monument sign at 2802 Calumet Drive. Steve? All right. Roger Osterman is here from RLO sign representing Wells Fargo. At the last meeting, we tabled this matter because there was some information we needed to gather with Ryan City Engineering with regards to the improvements to Calumet Drive set for 2016 or 2017. So we just wanted to make sure that the location of the proposed sign was going to be outside of the right-of-way area or the areas that the city needed to acquire for a part of that project. And based on meeting with Roger and Ryan at the site, it looks like that's going to happen. So what they're looking at is installing a new monument sign. It's approximately 48 square feet, 8 feet high. It's a double-sided illuminated monument sign. And the reason why they're making the changes because Wells Fargo is updating their logo nationwide. So the applicant is requesting a variance to install a sign with an 11-foot setback to the property line instead of the 12. And staff was recommending approval of the proposed sign permit provided they meet the following conditions. Just obtain the necessary sign permits meeting the lighting requirements. They meet the 11-foot setback and the maximum height of the sign is 8 feet. Is there any public input on this item? Seeing none, Roger, would you like to add anything to the presentation? Please use your mic. Yeah. We met at the site. We determined the grade works. The distance from the road works with the new construction. And it allows us to leave the sign in the same location. If we had to back it up at all any further, we'd end up in the driveway, which would end up with black top work and everything else. So this works for us and it works for the new road construction. So it should be good for everybody. Okay. Thank you. Commissioner, is there any questions? Go ahead, Mr. Alderman Bellinger. I'll just make a motion to approve with staff recommendations. Second. Thank you for that motion and support. One last call for discussion. Seeing none. All those in favor, please signify by saying aye. Aye. Opposed? Chair votes aye. Motion passes. All right. Thank you. Thank you, Roger. Item 2.3 is Geo number 47 of 1415 amending the city of Sheboygan's comprehensive plan, future land use maps to change the land use classification of the property located at 3306 Seaman Avenue from public parks in open space to institutional and community facilities classification. Steve, we'll turn it over to you. All right. Sounds good. Thanks, Mayor. The plan commission's aware of our healthcare is proposing to purchase the Field of Dreams property from the Sheboygan area school district in order to construct and operate a new medical and office facilities from 3306 Seaman Avenue. At that February 24th meeting, the plan commission recommended approval of the proposed rezone from suburban residential to suburban office with two conditions and those conditions included that the comprehensive plan be appropriately amended and the applicant shall take ownership by December 31st of 2016. Today, the plan commission is going to be reviewing the proposed amendment to the ComPlan, the future land use maps of the comprehensive plan to convert the Field of Dreams property from public parks in open space to institutional and community facilities and based on the recommendation or based on the action that took place by the plan commission at the last meeting, staff is recommending that the plan commission make a recommendation to amend the ComPlan future land use maps from public parks in open space to institutional and community facilities. So I can answer any questions. Thank you very much. Is there any public input on that item? Would you like to step up to the podium? And again, three minutes and then your comments should relate to this item and the change of use in the land use plan. And whoever would like to go next can just take a seat next to her and be on deck for the next presentation. Thank you. Okay. Your name and address? Oh, yeah. Debbie Demoulin and my address is 1704 North 35th Street, Sheboyga, Wisconsin. Okay. And you'll have three minutes. Okay. So currently Memorial Hospital is on 7th Street, a residential neighborhood on, okay, yeah. And on Tuesday, March 3rd, Dave Graydmer held a meeting for the current 7th Street neighbors to allay their concerns. At this meeting, the neighbors' comments, complaints from congested streets surrounding the hospital from unsightly dumpsters and an unesthetic view to the exasperatingly noisy truck deliveries starting at 5 a.m. with a frequency of nine trucks daily. This is proof that a medical facility located in a residential neighborhood is poorly tolerated. Neighbors don't sleep well and cannot easily enter or leave their homes. Neighbors were concerned that Aurora would abandon the 7th Street hospital leaving an empty building. One neighbor said, when you transfer the hospital to the Field of Dreams, will you run a highway through Maywood? Graydmer replied, no, Maywood is forever. But the man said the Field of Dreams was supposed to be forever as well. Graydmer only committed to staying at the Memorial site for 10 years. He repeated that Aurora was landlocked at the current location. Aurora's destination for the hospital is the Field of Dreams because their sketch places the surgery center on the north side leaving a place for a hospital on the south side of the Field of Dreams. Graydmer said that Aurora still has to pass by DNR approval and city zoning, but he told the neighbors that Aurora was committed to staying in Sheboygan that they would find another site in the city if their current project doesn't pass DNR stipulations and city ordinances. Therefore the city will have the same tax revenues regardless of where Aurora builds. Aurora does not intend to leave the city. So my question is why lose a popular community park when the current Aurora neighbors have numerous complaints about the hospital's inevitable activity, especially when there is no need. Let's not make the same mistake twice of placing a hospital in a residential area. The Field of Dreams was built with a 92,000 DNR grant on the condition that it stay a recreational facility forever. Transferring it requires the conversion be necessary and meet or exceed its current recreational amenities. To make this truly a win-win-win, let's find Aurora a south side property commercially zoned that does not take an existing recreational facility to provide medical facilities on the south side as well. This way we will keep all of our existing green space and still retain Aurora's services balancing the medical and recreational needs for both the north and south sides of Sheboygan. Thank you. Thank you for those comments. My name is Matt Brush. I live at 3320 Geely Avenue. Please go ahead Matt. I live in the neighborhood where the proposed medical facility is going to occur. And I just want to tell you that I'm really excited. I'm really excited because I heard all across that the parks in the city of Sheboygan are available for commercial properties. There's some landlocked businesses in the commercial areas that would like to expand as well. And I would just like to say businesses like Wigwam, businesses like Jail of French used to be, they'd like to expand it in nearby parks. They're interested in the Volrath property. They're expanding into areas like Cole Park. They'd be like to expand into Moose Park and Park, Roosevelt Park. These are neighborhoods where you all live. And I know where most of you live and this is parks that are your neighborhood parks. So these businesses are also excited about this. And this spot rezoning that we're proposing doing to accommodate Aurora seems really nice on the face of it. We get to develop a south side property with money that they're providing. And they're a multi-million slash billion-dollar business and they have money to throw around. We have property that we just inherited. We have three properties that are paid for. They're clear. We own the Field of Dreams as taxpayers. We own the property where the Mung Gardens are. And we now own the Boots and Property, which was a wonderful gift that we received from that family. We own these. We can develop them as we need them. We can improve the sites that we currently own. If there's problems with the Field of Dreams Park, the soccer people had said, it would be much easier to spend money to improve that park and to provide better soccer fields for the kids that enjoy that park. Again, we own it. We don't have to make any accommodations. It would be very inexpensive. It's like buying a used car. It's a good property. We've enjoyed it. We could pass it down for another generation. It's only been there for one generation. Looking for land. Aurora's been looking for land for quite a while. According to them, they've been looking for expansion for 20 years. They need to make a move. This opportunity just presented itself as when the Boots and Property became available. If they've been looking for 20 years, we've been expanding to the north. The area up by Menards. They've missed the Kmart property. There's probably been an opportunity at the mall area. There's been land available. They haven't made a move. This is an opportunity they've jumped on because it requires the least amount of financial impact on them. They don't have to build an off-ramp. They don't have to build over a railroad trestle. They don't have to do any infrastructures. They don't even have to cut down any trees. It's easy for them to do. It's easy to wreck a neighborhood by putting some business or a corporation in it. There's a reason we have zoning. We could put whatever business we wanted over there. There's been many opportunities for other businesses to grow and develop into our neighborhood. But it's been designated a park for a good reason. I'm not sure why there's a sudden change of heart and why that park isn't considered to be preserved like the rest of them. But I propose that if we start with this, we should look at Maywood. Excuse me. Excuse me.  I three are up? Yep. Can I just finish my thought, please? Just wrap up, please. Okay. I propose that we look at developing Maywood in the other parks. The financial impact would be great for this city. Thank you. Thank you very much. Is there anyone else wishing to speak? Please come forward. Anyone else can take the on-deck circle? My name is Linda Schemann. I'm from 226 North 27th Street, Sheboygan. Linda, you'll have three minutes. The history of Sheboygan is that industry and commercial buildings were kept in a separate part of the city. What are we doing now? What we are doing now is a major departure from that history. Is that the legacy that we want? I read a study that the more hospital capacity in an area, the higher the cost of health care. Aurora already has more than two clinics in the hospital with other agencies serving our area as well. With all the property Aurora now has, we can speculate that they intend not to just replace but to add on. That is what Aurora is doing all across the state. Again, do you want your legacy to be that? Because of you, the people in Sheboygan, in the Sheboygan area will have higher medical costs. Higher medical costs could eat up any benefit of jobs created by this deal. How many jobs will be added? A few doctors, because doctors will be shifted from existing facilities, both Aurora and others. That means the new jobs will be mostly custodial and in the records departments. Have you analyzed the wage scales of these jobs and the working conditions? Will they pay these workers enough so they don't have to apply for government assistance? How much property tax is the city given from all the Aurora facilities right now, especially the clinic and the hospital? Can citizens get these records to verify that we really are receiving tax revenue? One of the enticements cities generally offer to businesses to build in the city are secret tax breaks. With all the secrecy that has gone on with this campaign, how do we know that we are not giving Aurora tax breaks? What free services will Aurora be required to do for free? When I researched other new hospital constructions, everyone had a provision that they had to provide some free service to the community. Is that what Aurora is going to do here? What governmental agency is overseeing the south side property, not the boots and farm, but the other one they admitted that they had near river tracks or whatever? We have the constitutional right of due process of law. That means we can request more time for investigation into these arrangements. Thank you. Thank you very much. Susan Sunquist, 2338 West Shelly Court. Welcome. You'll have three minutes then. Thank you. I would like to say that many other people have a lot more information than I do. They've researched it, spent a great deal amount of time, and I applaud those people who are trying to save the field of dreams for the community of Shaboy again. I am speaking from a personal aspect. I am a special education teacher in the Shaboy Ghaneri School District. In our fifth grade social studies unit that I'm preparing for now, we are studying the trail of tears. My special education students understand how the colonists did what was best they thought for this country by moving the Indians out, took their green space, took their land, took what they appreciated, took what they worked hard for. I can't help but think that what is going on here is extremely similar. What is best for the community? Is that really another hospital next to another hospital, next to a surgery center, next to another surgery center? Is that really helping the city of Shaboy Ghan, the community? Aurora has talked about making services available for the students, I just came from teaching, for the citizens that it be easily accessible. Easily accessible does not put all facilities in one area. You distribute the facilities among the city. Aurora talks about patients being able to take the bus. If you're having surgery, you do not take a bus to have surgery and hop back on the bus to get back home. You have to make arrangements for someone to drive you there. So the argument that it's available for students or for citizens, I'm sorry, on a bus does not make sense to me. I ask that you consider, do the needs of Aurora outweigh the needs of the children and the citizens of Shaboy Ghan? Like the colonists, thought that their needs were more important than the Indians. Please consider that when you make your decision. I appreciate your time. Thank you. Thank you. Hi, I'm Melissa Brush from 3320 Geely Avenue and I just want to simply say that I don't think Mrs. Bootson's plan of giving us the land and having more screen space was to destroy another park. That was absolutely not her intention. Her intention was to give us more green space and have more park space for our community. So why are we destroying a park that thousands of people use every single year for numerous things? And I know Mr. Jones that they don't use them for football. I realize you're on the board for that, but I know my kids usually at Kiwanis, or sometimes if it's not too wet north or south high. But why are we destroying a park? That was not her intention. She was giving us that space to add to our green space and our parks and our community, not destroy it. Please, let Aurora build somewhere else and preserve our parks. Thank you. Thank you. Anyone else wishing to speak? Please go ahead, Scott. I'm going to use the microphone here. That'll be fine. Three minutes, Scott. Scott Levendusky, 2201 Uri Avenue. And I'm not one of the neighbors of the field of dreams, but I am a citizen of Sheboygan. And unlike many of the Aurora people that I've spoken, I do live in the city of Sheboygan. And I want to keep the field of dreams for the kids the way it is now. And I have some questions that probably nobody has asked before. But how much is this really going to cost the city taxpayers? We were already told 800,000 would have to be added about what Aurora is going to pay. But there's a lot of stuff that has not been mentioned that's going to be required. At the meeting at Cooper School a couple weeks ago, somebody from Aurora said that we're going to need traffic signals on Taylor Drive at Dealey Avenue or Samon Avenue, probably most likely both, and maybe around about. Who's going to pay for that? It's going to be city taxpayers. On the south end of the field across the street where they want to move everything, there's contaminated soil from the city dumping stuff there. I'm sure the school district and Aurora are going to say the city's got to clean up that contaminated soil. On the south side by the property where they want to move everything, for three-tenths of a mile there's no sidewalk. So again, that's going to be another city expense that nobody's brought up. And I have heard Aurora say that this is the only possible site they can build on. Yet I have not heard them mention why that is the only site the other ones do not meet their needs. And how do we know that everything that is promised to the kids in the new sections of the parks are actually going to be given to the kids? In New York City, they have a park in the middle of the city named Central Park. With the price of land in New York, look how much money the city of New York could make by selling that land and turn it into development for something. We're asking the same thing by Aurora's.  We just want the land. Thank you. Thank you for those comments. Is there anyone else wishing to speak? David, please step forward. Dave Grabener, 3618 North 6th Street. Three minutes, David. Thank you. There were several questions answered and I asked. I wanted to make sure we're answering those questions. I'll start with... First question was around us building a hospital. We are not. There's no plans to build a hospital. There's no plans to build a hospital. There's no plans to build a hospital. What I think the neighbor discussion that you heard was something that several alderman talked to us about because they heard concerns from their neighbors around what's the plan at the existing hospital. So I reached out to our neighbors as I've done periodically since I've been the president of the hospital. And so the discussion with the neighbors really is to help them understand what the project is, help them be reassured that we're not going to lead a white elephant in their neighborhood. That's the concern that we heard. And so I think the 10 days was taken out... 10 years was taken out of context because what I talked about is the investment that we're making in our current facility. We've spending $11 million since Aurora made the approval for the outpatient facility. We've invested $11 million in replacing our windows in adding fire protection, assuring that we're updating our current facilities. Just signed off today on another $1.2 million in a new call. Light system for our nursing staff, and we're going to continue to make investment in that facility. And we're also going to continue to work with the neighbors as we've reached out to the field of doing neighbors to make sure that we understand the issues. Part of the concern that was raised at the meeting is because we've had so much construction, there's a lot of debris, and it's the management of that that we wanted to hear how we were doing and what we could improve that. There were also some questions related to the jobs and the number of jobs, so I want to make sure you have the facts. We're talking 37 new jobs, seven of those are physician. The added tax base to the annual salary dollars is $5.2 million annually to our community with the increase that we're making. We pay $403,000 in taxes. The clinic does that as far as the nonprofit we're estimating another $200,000 associated with this project. Someone mentioned the east side contamination as part of the condition of sale and with the west property we are committed to work with the school to assure that we're meeting all needs that we have around the fields and making sure that's safe. So that's part of the condition of the sale. And then I think regarding adding or subtracting the number of parkland to our city, as they've mentioned, the school has mentioned we're adding 40 total. So the Butts and Farm, the relocation of the east parcel property and additional use of that is adding 40 acres to the city's use. I don't know if there's other questions that Planning Commission has. We'll get to that later than David. Thank you for that presentation. Is there anyone else wishing to speak? Okay, then we'll return the conversation to the commissioners. Commissioners, did you have any questions or comments or motions? Alderman Bellinger. I move to approve the motion with staff recommendations. Subject of staff recommendations. I'll second it. Thank you for that motion and support. That motion is before us. Is there any other discussion? Seeing none, all those in favor, please signify by saying aye. Aye. Aye. Opposed? Motion passes. Next we'll go on to item 2.4, which is RO 255 of 1415, submitting a communication from CVS store number 10549, Sheboygan, Wisconsin, being assigned a Notarized Certified Survey Map. Steve? Planning Commission remembers in June we approved the conditional use permit for CVS to construct a new pharmacy at Erie in 14th. A couple of conditions required them to combine all of the 10 lots into one lot, and then they were also dedicating a portion of property at the southeast corner along 14th Street near for a dedicated right hand turn lane. The CSM is basically creating the one lot and dedicating that property to the city. So anytime the city is receiving a dedication of property, it's got to come to the Planning Commission and Council to receive that dedication. So staff is recommending approval of the CSM. Thank you. Is there any questions or motions? Move to approve. Second. There's a motion and a second on the floor. Any questions on the motion? Discussion? Seeing none, all those in favor, please signify by saying aye. Opposed? Motion passes. Next is adjournment. Move to adjourn. Thank you for that motion and support. All those in favor of adjourning, please signify by saying aye. Aye. We stand adjourned. Thank you very much.