Hello and welcome back to the craft box.  I don't only play this game. Okay. I don't want to get it wrong that I only play this game I play other games as well. Anyway, hi Rose. How are you? Actually? I know how you are. I Already know how you are. I don't even know how you are. I already know this um But yeah, let us do some badges and I have a story for your look today Was it one-on-one is the only game that exists? That's debatable. Honestly, I feel like that's false, but you never know. Maybe it is You know exactly how I am. Yeah. Yeah, I do. I do unfortunately Hold up, right. Um, right story time. It's so I Need to I need to get to a badge place first We need to do badge first. Hold up. We need to get we need to get into the mindset of badges Oh my gosh, my mount is done. Okay. Next one next one next box plane. I love this mount so much. Honestly, it's great Can I please game please please am I am I Am I lagging or something like what the heck am I lagging geez? I have no idea. What's it up is my stream, okay? My stream is fine. Why is lag was why why anyway?  What is this lag? Dude. Oh my gosh. Oh My gosh, it's worse than normal like I'm usually not one to complain about lag because I have a lot of lag But like this no, no, no, no, no, especially when I'm going in and out of dungeons. Holy moly. That is not good enough not Good enough at all. Is it gonna do it again? If I leave it here. Is it gonna like go up?  Welcome. Give you the story. Okay. I'm just I just need to get into the badge first and then I'll give you a story Because otherwise I'm going to be complaining the whole time What is this? Oh my gosh, I just want to play whiz and tell story game Is it that much to ask for seriously? Holy moly Why is whiz so bad today? Boo stalling. I'm a stick. Hi. I'm just waiting for everybody to get here. That's what I'm doing obviously. Oh My gosh, oh My gosh, what is this? Is anybody else having these issues like? Because that is pain that is pain Who are we defeating in caramel today? Let us to beat some that's the wrong one caramel Okay, we'll do it then we'll get this one over and done with ideas Stolen to make up a story. No, I know I have a story. I do have a story guys Don't worry. It happened to me today and yesterday at work. I am seriously. What the heck? What the heck? This is pain this is pain Okay, so story it happened at work Just just what one is today and one is yesterday for those of you who don't know where are they for it? Oh My gosh, oh My gosh, this is terrible. Hold up. I need to sort out my deck. I'm really stalling now. I can't play whiz This is unfair. I don't I don't know how to play whiz. We're moving we're moving to this We're not playing whiz whiz is broken. That's cool. Anyway, story time story time while I restart whiz Yesterday at work For those of you who don't know I should probably preface this for those of you don't know I work as an audio visual technician Which means I do things and stuff Pretty self-explanatory right any questions about that? No, okay moving on. I I was that work We were working at a hotel in the center of the city and She probably record this so we were working at a hotel in the center of the city and We were doing a setup for today's event, which was a live stream a sound system and Livestream sound system projector Just just pretty much a pretty standard setup for us a pretty standard What is your real name Brent? My real name is Brent. That is not a joke because who would joke about having the name Brent That's weird. That's weird That's such an obscure name that no one would joke about having the name Brent But yeah, anyway, so I was setting up I was sitting up at the hotel that I was at today and yesterday to set up this event today and and we were sitting up perfectly fine and Like this is with it the lady who is the events organizer at the hotel comes comes into us She's like she's absolutely like over it over the day And this is this was at like what? 10 o'clock in the morning so like already at 10 o'clock. She was over the day. She's like I can't I just can't this is not cool I can't anymore I can't can't So she She like comes into the room that we're sitting up and she's like I am so over this day I just I just simply cannot honestly It has been a terrible day. Basically. She had a she had an event that was meant to be um That was like penciled in but that day yesterday But they didn't like confirm or anything because one of the the lady who was organizing it got sick so they obviously didn't like book it in and They didn't have any catering for them obviously because it was not meant Sorry, I'm playing was again And like I can't even add spells Like it's just Why why is this like this? Oh my gosh? Yikes, this is pain This is pain. Oh Man um, but anyway What is this lag holy moly, I've never seen it this bad. I have never seen it this bad. Holy moly. Anyway, um We we were sitting up. She's like it's been a day. It's been honestly a day I'm over today. I want to just go home and cry. I'm like, you know what? I feel you sometimes I feel like that as well and she's like you don't even know the half of it basically She had a group I've already explained this she had a group that was meant to come in but wasn't actually scheduled to come in It was a government organization thing Of like 60 people they they weren't actually booked in they were they were they had been like talked to But they hadn't actually like booked in because the lady got sick to actually book it in and she didn't like Delegate it to someone else to like book in the hotel So anyway, they come in at like eight o'clock in the morning 60 people into the hotel lobby and they're like so we're here for a conference today and The events organizer is like you're here for the walk today. There's no conference today So she she starts stressing out because they're like crowding up the whole like lobby of the hotel And this isn't exactly like like a rundown sort of hotel This is this is like a decent hotel like a really decent hotel here in Christchurch So they're like just standing in the middle of the lobby just like taking up space when it's like peak time for people to come and check in So she's like we don't have you booked But if you go go like elsewhere go outside and wait they just stay in the lobby They just stand lobby they're cramming it up and she's like I've never had this much anxiety That something bad is gonna happen then I had today So anyway, that was the start of the day for her this was before we got here So she was already over it and she's like this day is cursed for he's going backwards. Did you see him? He's going backwards. He was going backwards This this this whole game is cursed. I swear Um But anyway, it's just like today's coast today is just absolutely close and you're here on the wrong day So anyway, we start our setup About 15 20 minutes and we've got the sound system all hooked up and stuff It was it was going wait and then all of a sudden we hear smoke wait. No, we smell smoke We don't hear smoke. We smell smoke and we're like smoke. Hmm. That's interesting. Where's that coming from?   So we're like, oh shoot. It's smoking. So we quickly unplug it and Honestly, I'm surprised the fire alarms didn't go off because that was it was quite smoky. It was quite smoky. It was bad So like we unplug it and we like usher it through the whole back halls of the hotel to get it out of the building So it doesn't set off the smoke alarms Meanwhile, we're trying to find the key to open up the windows because we're on the second floor and the windows are all locked for safety reasons Because apparently it's bad if people just like open the window and fall out or something. I don't know It's a bit strange like who's gonna fall out of a window purposefully. I don't know Especially if a second floor like if you're doing anything like that third floor is better, you know a second floor is just terrible Um But yeah, so anyway, we rush around Trying to find the key who has the key to open it and the maintenance man comes in later And he's like man, it smells in here You guys need to open the window and we're like, we know do you have the key and he's like, yeah Let me go let me go make a copy for the event Lady here and then and then we'll give her one so she can open the windows whenever she wants and she's like Oh my gosh, this is so nice. I can open the windows to jump out all the time. This is great So that was fun. So you open up the windows starts to clear Starts to clear and we're like, okay, that's fine You know one one one makes up, you know, it happens it happens some things just explode and smoke or something You know, that's that's fine Fast forward to today fast forward to today today the day of the conference It's super simple Livestream I do that all the time. Okay, like obviously, you know, no, no, no problem. I can I can deal with the live stream That's simple. Okay, you know, I do worse live streams here at home than I do at work because like we've actually got like a dedicated like like a massive like rack built for like live streams So like I know I know what I'm doing. I don't want to put my own home, but I know what I'm doing So anyway, we get in I'm like, yeah, this will be easy super duper easy We set up everything we do the mic checks. Everything's looking great. The camera looks perfect apart from the fact that it is too dark So that's cool. So we go and ask hey, can we turn on some more lights in the room? Because there's like on the ceiling of the room There's like the wheel of like lights that like have a little bit of glass over them But like they're like on the ceiling so like the light class is here and the light is just above Whole bunch of them just never turn on. We didn't know this until today until we needed them So we we got to the event organizer and she's like I'm pretty sure all of the lights rock and we're like, well, there's some lights that are not on Could you possibly turn those ones on and she's like There are lights that aren't on I've never noticed that before because they've never turned on ever and we're like, oh That's cool. That's fantastic. I I love that. It's great Um, so she's like I couldn't see what I can do So she goes over to the light switch. She turns off all of the lights and turns them back on again And she's like, well, that's all the lights on Meanwhile the lights still not on so um, we've we got like a room that's like half of it So the camera is super like super dark because the projector is way too bright and the camera has like also Also, um The way that the camera works with the exposure is that it automatically detects What's the brightest thing and ducks it down even if you turn it onto manual It still does that which is weird and crazy and I hate it so much But anyway, so it didn't solve the problem So we're like, okay, so now we need to move the lectern so we can actually see um The people at lectern because it's kind of important to see people So, um We we moved the lectern Into a spot that's not too bad. The client comes in and she's like, oh, that's perfect. We wanted the lectern there anyway So we're like sweet. Okay problem solved looks better not next to the projector screen Easy peasy. We're all good. We're all good. So we start the We start the conference going well going perfectly well 20 minutes in The monitor that I use to like um display like um, I don't I don't know I don't know how to like put this into an easy term but like On on our system that we use we've got like a multi view So you can like switch between like different cameras and stuff and know what's coming up next or what's like ready to go and what is live um And that one just turns off completely So i'm like that's interesting Luckily that was the monitor for the room because we have to have two switches because we were switching the projector in the room And then also the live stream, which is in the different one Uh a different switcher. So that's fine. I'm like problem solved. I I um look at the live stream Like menu and it's like unknown status unknown. I'm like, why is it unknown?  Um The internet is gone So i'm like, oh Absolutely perfect.  Okay, we're getting paid for this You know, I can't exactly screw this up um So i'm like, huh, what am I gonna do? So I I go to the back of the rack and like what's wrong because clearly something's come unplugged That's obviously got to be it.  It doesn't work because it was shining. It was light up. It was perfectly fine But it wasn't getting in the internet So that was cool and great and i'm like, man, this is perfect. This is honestly perfect I can't expect anything better from the situation so I I quickly like get on the phone I I hid out of the room and I go and talk to the conference. Um Manager who's at the at the hotel and i'm like, hey, um Please don't panic and she starts panicking immediately I'm like we don't have any internet anymore. Um, and the live stream is currently down Could I have access to the room where to your server room so I can see what has happened? So I go into the server room perfectly fine working like normal, you know All of the lights blink and lovely green colors amazing. No problem there So i'm like that's interesting. So I um I get on the phone to my boss like hey Um, got a slight problem Uh, we we don't have any internet and the screen is flickering on and off And he's like, hmm that's strange because we've only just built this rack and all of these parts are pretty much new And we've tested it a lot What what's wrong So he he comes in mind you this is in the center of town. Um our office is like Like 15 20 minutes away. This was that nine o'clock in the morning. This is like peak rush hour traffic It takes them like nearly 45 minutes to get in Pain so anyway, I I managed to I managed to hook um things keep getting worse.  That's for sure. Um, so anyway, I'm like well I can't just sit here for the next 45 50 minutes until their break Without a live stream. Okay. So I somehow connect.  So we use um, a thing called like an ATEM mini extreme switcher um To switch our live stream today um, which is like a pretty like A pretty professional sort of PC gear But it comes with a like a usb c output that you can connect to your computer. Usually it's for like, um For like updating it and stuff updating the software on it But you can also use it to get a webcam feed out of it. So I'm like, okay Let me rush to where we store where we were keeping The box which has the um usb cable in it and go and get the usb cable I bet you're thinking that there was no usb cable in there. Luckily there was there was a usb cable in there Don't worry. So I plug it in I'm like perfect fine. Let's crack up. Obvious boom problem solved Easy clap will be the best the best live stream ever. So I crack up.  Obvious. I'm like, okay. So let's let's add a new source I'm like, okay, we need to go into video capture device and we add one I'm like ATEM and then I go like, okay, and then I select the the source that I need And um, it doesn't show up I'm like, why does it not show up? Why why is it not showing up? What the heck? So I so I unplug it and plug it back in because you know, obviously obviously it doesn't work the first time you need to do it again Still doesn't show up. So I'm like, hmm. Okay That's interesting So I go I go to the other device that we were using which we used to actually stream the web presenter Which is basically a wee little box about This big that um gets an ethernet line in and just sends out at live stream problems of easy peasy You just give it feed in feed out Good thing about that is you can connect a webcam to it. However, the the webcam output thing Um was the usbc cable Now the laptop that I was using only had one usbc input that was for the power So I'm like, hmm. Okay Could I theoretically possibly maybe Not have it powered for the next 30 minutes while live streaming So I try it. I'm like, okay. What's the worst that could happen? Plug it in Doesn't recognize either I'm like, why why do you not recognize this? This should work. This should work So I'm I'm I'm like ripping out my hair at this point.  That's so true. Oh my gosh. Um, but yeah, so like I'm I'm stressing I'm really stressing. I don't stress much, but I was stressing hard Um, so I'm like, okay, that's not gonna work. We're gonna scrap that. I put the cable back away I'm like, okay. We got to try with that. We've got to try with the the other thing again So I plug in the usbc to the usb A whatever cable again try and work it again open up obis I restart it. I restart it. We wanted to do updates, but I'm like, no, we can't do updates We don't got time for updates. So I plug it in. I'm like, I go I I need to explain I go here. I'm like, uh new new video capture device. I'm like ate him, okay And then and then I go into properties and I make managed to make it work. It works. It's amazing It's amazing. It works. The problem solved problem solved However, I wasn't getting any audio, which is kind of bad because if you're using a live stream, you know It's it's not great if all of a sudden I know you couldn't hear what I was saying. That was on purpose. Imagine. You couldn't hear anything I was talking, but you couldn't hear anything. That's not good Especially for a live stream. That's bad This could be a whole sitcom episode. I know I know, but this is real. This is real um So, yeah, we're like Perfect. Okay Great. How am I going to do this? So I go into I go into audio devices because for those of you who don't know Hold up. I need to show you for those of you don't know you go add new Audio um audio input device basically a microphone. Obviously. I've got some already. I know how to do this I use obvious all the time. So I add one Nothing coming through just just nothing nothing coming through at all. I'm like, oh, that's fun The one the one name black magic because it's black magic who is the company that make the thing Not coming through nothing at all even though there's plenty of stuff theoretically coming through it. I'm like So does it not come through usb?  Nothing Just just nothing nothing nothing at all from the laptop's microphone It was working five minutes ago, but now it's not Pain pain is all I'm thinking all I'm thinking is pain. How is this possible? What story is this is the story of my day for the last two days? You've missed so much and I'm not going to recap it at all Um, it is pain. I can tell you that um Basically my pain painful experience of the last two days So anyway, I'm like, okay What what are we doing? What are we doing? Like what how can I do this? So I go to this the third option. There's three options There was the there was the black magic option, which wasn't working There was the the laptop's microphone wasn't working and then there's the thing called microphone array And I'm like, what the heck is that? What does that even mean? I've never seen that in my life ever Um It'll be going up on my second channel. Actually, I might make a main channel video out of this just for fun I don't know but um it'll go up on my second channel later. Um I'm like, what the hell is this like I've never seen this option before so I put it on and it works I'm like, okay as long as it's working. That's fine. The problem is it sounds so bad Like it keeps dropping out It's like if you're going through a tunnel and you're trying to talk to someone on the phone and it like cuts out All the time and it's really muffled.  So I start the live stream I start the live stream during this by the way during this the um The lady the lady who's our client the event organizer for the event itself Not for the hotel for the event Has emailed me like six times in like five minutes saying um the stream has died the streams died Why can't people see the live stream? Please help help help And then I think the last one she said was help in all caps with three exclamation marks And I'm like, oh gosh, please my phone's blowing up Um while I'm trying to fix this issue then she comes over to me She comes over to me. She runs over during the middle of someone's speech I have absolutely no idea what's happening in the room. I hope that everything's fine because I'm not focused on that at all um So so she comes running over to me.  She's a nice lady Nothing against it. She runs over to me and she's like, um, I've had reports from email that the live stream is down Do you know anything about this? I'm like, yes. Yes. Yes. I'm working on it working on working on it and and I somehow like managed to get her to go away because she was flustering me even more and um She's in five minutes later since still not working and then I'm like So I managed to get it started again and then and then I know the audio is bad I know it I go out and call my boss and say right. I've started the stream up again audio terrible What can I what what do we need to do to fix this problem? And he's like, I'm on my way I've got another I think I know what the issue is. I think it's our 12 volt um power supply that we put in um that's supplying the monitor the ethernet switch the router and also the 4g router that we have which is the backup in case our internet died theoretically the way that we've got it set up is that if if our internet dies From the venue itself it goes to a 4g router Which is basically like you think your phone has like data that you can use from like 4g and stuff That's what it is. Basically. It's basically a phone router You slap a sim card and problem solved easy peasy. However, that was also not working So we couldn't even get internet from that So I'm streaming off wi-fi, which don't do that not recommended at all Um, I mean I used to do that for these streams, but like if you're doing it in professional settings never do that I'm surprised the wi-fi stood up so well. I was honestly surprised by that but anyway So I'm I'm like on the phone to my boss. What what do we do? And he's like, I know what it is. We need to get the 12 volt that battery supply sorted So he's coming with a whole bunch of extra like plugs and cables and things and Mabobs and all of those sort of stuff, but he's like 45 minutes away because traffic Great, he's like wins the wins the break And I'm like it's technically at 10 So it's like, okay, I'll be there. I'll be there before 10 and we'll get it sorted Go tell the client that I will fix it in the break. I'm like, okay, bet So I go I then I then as soon as I finish that call the event lady at the hotel She comes up to me and says is everything fixed is it okay? Is it bad?  I'm not kidding. She's nearly in tears. She's had an absolute nightmare of a week um It's only Thursday She's in tears basically and I'm like, no, but I'm working on it. It'll be fine It'll perfectly fine. Calm down. Don't worry. It's all okay. And she's like, I'm sorry. I've dragged you into this You've been cursed by me I'm like, I don't think it's your fault, but like, you know, I'm sure we'll we'll get through it. It'll be fine She's so nice as well. She's so nice. I felt so bad I felt so bad, but I can't be I need to be like I just need to reassure her because I'm I'm feeling exactly the same, man um, but anyway This is such a ridiculous story Man, this is not scripted um So, so yeah I I finally figure out how to get the issue fixed with the audio. Okay So I start google them because google solves every single problem you need and more However, all of the things to solve this problem were youtube videos and I'm like, I can't watch a youtube video On my phone in the middle of a conference That's way too much noise. So I'm like, okay. I have to do it myself. I have to become google So I became google I became google. Okay. I I plugged in things and I fiddled with dude ads and did all of the things and somehow It just started working, you know, it just started working out of nowhere the audio input that I Swear I had in for the first time just started working problem solved. Everything was great All good then comes along the break My boss comes in and he's like right I know what the issue is. We must fix it now. He's a man on a mission He's furious. He's fuming. His steam is coming out of his ears He's like, how can we have two gear pieces failing in one one job? This is not good enough I'm like, I know I know but it's not my fault. I didn't do anything. It's not me And he's like, I know So we fix it problem solved easy peasy all good problem solved we Just before the break ends everything's working. We've got the stream back on to the normal thing Good good shout everything is great I go and get some morning tea because free food who's going to pass up free food not me either Don't even pass up free food. That's the best part. Um So anyway, like problems all sort of won't happen again easy peasy So we're in the next session the session goes from about 10 15 till um Till 12 15 12 30 finished at 12 30 about basic I think it was like 12 25 or something and then actually finish but just just pretend this is this is between 10 10 15 and 12 30 This afternoon this morning this afternoon. So So it's going well.  So for example, I have I have a stream deck Yeah, this is it Sean's uh, I can't turn it around enough my I can USB not quite long enough I have a whole bunch of buttons here which I can like press and it changes things So if I press that that's the that's my face if I press this And this is this is my face. This is the gameplay. So Well, I'm while I'm trying to switch the um stream to make sure that people on the stream can see things all of a sudden as soon as I switch from um The camera to the slides it starts like absolutely tanking um How can I easily explain this? um So things like youtube Hold up. Let me just pick a card when it will load Game's being really broken by the way today in case anyone's just joining. Welcome those of you who are joining I'm not really reading shade at the moment. I apologize um, anyway things like youtube we're not streaming on youtube, but Pretend that it's like youtube they'll give us things like stream is healthy, you know that no That's how you know that um your stream is doing. Okay. So like generally the levels are green orange means You've got issues red means very unstable. Please fix and if there's nothing there that means that the stream is down Like completely down Um, and that's not good. That's not good. You don't want that um So all of a sudden There's this thing called bit rate Which is a funny wee little thing. That's basically how much like Data you're sending out to the platform Um, generally if you're doing a 1080p stream You want a decently high ish bit rate usually around like 6 000 Ish is pretty decent For live streaming. That's what I use while I'm streaming to youtube usually around 6 6500 ish around there um If you're doing lower quality you can go lower, but generally around 6000 4000 to 6000 is kind of where you want to be set um, all of a sudden My uh, my wee little device that i'm using to stream off start sending like 17 17 000 And in the bit rate which is like ridiculous. That's like super duper high. That's like that's like higher than 4k quality And then all of a sudden it starts caching now This goes back to a previous a previous problem that we've had um Caching is an issue basically where the amount of data that is getting sent out of The computer or whatever you're streaming from does not match what The system is taking in so you've got more data in the in the storage of the or the memory of the device Than is being sent out, which is not good. If that gets too high, it'll crash the stream So you want it to be like a significantly Ideally don't want it to be high at all like you want it to be zero percent zero percent cache is the best because it means there's Everything that you're sending is going straight out. There's no delay nothing Um, and it's not like infinitely stacking the cache which will crash the stream So you don't want to do that all of a sudden we start getting cache numbers of like 80 percent Which is ridiculous because that's nearly a hundred and at a hundred percent the stream dies And i'm like why why is this doing that only when i change scenes? So i stopped not changing scenes. I'm like, okay. I'm just going to put it on as like The the picture or like the slides and then a wee little image of like the the room down the side And then all of a sudden the stream just dies I'm like, oh That's great. Luckily. I picked this up really really quickly. This wasn't an incident issue this time I have no idea what caused it like absolutely No clue at all because at that point our cache was down completely. I had gone down significantly by then I think it was at like two percent which shouldn't Shouldn't have any issues. Um, but the stream just dies just cuts off completely. I'm like Great That's fantastic. Absolutely tremendous So anyway, I try started again.  I'm about this Like actually scream So i'm like, okay.  Luckily it starts And then comes the issue The audio quality after I started that stream again all of a sudden There was just a wee little Like just in the background I'm like How how How how how does this happen because it wasn't there before So there's just this annoying beep and I can't unhear it like I if I hear something and like a piece I just cannot unhear it like I will be fixated on hearing that That's why I hate editing things Sometimes because like if I hear just one annoying little thing I'm fixated on that and if I hear it over and over and over again I will have to like completely redo the take or whatever, but like this is live I can't fix this. I don't know where this beep is coming from why it is started So I'm like, okay. Let me try and diagnose where it's coming from So I take out the headphones from the laptop that I'm monitoring the stream on I put them into the um to the switcher Because there's a headphone jack on there. I'm like, okay. So is it coming through here? So I put in the headphones listen. I unmute it. I listen I turn up the volume to max and all I hear I'm like, oh my gosh. Why is it beeping? So I send a text to my boss saying we have another problem Beeping it is happening and he's like, what do you mean beeping? So he he goes on to the live stream on this phone And he's like, I don't know what you're talking about. I can't hear a beep But I swear I'm hearing a beep like I can hear a beep. He's like I can hear static I'm like, I know I know the static. That's the least of my worries at the moment There's a beep and it is annoying And I don't know what's causing the beep either So I'm like, okay. We'll just finish off. Well, luckily luckily this was like 11 30 when I noticed this And there's only like an hour left slightly less So um So I'm like, okay, it's annoying, but I need to deal with it. I just need to just understand that it's there and deal with it But I just cannot I cannot and I'm like I'm like raking my brain What the heck is causing the beeping because I have no clue I have never heard this beep ever in my life ever Like it is just it's not even that loud like it's the most faintest beep ever But it's so annoying and it's just in the back of your mind and it just eats at you So I'm I just I just kind of lose it. I'm gone like insane here.  What is my life? And there's just still this beep. There's just annoying beep in the back of my headphones So bad anyway The stream finishes problem solved all is good And that is the end of I don't really have a satisfying ending because it's just the beep That I don't know and just the dropout that I just don't know I'm sorry. I don't have a satisfying ending, but that's kind of that's that's my last two days um That's my last two days. Um, welcome those of you who have joined us while I was saying um But yeah, um That that's just that's just been my last two days today and yesterday and honestly, I'm like I like came home and I screamed a little bit not gonna lie pain Pain anyway, hi river. How's it going? Um, isn't that like a stream dick?  Yeah, they are they are being really weird really really really weird Um, I try so hard to annoy annoying background noises But trying to annoy it makes me pay more attention. Agree completely agree completely agree That was adk you should probably get emotional compensation. I wish I wish dude I wish So yeah It's just been it's been a day. It's been an absolute day. That's for sure Man But yeah, never a dull moment. Well, I hope that story was worth it. I hope that story was everything you wanted and more because um I have to say it hits me quite close to home You know, I am in this story and I don't like it We're at that kind of level so yeah, how long have I been live? Oh nearly a whole hour. That's not too bad Right.  I don't like defeating them. They're too nice They're so cute and adorable and I don't like defeating them. It makes me feel so sad So sad But yeah, I hope you enjoyed that story I don't have stories often. This is like the one time that I actually have a story I don't do these sort of story times often because I'm just a boring kind of guy I don't really do much other than work and sleep and eat and stream That's literally all I do Also, should I release a video today? Should I release my update video? I feel like I've just been sitting on that for too long And I feel like I just need to release it even though the update hasn't actually come out yet Should I do that off the stream? Should I release that video? Should I publish that?  It was pain.  Oh my gosh game, please I'm probably not going to be playing much longer after this though because the game is being so bad So bad and terrible I really wish they would just release the update No, you shouldn't I should wait. I should wait until the update comes out Is that what I should? When I got home today, I went to bed for like four hours dreamed about my job And I deserve to be paid for dreaming about this Fair enough Fair enough Man, I was talking to my boss yesterday um I don't know how many of you know, but um The city that I live in we had earthquakes Like pretty like bad earthquakes that basically destroyed the whole city um in like 2011 um So like our whole city is basically being rebuilt over the last like Over the long since 2011 way too long way too long and we've got a new convention center that opened up earlier this year um and um, we were talking yesterday about like How how inflation is going and stuff because like it's pretty high. It's pretty high um And they're like the the event lady at the hotel was like, we just can't find any stuff. There's just like no stuff at all like We're paying like we're trying to we're putting jobs out there that are like worth way more than they should be But we're just still not getting anybody and then my boss is like Yeah, I probably need to find some more stuff as well. He says this in front of me, which is kind of funny And and i'm like Yeah, well, I mean the problem is that eventually if like companies put up their like like wages way too much It's gonna increase inflation anyway, which is not a great thing I don't I don't agree with people being paid absolutely terribly But like if you increase wages significantly, which is what you have to do now to get stuff um Then that's also going to like increase inflation a lot because a lot more people have cash and especially those lower generally the lower income earners Statistically spend a lot more which doesn't help inflation Um, because they've just got cash to burn and that's not great. Um, because it just means more cash is cycling through the economy Which is generally not great um, but anyway, and my boss is going on like saying the convention center Um, and then the hotel lady starts laughing because it's an absolute. It's it's I've sworn a lot this stream.  They they recently fired Or not fired the three head operators of all of the technical stuff So basically like the key people to make the to make the whole conventions into run Just resigned all of them on the same day Or because they couldn't have enough staff and they were working like they were working literally Well over 72 hours a week Which is like ridiculous because generally a normal working week is like 40 hours You know in this in audiovisual industry, you know, you can you can kind of think that you'll be doing about 50 Maybe 60 if you're pushing it, but 72 that's what? 48 like three straight days. I think For 72 divided by 24 Uh 72 divided by 24 three straight days of work like three full days. No sleep. Just work Not that they were actually doing that but like 72 hours in a whole week is ridiculous like absolutely ridiculous Or because there's just no staff Like absolutely no staff at all So they all quit on the same day Which is great. So they're currently looking for like a whole bunch of people to replace them and They're being they're like offering 23 dollars an hour Which in new zealand are very very low the current minimum wage Let me just find that I don't know that off the top of my head. Um Um Is $21.20 an hour Plus with the like the cost of everything at the moment like skyrocketing by the day 23 dollars an hour For like a hit job where you're going to be like guaranteed to be working like 72 hours a week Is like nothing absolutely nothing To put it in context My job that I do now I get paid 25 dollars an hour And I would be qualified enough Even though I have only done this job for a very small amount of time Um to be to be at that top job Not that I would ever do that but like I'm getting paid 25 dollars like That's pretty low So anyway, they've got major staffing issues and they've had to cancel like half of their events In the new convention center since they've opened, which was only like two months ago Like it is so terrible So terrible, um It's about 15 us after conversions so double our minimum wage. Yeah our minimum wage is quite high But we also have like significantly higher prices of everything as well um Because we're so isolated Um that we we need to like import a lot of stuff So our cost of living is generally a lot higher than it is in the united states So our minimum wage has always been higher than that, but over the last few years it's gone up quite significantly Yeah, yeah, so Yeah, that's low for new zealand, but the u.s would have people scrambling Yeah, but like the issue is that If you if you put that into perspective of like how much you pay for stuff in the united states Generally, you're getting a lot of the a lot cheaper stuff apart from like, you know, healthcare and stuff Most of that stuff is like a lot cheaper than it is here because you don't have to import it um Because you guys actually like produce stuff yourself a little bit whereas we we have to import like literally everything Like nearly everything we get imported at this point because the past governments have like destroyed all of the industrial Like side of new zealand and even if it is the high like wage rates make it basically like Not worth for a lot of like a lot of the basic sort of things to be operating here John's dead It's all done in like china and stuff where it's like super cheap and you can just have slaves because you know What's cheaper than paying people forcing people to work for you illegally? so Yeah, thanks a lot china appreciate it not um so yeah but Yeah, but like it's it's it's so frickin stupid So stupid I'm right though. I know I know I'm right like it's terrible. Don't get me wrong.  Otherwise, you're not really doing a business which Sucks because like you You're gonna have people trying to squeeze as much profit out as possible I don't agree with people trying to squeeze as much profit out as possible But I agree that there should be profit for business especially in times like at the moment where like All of a sudden your whole business could be just not allowed to operate for like a whole year, you know How are you gonna like pay your staff if you can't operate for a whole year? You know Unless you're making a profit and saving it um But like then I also Don't agree with them just being like, okay We're not gonna work here anymore and we'll just go and do slave labor in china. You know because That's not right either. That's not right. You can't do that. That's like terrible But yeah, it's it's a bit of an issue. I think I think that's one of the issues that the world is facing at the moment like how it comes to making making monies But also being more ethical about it Um, and I don't think china is helping the problem at the moment But then also do people like buying cheap chinese knockoffs Like yeah, they're cheap, but they're gonna break very soon Oh my gosh, I didn't tell you the other part of the story Oh my gosh So we have we um, we have tv's at work um that we hire out This is this is funny as well. You're gonna love this chat. This this kind of goes back into the first story that I was telling um We have tv's at work And you know They're fine. They work they work Um a few months ago. We were at a job at the convention center. Funny enough Um, we're all of a sudden just the remote wouldn't work It's funny because like tv's nowadays They used to have buttons on the side where you could like turn them on this one just doesn't it's too new They just don't they just don't do that anymore, which is so stupid. Don't understand why um But yeah, so it doesn't have any buttons on the side the remote stops working And we're like, hmm that's strange. So we replace the batteries still not working Not at all. Just just completely broken Not working So we go we go back and we get another remote because we have we have another tv That's exactly the same and the remotes can work for both Problems solved that remote works easy peasy. We go back into the back into the workshop And we're like, okay, we need to fix this remote So I plug in the tv that wasn't working with the remote um And I go and press the remote And it works So we're like, hmm, that's interesting. Why does it work now? So That's fun Fast forward to uh earlier this week My boss goes and gets that tv again because his one at home has like It doesn't do it doesn't connect to his like ethernet anymore because of a system update or something So he's like, okay, we'll just retire this one into into the workstock and um, I will I will take one off work Which um, which is like that's fine I don't see a problem with that. I don't think the accountant would be too happy with that, but like I would see um, so he's like, okay, I'll just I'll just use one from work. We'll swap it out. He in the weekend he goes and um He goes in and uh, like Takes it out the case goes and starts it up uses the remote doesn't work Doesn't work the road just doesn't work. So it's like, hmm. Okay, Brent Uh, when you get in on Monday, I need you to fix this remote because it's not working So I go to work on Monday. I'm like, okay remote needs to be fixed Let's do this problem solved easy peasy. First thing I do replace the batteries starts working and he's like Ah, that's interesting because I also replace the batteries before I started testing it again And I'm like, hmm that's strange So I take a video that it works after I replace the batteries because I'm like He's he's not gonna believe me.  I've got I still got on my phone, but I'm not gonna show you It's just be turning on the tv with her about nothing Nothing special about that Um, he comes home later later that afternoon And I like telling I've got it. I've got it all set up. You can go and test it yourself Um, but it was working for me when I tested it and I tested it again Um, just before I was about to leave And it was still working then so it should theoretically be working. I shouldn't have hit it. Well, um So he goes in that evening And it's not working again Nothing has been done to it since I last tested it and it was working fine So he's like we're gonna have to go to Panasonic and get a refund because what the heck is this remote? How does that work though? Like how does it all of a sudden it just doesn't work like I don't understand I don't understand. Have they released any new worlds? It depends when you last played. They generally release a new world every, um, October November-ish Um This is caramel There's La Maria, which comes after caramel Um, and there is a new one coming at the end of this year Which I'm not going to spoil the name of because that is way too big spoiler Way too big spoiler if you want that go to twitter Um, yeah But yeah, that's like This man's just doesn't want to fight me. I get it. I'm ugly Yikes, um, but yeah, that's That's been that's been my week. It's been a pretty terrible week. Not gonna lie Not gonna lie being a pretty terrible week But anyway, I feel like um I feel like um, that is the end of the stream. That's all I had to say That that that was my story time. I hope you enjoyed I might put that on my main channel because it's kind of funny I don't know if I should though I don't know if I should put that on my main channel Probably I'll see I'll see Anyway, um Yeah, I will be streaming again tomorrow. We're gonna be doing, um, stardew valley tomorrow So if you're interested in seeing me play other games That would be cool. You can come stop by, um, I don't Why know what time it is? It depends when I finish work tomorrow, but it'll probably be in the afternoon Unless I do it early What do you think do should I do an early stream tomorrow at like uh, 3 p.m. Central time or should I do um Or should I do like a stream like at this time? Whatever this time is What what do you think jet? What would you prefer? What would you prefer? No, because your ac won't be fixed before then Do what you want. I don't know what I want. I might do it this time. I'll probably do it at this time Yeah, I kind of like this time because I don't have to get up early I don't have to get up early and I'm lazy. I like staying in bed This time preferably. Okay. I'll do it. I'll do it at this time. Sounds good. Sounds good Might be slightly earlier than this time, but yeah, it'll be it'll be a long stream It'll be probably be about three hours. So Yeah Probably starting at about two or three p.m. My time, which is um three four five seven eight nine 19 p.m central Starting around then I would say and it'll be for like three hours. So if you want to come it'll be there'll be a link on Uh, youtube earlier. It'll be like a countdown like always. But yeah, otherwise. Thank you for watching. Hope you enjoyed Um, go check out my second channel craft box extra um It's cool. It's where all of these go afterwards or you can go to the playlist which is linked in the description below Um, but otherwise I will see you all tomorrow. Thank you for watching. And as always remember to craft outside the box


The game we are playing today is called Functions Go. It allows you to show your understanding of properties and representations as you match functions to earn the most points in this drafting game. The materials you need to play this game include functions go cards, functions go scoring cards, and a notepad to keep score. Give each player a scoring card. Shuffle the cards well. The happiest player deals first. Deal out cards depending on the number of players. The Functions Go scoring card shows a list of function families in the game. Use the list to sort the cards into function families to accustom yourself with the functions and the cards. Now we can begin the game. Deal cards to players as directed previously. Place extra cards face down off to the side. All players simultaneously play as follows. Choose one card from your hand that you would like to keep.  When each player is ready, reveal your chosen card. Pass your remaining hand face down to the player to your left. Pick up your new hand. Repeat these steps until only one card remains. There are two types of bonus cards. The first is a triple the points card. Apply this card to one trick function family. Sign, cosine, or tangent. Triple the value for the function family when scoring the round. The second bonus card is a swap for double the functions card. Play this card on one turn.   Then place two functions go cards face down on the table. When you receive the final card of the hand, place it face up with the other cards you collected. Score the cards you have collected according to the directions and the functions go scoring card. Collect and shuffle the functions go cards. Repeat these steps to play a total of three rounds. Follow up conversations about how students thought about the function families could benefit the whole group. Pause the video here to read our think about it questions. A variation to functions go is changing how the players interact. Alternate which way you pass the cards each round. In rounds one and three, pass each hand to the left. In round two, pass each hand to the right. A helpful hint is to sort the cards in your hand by function families as you did in the pregame play. Do this before deciding which card you will keep for that hand. As you keep cards in front of you from each hand, make sure to keep them sorted by the different function families. Click on the instructions link on the Regional Math Science Center website to read the game rules or just re-watch this video pausing as needed. Thanks for playing!


Aaron Donald said, if this was to end, not a big deal. I'd sleep easy. I'd be at peace if my career was over. There's some contract stuff that has to get figured out. Aaron Donald was speaking with the I am athlete podcast. I don't know if it was I am athlete tonight or the I am athlete podcast, maybe both of them. Aaron Donald said that he came into the NFL predicting to play about eight years. Winning a Super Bowl makes him want to run it back. His contract needs to be worked on first. If not, he's at peace with moving on. Aaron Donald is an absolute stud. Obviously, he's a Pittsburgh guy. Has been lifting since he was 11. People talk about him being the greatest player in the NFL, maybe the greatest player of all time, the way he's changed games. Now he's a Super Bowl champion. He's very well-paid. But there's a contract that he has to deal with with the Rams. There's no way they want him just gliding off into the sunset after what they were able to accomplish. And finally, getting a quarterback in Matthew Stafford, right? No. There's no way they're going to let him go. And I saw something. What's he getting paid this year? It's nothing compared to what the highest-paid defenders are. So yeah.


So, my name is Marcin Biz and you can call me Martin and I would like to tell you about a secured embedded product I did for one of my customers. It actually works, it's a story. Okay, I'm... I wrote a book about Linux, it's quite popular in Poland. I'm also related in some... doing some engineering and also some business related tasks with embedded Linux, real-time Linux, and I'm especially interested in industrial and home automation control devices. So as soon as I did working real-time prototype for my customer, they asked me, okay, now let's make it secure. So it will be about that. Okay, so there will be some trivial things. We'll get through this very quickly and what I would like to show you today is a practical example of a device secured. So no abuse user or anything else can steal or see the added value of the product. Of course, it's built on Linux, on open source, on some popular microprocessors, SOCs. Okay, I will not be talking about Android, web applications and so on. It will be somewhat about infrastructure. Okay, so first trivial thing. There's something like attack of the surface of attack. Here's the sum of the inputs that user or abuser can use to attack the system to get access to the system or just to influence the normal standard user to do something bad. So in case of embedded devices, we have some of the attack surfaces, which are different than the ones in the standard systems. The typical example, this is the quite old iConnect, the remote storage, it's got a USB, it's got flash device serial, network, actually wireless and also internet. Some of them are like normal things we can think about when trying to attack a device, like just try to see what's on the network, just try to look what's in the USB. Some things are not so straightforward, like some things here, probably JTAC or maybe something else. So we have one or more attack vectors on such a device. So the idea is to exploit a surface in an abusive way, the common ones, network, CPIP, Wi-Fi, Ethernet, all levels of the software stack from low-level Ethernet to high-level IPsec and so on. There's an application in the industrial control devices or home automation control devices. There's usually one application that user interacts with. It can be of course the network and the touch screen, but the application is working on the touch screen. So application must be do well. Okay, there are some less obvious, oh, the serial part is also quite obvious, less obvious vectors of attack, USB, it's quite easy to analyze what's going on the USB. You can just have access to system and run some kind of wire shark TCP dump that can just use the USB devices to look at it. iScore C, this is the quite common bus, this device somewhere there has a temperature sensor for iScore C. The idea is that iScore C is usually used for some trusted models, so trusted chips or some external chips or as a inter microcontroller connection device to just send data to some external microcontroller that is, for example, doing the real-time stack. Okay, solid state memory flash here it's QFP, so the pins are easy to connect to and analyze what's on this. Bluetooth, GPS, that's more for phones. Well, the less obvious the attack vector, the more dangerous it is for us. This is the more he takes. Okay, some differences embedded versus standard like server-like or desktop-like devices. Well, some attack vectors are unique. Usually they do not have iScore C on a laptop. The software update is problematic. Okay, we have Yocto, we have Android when the software updates usually work, but on the very, very embedded devices that has to be working in some factory facility and working well for 10 to 15 years. There's usually a root file system by build-root and there's no option to update it. Sometimes it's done, but in a way, so if the not well-behaving version of Apache, for example, Apache was used, it's a quite a problem. It's quite software monoculture. The system is done and nobody will update it. People will usually not treat embedded devices as computers, like we do with our phones. Phone is for calling and for texting and it's got a quite big CPU in it. Some people do not know, see this. Okay, on the other hand, we have the same problems, the same programs, the same software demons services as on the server, like Apache, OpenSSH, Perl, Awahi, DNS, OpenSSL, etc. Maybe not in the factory floor, but for example in home automation, automatic configuration, automatic network discovery, it's a quite nice thing to have, to have your iPhone detect your device in it. Okay, there are some infamous examples of misuse of security in embedded devices. Stuxnet, it's not actually a Linux related. Idea was to it was a virus running on some Windows software that was updating the unsecured logic controllers, spoiling the devices. In my work, I usually see some devices that are just have downloadables, locks, your FTP, and usually that's poorly secured. Admin default, it's actually my neighbor's network and it's router. There are more examples, quite old on this ELC page, and this is from yesterday actually. It's hot update. It's iZone, the camera you get in App Store, and it has the hard-coded root password. So this is a root password. I didn't show this. So it's a huge problem. They sell a few million of the devices. You put it in your home and you get to get a string from it. So this is a hot story. Misuse of credentials. Okay, so common methods are easy to fight with. Just use the strong passwords. Restrict the shell access. The idea with this iZone camera was that it was having a web page on which you get the access to the actual camera stream without any credentials, just security by obscurity too. So avoid other methods to access system shell. Web shells, SSH, Talonet, don't use it or use it with secure passwords. So strong passwords once again. In build-routed environment, I used to usually run everything from a root account. It was easy to develop. It was easy to run, easy to debug. But on the production side, it's nice to have separated credentials. Even standard Unix credentials. Might be SL Linux and so on, but it takes time to implement it. So we need to use bug-fixed component. This is usually not a problem at beginning of development where we have the quite nice version, but then there's a quality assurance, the test and so on, and there's no possibility to update it. Even if it's bug fixes. So this is a problem. Components should be up-to-date. And there's an ultimate question. Do it yourself or use the open source software. We can quite easily write a web server for web services in Python, in Java and other scripting languages or use Apache. So if you are using something, writing something in home is a defensive programming. It's like it's a nice buzzword, but the idea is to think about the other parts of the systems as are possibly already failed. So if I'm writing a program that is connecting the socket, I cannot assume the socket is okay. It's working. I assume it doesn't. If I get the temperature from temperature sensor, I do not assume that the temperature sensor is working. It's giving me the right temperature. It's a bit of paranoia. It will be well developing programming. Okay, so this was a trivial part and what I actually want to do today is to tell about passive security. This is all active security, like fighting the abusive users. And in passive security, we are defending our software and hardware from being still being analyzed by a maybe competition. So how the customer, how my customers actually see what I'm doing is the product. I'm doing all this part. The free software, the Linux, BSP, SDKs and so on, all this that support the foundation of the development. And what the customer really wants to sell is the added value. It's his fancy algorithm of doing some component extraction for some biochemistry process, for synthesizing the DNA, for example. This is the added value for the system. So the next thing is that hardware becomes cheaper and cheaper. I mean hardware that is capable of running Linux. You can have quite nice, but quite old, SOC for one and a half dollar in a bunch, for example. Just add a DRAM, just add a non-storage and you have a quite nice system on chip, a system on board. So expectations increased. Like let's put a 10 inches display for our device, because our customer will buy by seeing the device and the competition have a 7 inch display. So it's not possible to do in microcontroller in a nice way. So Linux and Open Source is a default system for Embedded. It's going to be, or it actually is, so people are trying to build the software products on the Embedded Linux basis. So just add value. Okay, Open Source gives us freedom so everybody can do it. It's like a company that are free engineers working on a product can have the same starting point. As a huge mega-corp. That's starting with the same basis, which is what Open Source and Free Software gives us. Okay, so customer will want to make money on update value. We have to integrate the added value to the system and secure it. Of course, we have to do it according to licenses. GPL or GPLBSD, you know the differences. So how to do it? It's just an idea. There are many ways to do it. Actually, you can just run a program application that is proprietary on the base of Open Source system. Other things like executing GPL applications or doing the kernel models are more or less fragile. You have to have a low advice on this account. Or just do an Open Source GPL code. So how to secure out that value?  Tivoization was a quite nice set-up box in the US. It's running Linux and of course you get the source code, but there's no possibility to run the compiled source codes on the device. It's sealed. So actually we can do Tivoization right now with Linux and this presentation will do it. But there's a slight moral question about how to do it and how to do it. So it's right to the acceptable for the customer and for the community. Actually, only GPL version 3 license denies the Tivoization. You can do it in Linux and other parts. Linux kernel is GPL version 2 right now. Okay, so let's see how to do it. At first nothing will stop user. Abuser for analyzing the product. You can buy a product. If it's a mass product, it's easier. If it is small volume like very specialized industrial automation, it's harder. So you can just desolder the CPU and analyze the logic states, in theory of course, because the paths should have a proper capacity. If the sock is running on one gigahertz, it's harder and so on. So it only depends how determined abuser is and how determined we are for secure device. So the main idea is that security is not a product, actually. It's a process. You are just chasing or you are running away from abusers who are chasing us. So we can use different security methods in devices that we produce in 1000 pieces a year and quite different should be used in the devices that are produced in 10 millions per year, for example. Okay, so there are hardware methods. I don't do hardware usually. I just write a specification to outsource it, but there are some tricks that the company that is doing hardware can do. First of all, example is BGP. Both are on the bottom of the CPU and other components. It's harder to analyze the actually tracks and signals for it. Next thing is the multi-layer PCB. Eight layers are quite common. Four layers are quite cheap. So usually the signal paths are in the inner layers of PCB. It's harder to analyze the logic states and it gives us the better emission of the electronic noise to the air if there are huge surfaces on the external parts of the PCB. So if using the multi-CPU design it's quite common in some industrial automation devices to do user application CPU to display the state of process on like 10-inch LCD and then use the small microcontroller to do the hard drill time path. So if there are two chips they can check each other. It's like watchdog. Microcontroller sends signals, timed signals to application CPU and vice versa and if there's no response everything shuts down or resets. So it's avoiding connecting a JTAG and just stopping the CPU. It stopped, it will get down. Okay, there are some nice TPM chips. You can easily find some presentations about adding TPM chips to your design in LC even. Just quick examples. This is a BGA CPU.  It's very hard to analyze what's here, what's the voltage on this. This is a multi-layer PCB. So you can see so we can use something like hidden vias between the layers. Images are quite simple from Wikipedia. As you can see this is also quite nice thing to have. It's called a sandwich. It's Texas Instruments CPU, the DM, on top of which there is a chip with RAM and NAND memory. So it's here. There's no problem just for paths on PCB. All is done in CPU. Quite nice. Hard to analyze. This was the IG board actually. The problem with securing or obscuring the hardware is that the hardware actually is quite hard to debug if something goes wrong. We spent about a month debugging this and it appears the solution was quite easy. They misaligned the D plus and D minus on USB port. Well, it hurts. Okay, let's secure data actually. I'm a software guy. I'm not doing the hardware. Okay, the first idea is to sign it. Use TPM chip to provide a secure way of storing the keys or high assurance butte. It's vendor specific for free scale. The idea is that software is signed. User cannot change the software abusing the usage of the device but still can analyze our algorithm. It's not a way I want it to go. My idea is to encrypt everything. Well, the encryption should be fast because it's a real-time industrial control device. So performance penalty must be very, very low. Especially real-time and deterministic performance shouldn't be touched. Okay, well, the algorithms are there. The tools are there. The main problem is where to store the key. So we can have the root file system encrypted, Linux root file system, the busybox plus application gives us 4 to 8 megs, depending on how much resources the application have. Of course, application is then decrypted and copied to RAM. But RAM is BGP. So it's hard to analyze. Okay, I don't know. Probably there are some encrypted RAM chips right now. So the idea is to boot a kernel with device strip, probably, new kernel. Maybe RAM disk, maybe not, we'll see. And then have a script in RAM disk that will decrypt the root file system. And the problem is where to store the key for this. Okay, so let's see what's what the possibilities are. If you are using the block device, SD card is actually not a good idea because SD card has a this mechanic part so so it can loosen it can if there are some tremors in especially in factory. So EMMC is the solution used in phones and and other wearable devices right now. And EMMC is just a block device. So we can easily use DM crypt on it. No problem. One crypt setup has a tutorial and so on. There's a Linux Unify key setup standard of writing the data so it can be easily discovered by crypt setup and other tools. So if you try to connect this to the PC it will show that it's a Linux partition and try to decrypt it. Okay, but this works for block file system. Another solution which is which work on top of the standard Linux file system is a crypt fs. Just mount a crypt fs one directory to the second directory. It is working on top of file system. So in one directory we have encrypted files and another shows us the the real files. There's a problem using a crypt fs as a root file system. So there's a problem if initram disk starts and then tries to switch root or pivot root to the the crypt root file system. It doesn't work. Still it can be used for encrypting a part of file system just. Okay, but in my case in industrial automation customer wants to use NAND the RoNAND device to have a wear leveling to have a bad block control because it's producing a device that will be standing here for 10 to 15 years and must work or must signal us if it's going to fail. So no use of black box like AMMC. NAND is nice here. Okay, how does it how does a NAND work? If somebody do not know yet. NAND has a IO blocks which you can just read or write but the only possible right operation is switching 1 to 0.  Well, next thing is that you can have bad blocks on the NAND. It's normal and the bad blocks appears as the erasing is done. It's like the modern NAND chip will fail about 10,000 erases. Some of the blocks will fail. So EMMC is just throwing a flash transaction layer that is doing a block device. Standard block device you can do input output. So just read the block write a block flash transaction layer will do the the rest of thing. So that's the main problem. We do not know how the flash transaction layer on EMMC work. We do not know how it is tuned how it is optimized. So it will be a problem. The solution is to use some kind of flash aware NAND aware file systems. JFFS2 was a standard. It's still used for quite small NAND chips. So it is doing the log based storage with ware leveling. It is aware of bad blocks and it will just store the data in a nice way. Okay and next standard for doing it for a big NAND devices is UBI. UBI is more complicated. It's actually the idea was not to do everything in one driver in flash file system but to just cut it in half. This is only doing the file system and this is doing the ware leveling and bad blocks handling. So UBI is nice for big NANDs. Actually benchmarks are quite for 2.3.2 kernel are here. But there is a problem how to add encryption to UBI FS. So it can emulate a block device. It will look like just upping another layer on top of this schematics. I don't like it. Use encrypted FS. There is a problem with switching root file system to a encrypted FS.  So it's a problem. Or to use the source code. So let's look at the source code, what's possible. The nice thing about flash file system is that UBI FS and JFFS2 also compresses data while writing it and decompresses it while reading. So you can actually store more data on flash device than you have. Quite easy algorithm like LZO or GZ. So maybe in this function that do the compression it can be also be encrypted. So just search. There's a crypto API in Linux that can do encryption of course. And patches are already there. They are floating somewhere. It's from 2012 actually. Joel updates some functions for UBI FS to make the encryption next to compression. And the other tricky thing was that UBI FS not always compresses data. If it compresses a block and sees that the compressed block is actually bigger than uncompressed it will store uncompressed block. But encryption should be mandatory. So the encryption is done. The patches are not in the mainline. Probably won't be. They look like abandoned. So the idea is to just grab it and apply it. It doesn't yet do the encryption. You have to provide a key. There is something like I don't want to go through the code. But there's a variable. A variable is set. It will be used as a key. If not, just pass. It will work as normal. So the code actually works. Okay. On the other side of the crypto API we have encryption is using crypto API. You can read here. This is counter mode. The other side of the crypto API is that the crypto API can use some actually hardware to speed up the encryption. So this is for DCP which is found on free scale. Sorry, it's not a free scale advertising. I'm just using chips in some of the designs. So it's forward ported from the vendor tree. It's not in the mainline. I put it in stagging. So it is. It just integrates with crypto API and then IPsec uses it just like that. And also we will be using it if it provides, of course, the proper eyes. Okay. Watch out. You have to use the proper block cipher to encrypt your data. This is important. Well, the first two images can be found on Wikipedia, I think. But the idea is to get the PPM image, which is just a text data, and then just encrypt it. Open SSL can do it. So let's use the ECB. This is the first. This is a block cipher. The data is cut into the blocks. Block is 128 bytes. And then each block is encrypted separately. So actually, you can see encrypted data here and how they look. Well, if you are storing some tables of data, like in an injection engine, for example, there are huge tables of parameters, this can be easily worked out how it works. So we have to use some mixing in block cipher. There's a CBC. Each block is just sort with the previous block. And the idea of the patch is to use the counter mode. Counter mode is nice for encrypting stream of data as used in IPsec. DCP accelerates this. And in counter mode, the result looks better. The last one is counter mode. Just CTR here, I think, to do it. Okay. So we can actually produce a secure device based on the FreeScape chip, for example, IMX28. Well, the key is stored somewhere there, still. In its RAM disk, it's not needed here because UBIFS can be compiled in. The key can be stored as a variable, for example, put in device tree as a parameter. Then you can encrypt the root file system and just mount it. One notice here, in case of a cryptFS and dmCrypt for block devices, all data are encrypted, like the file names and the headers of the file system, too, because it's on the block layer. And then we are doing encryption of the file data. So only the file content is encrypted. So file names are still there. If you will be able to connect to this flash device and just get the data from it, you can see the UBIFS structures. Okay, you can see that there are files. Just name the application up one and data one, data two, data three, security by obscurity, in some way. Okay. But the problem of storing key still exists here. So how to do it? We have to use, unfortunately, some vendor-specific stuff here. So I put a key in the device tree attribute just to have the same kernel for all the devices, and the DTB is the one thing that changes. For all root file system I built, I have the same kernel, because DTB is the part that changes. It can be easily put somewhere in the attribute. The driver can use it. It's got just the functions to call the device tree and get the attribute and use data. There's it. It's like 128 bytes for ASK. Then encrypt kernel plus any tram disk plus DTB using the chip-specific features. So for this device, actually, I have to switch to reference manual, and I can see it has the, sorry, the data processor here, and as for boot modes, okay, the previous chapter, it's got the disk configuration, the assurance boot, and secure boot also somewhere. Okay, encrypted boot feature. It's here, like secure boot. So it's from a vendor. So idea is to store a secret key in some registers of the chip, then to blow some fuse, one-time programmable fuses, to make the key storage sealed. Then the key is copied to the DCP engine, and then the DCP engine unencrypts something called BootStream. This is a bootloader image for the CPU. The idea is you cannot read the secure key from CPU. You cannot change it. You cannot turn the encryption off. Well, it's a vendor promise. Well, so, yes, I don't know if I will use it for iPhone, for example. Okay, it works. Well, another, the last words, oops, sorry, mistranslated, written in Polish. Okay, security is not a product, it's a process. So, you know, this is secured, but it's not really a product. It's just a process to running away from abusers. For example, I do not know if, in a 10 years lifespan, the IS will be, the AS algorithm will be still secured, like the RC4 used to be, right, 10 years ago secured. So it's a process. It's just abding another IS. Okay, don't use security by obscurity. It do not work, the camera example, use the proper algorithm. We have to, you know, depend on something. So in this design, I'm depending on the vendor promise. I think it should work for now. Okay, and the next problem, less from engineering part of the problem, but more business part of the problem is the source of the attack. The many attacks for the software are from inside. There's a WikiLeaks and so on. So people working in the company can just publish the code. So this is the huge problem for business. So in case of security, I even do not trust myself in this topics. The idea is that this process, which works for booting the secured product, and the product is secure, is to be put in some kind of, you know, build infrastructure and business infrastructure to decide who has the access to the keys, who will program the CPUs with the internal keys, and how the keys are stored, how the keys are used in a build system, and some trivial questions. How do we just build a root file system for every device, or just use the same root file system for quite a few bunch of devices, if we have produced million devices, for example. Okay, so the next problem is to just make it built in a secure way. Probably the build root installation with all the scripts done from beginning to end is not quite a good idea here. Okay, so it's not a product, it's a process, and beware of internal attacks. Don't even trust yourselves and encrypt your devices, for example. Okay, that's all. Thank you. Any questions? So, I don't see the mic. I can just repeat the question. So the idea is, is it possible to remove that? Going back to what you're saying about your root of trust, yes. And necessarily, you know, today, that being whatever the vendor offers for the hardware that you're using, and taking the approach of encrypting the image, the kernel image that you're loading in, I mean, great stuff. How does, you know, what effect does that then have on our ability to do a remote update of the system in the field? You can still do the remote update. You have to just think about the secure channel of putting update, via some network. The idea is that if you are building an image for update, you have to encrypt it with the key for a single processor. So every device is encrypted with different keys. So you cannot just do the mass update. Specifically with FreeScale. Yes. A signed kernel for the particular device with its particular key. And I did a partial update and that failed. Have I got an ability to fall back to the previous signed image? You do not have. Yes. You have to think about some, maybe, I was thinking about it like that. I can have an image that is encrypted with multiple keys. It's for FreeScale specific if you are down. So I just create an image that will run on 100 CPUs, and this image will do all the checks, the defensive programming. If all the checks are okay, it will update. And the quality assurance assures me that it will work. Of course, there is a problem that if update fails, the device is bricked. So the CPU still have some recovery possibility. Like for a FreeScale specific, you can just turn the boot, you can just change the resistance, press a button for a device and it will boot in the recovery mode, waiting for a USB for data download. So it's probably not, user cannot do it probably on site, but the technician could. Yes. You can always use a dual-boots kind of concept. There usually is a first-stage boot that takes over the chain of trust that you can keep static in there and then you would be able to switch between two kernels and router faces that would always allow you a fallback if the firmware update is encrypted. Yes, that's a good idea. You can use a secondary image or bootloader or even kernel that will be the fallback image. Some CPUs have the ability to just try to boot something. If it fails, go to another image or another offset in NAND. It can be isometric and then there's some logic in it. Store the isometric key and then store the symmetric key somewhere in the script, which is just encrypted by the private key of the symmetric one, but the idea was to boot it quite fast. So storing the symmetric key and using this key in DCP accelerator was the fastest way. You can always add some other features of the chip. This chip has the high assurance boot that the whole image can be sign additional to be encrypted, but it's not giving us anything new in this case. Chris Peele has support for this hardware acceleration of the encryption, the DCP. What about the microcontrollers or chipsets which doesn't have the acceleration and we would like to have them in software. Do you know any benchmarks which show how long does it take? No, I do not know any benchmarks I do. I did myself on the DCP versus software and DCP was faster. How much does it take? Do you see any difference between unencrypted and encrypted? Yes, I can see about 10 to 20 percent slower read access for an ant. It's just a figure. I do not have a test here. The idea is that the application is built to be a real-time process running on the preemptor t-cernel. So all of the application data or resources are preloaded to random access memory before going into the real-time state. So it doesn't affect the working performance, it affects only a boot time and start time of the application. You mean taking it from the flash to RAM? Yes, if it has to be the real-time application, it has to put all the data in RAM, do the memory locking, set a scheduler priority and then it can go into look for example to do some real-time stuff. If it would read the data during the real-time, the reading is not real-time, not deterministic in Linux. Yes, it's the feature of the CPU that it's very easy to encrypt everything, the whole boot stream. Yes, it's not trying to obscure. The idea is this is actually a devolization case. The user cannot change a kernel here, it's a problem, will be a problem at some point. The idea is that the same thing can be used just to encrypt a part of the file system, user application data. But at first stage we do everything, encrypt everything to show it. The idea is to encrypt only the parts that are needed, the encryption. The first idea is to just put the variable equals key in the kernel code. Just a quick hug. Just to show the customer. The device tree was the nice solution. It's just developed later on. Yes, for reading data from NAND, hardware encryption. I do not have it here, I can show you. Yes, I do not check the software encryption actually doing reads. I just run a row benchmark on the crypto API, which was faster, and the faster was DCP, so I then use... Yes, of course. So I used DCP and it's like 10 to 20% dependent application actually. It's not a setup. Yes, using hardware acceleration. Yes, it could be. So the idea is that if you know this and know this and encrypt the data and encrypt the data, you can work out what the key was here and it's not possible with AS currently. Probably would be in 10 years. So that's a good point. So just encrypt the unknown parts. So if you are encrypting everything, the kernel and the known data, you are giving the attacker a standard kernel that can build itself and the encrypted version of it. Yes. Awesome. And also the tricks that you can do with the linker to adding noise into your kernel, it slightly increases your kernel image, but it has a lot of random data to each of the releases. It might be really difficult for somebody to figure out what the... Yes, that's security by obscurity. It's difficult, so it will cost another 100 dollars to work out for an attacker, right? $100,000. It's not really a security, but it just means that the node is no longer in nodes. It makes it a lot less difficult to crack the keys. Yes. How did you manage that? Is it allowing again security? I mean the overhead of having everything encrypted, you will do the break, you will be back. Yes. So there is an overhead in development. You do not trust the developers in the first place, so developers do not like it. Well, the idea is to just develop the Linux application that is running on a known basis. So to the point, the application can be developed on the standard desktop systems, secured with some hard drive encryption, and then put to the real device, which is the main difficulty. The multi-thread application, the multi-thread debugging is quite the same on the PC and on the ARM. Talking to hardware is a bit different, because of specific interfaces. So I don't know. I'm not an application developer, but they are not quitting. Okay. So in this design, I actually have two keys. One key is for, next slide, one key is for to the CPU. No. This key is the different key for each single device. Yes. I have to store it on my own in some secure way. Yes. I have to get a key. Yes. Yes. That's true. The idea is that you can just write scripts on everything and put it on the server and store it in the secure place. And the idea is to have one or a few just management stuff who can just press the button to encrypt it. Yes. I've got to... Yes. Yes. That's true. It's like the devices are produced in a bunch. Like I've got a bunch of 100 devices and I've got a labeling process that gives me the labeled CPUs. Each CPU has a unique serial number and then I've got the data, the serial number encrypted key. This data is encrypted using the asymmetry cryptography and then the public key is on the pendrive in the safe place and only management has access to it. It's a case of physical security actually. Correct. Yes. Ten of these machines on the same place you have to give. But it's just but just some part of the application because maybe the added value is just a single program that gives you the specific algorithm and the data of the algorithm. It is... Yes. You have right. It is possible. Yes. It is possible to access this specific free-scale DCP from user space. The idea is that it's got a CPU key copied and then you can just ask encrypted using the key. So you do not get the key from the DCP in this case. But it's not universal. We have to stop. Okay. So thank you very much. We can continue during the break.