So we'll start our next presentation. This time we're going into the formats and codecs area. With our first speech is about Matroska and low latency streaming by Steve Long. Give a warm welcome please. Hello, so I'm Steve Long. I created not alone the Matroska, so for those who don't know, you might know the format MKV, so that was the other name for the format, the actual name. So Matroska was created as an open format from the beginning. So it was developed mostly on the web with people who mostly never met each other. So mostly on IRC mailing list or the Doom 9 forum. And it's mixing lots of technologies that were kind of emerging when it was created. So the idea was to replace AVI, there was org at the time, XML, Unicode and Semantic Web. And we mixed all of this together to create the format. It was developed in a way that's inspired by what was done on the web. So everything was open, everything was royalty free. And the original goal was to have a long-term storage format, something that would be usable for 10 years for when it was created in 2002. So in 2018 it is still used, so we reached that goal. And the other thing was to have low overhead, because since it's inspired by XML, it's very verbal, so we try to remove as much verbality as possible. And also streaming friendly, because the design was always... So it's the basis format for WebM, which was created by Google and Mozilla a few years ago. And so you'll know it's also used as MKV. And there's also MKA for files with only audio. There's also right now ITF working group to standardize Matroska. It's called CELAR, so it stands for codec encoding for lossless archiving and real-time transmission. So the workgroup is working on EBML, which is the binary XML that Matroska is using. Matroska, which is now split in actually in three different documents now. That might be all, but at the time each project had about 13 contributors, 300 commits, so mostly on documentation, cleaning the documentation that existed and improving it. The project, the workgroup, started in 2015 and still ongoing and still a bit of work to be done until we have final RFCs for Matroska. The workgroup is also working on the standardization of FFV1, which is a lossless video format and also FLAC, which I'm sure you all know. So my talk today is talking about the latency in Matroska and how it can be used and how it does it compare to other solutions. So first to make sure we all talk about the same thing, the definition of latency I found on Wikipedia. Latency is a time interval between the stimulation and response or from a more general point of view, a time delay between the cause and the effects of some physical change in the system being observed. So basically here there's a diagram of what's happening when you actually do some, for example, live streaming of video and audio. So first you have four video, you have frames which go into an encoder and then maxing, that means putting it in a format that can be, for example, mixed with audio and also adding timestamps and stuff like that. Then it goes through the network. Then on the other side there's the demaxing, so to get the encoded frames that go through the decoder and then you get the frames. So each of these steps actually introduces a latency. I'm not going to talk about all kinds of latencies, for example not the network, but some that are more involved in what Matroska does. So for example when you, okay, that's supposed to be red. So the video encoding latency, for example, usually you have frames one, two, three, four, et cetera that come from the camera. Then it goes through the video encoder and usually when you have something pre-recorded the codec can encode the frames in a different order than they are received to have better coding efficiency. But for live streaming you don't want that, you don't want to have frames that appear very late compared to what they're supposed to do. So for live streaming you actually have to put your encoder in a mode that always sends the pictures in the same order. So basically live streaming is always a special case of encoding and maxing and everything. It's not the general case of how most people use a video usually, but it's growing since most people now watch videos on YouTube or whatever, a new Facebook. And that's increasingly going through the web and networks so that's becoming more and more important. So how does Matroska store information? So that's how when you look inside a Matroska file you have first a very small header. The metastick which actually tells you where these parts are. So when you read here you can quickly go here if you want to. Then these parts are the actual audio and video that you're going to read. Here it's for seeking if you want to go quickly to a place. If you have that information you can go here. Chapters, it's regular chapters and tags for all metadata to tag, actually you can tag this information but also this information with tags from here. But in streaming you actually don't have a lot of these parts. You only have the header and these parts. The metastick since you're not going to seek during live streaming is useless. The queues you're not going to seek back so it's useless as well. You cannot introduce chapters for the same reason you're not going to seek in the file. The only one that could be used is tags that you had actually put before send the actual data. So WebM is actually exactly the same as Matroska. It just has at the beginning it says WebM instead of Matroska. Otherwise it's the same thing with some feature removed. So it's basically a Matroska profile. So in WebM's live streaming it would look exactly the same as in Matroska. But most people don't use actually live streaming this way. They actually use adaptive streaming so when your connection is better you can have better quality or your connection is bad. You have a lower quality version. That's why when you do live streaming in Dash which is one of the versions of adaptive streaming you actually have, for example, the audio can be split from the video so you can have a better quality video with a lower quality audio. But the file actually looks the same. It's always still a header and a cluster for audio and here it's still a header and cluster for video. The only special thing is that the transition here when you go to a better or lower quality of bandwidth the boundaries here have to match always and in all the versions of the same file with different qualities. So that's actually what you are using when you're watching YouTube or Netflix, that's the kind of technology they use. But live streaming is even different because that's for even regular files that are already stored for a long time but live streaming is actually even more constrained so live streaming is basically when you have as little time as we saw before the definition of latency as little time possible between the moment the frames are captured and the frames are received to the user and you try to remove as much as possible stuff that you have. So basically how does Matroska file in live streaming so basically a continuous stream of data coming it's not a file you're reading but a stream. So basically this part is always there and it's the same part, I don't know. So it's this part that we adhere, it's always the same structure that you have your audio and video. So I took an example of a file from Fragmented MP4 and I redrew did it in Matroska to see the difference. So the header on Matroska was 460 bytes big and then each cluster as a header we're going to see that after and then the data, the header is 25 bytes and then you have the data. So the extra data is here and that much data and for Fragmented MP4 you also have a header and then the data. The header was 2100 bytes and the equivalent of a cluster in Fragmented MP4 the overhead is for 850 and the data. So you can already see Matroska saving space. So that's in more detail what a cluster where you actually save the frame. When I say frame it can actually be video or audio it's exactly the same thing. So in each cluster you have one timecode for the cluster and for each what's called block you have a frame and a header. So each frame has a header that gives each timestamp and other stuff. So basically before you start getting the first frame on a cluster you have 25 bytes and for Fragmented MP4 the structure is a bit more complex. You have this atom which consists of more atoms inside which is like 1k and then the actual data where you have each frame.  So here what I call frame latencies how much time or how much data it's kind of the same. Do you need to actually get the end of the first frame that you can send to your decoder? So here for a frame of 40k you would have 40,025 bytes before you actually can send it to the decoder. For Fragmented MP4 you actually have to have this 1k and then 40,000. So again the latency is better in Matroska. Again that was a real life sample and I just transformed into Matroska to see the difference but it's the same. And the lower the bitrate actually the bigger difference it makes to use Matroska. So basically here the other thing you have is buffer latency because as you can see in Matroska you send one frame after the other with its header in Fragmented MP4 it's different. The header for each frame is actually stored at the beginning. So in terms of latency before you get this you need to know all the frames you're going to have in that whole big block. That means before you actually get frame one you need to have all the frames and wait so as we see earlier. So if you have blocks of two seconds in adaptive streaming then that means you need to cache two seconds of data before you actually can send it because before you can write this data you need to know all that stuff. So that's for example in that case two seconds latency or in the case of Matroska it would be just that amount of data so 25 bytes and then the amount of the frame. In this case it was 20 milliseconds. And then so we already see that Matroska has a big advantage here. The other format that's used a lot in streaming and that's actually the format that was used in Apple's adapting streaming format. They insisted on using TS but now they changed using Fragmented MP4 as well. So the same for one frame in Matroska how does it work in transport stream? So one frame is actually cut in 184 bytes and then each of these 154 bytes is encoded in 188 bytes so each has a header, so four bytes header then a part of the frame then a header part of the frame etc. So actually to get the first frame you actually have to get all the parts of the frame plus all the headers in the case of the 40,000 kilobytes frame the overhead, the extra data you have to download to get to the end of the same frame is 869 so the 869 is the difference compared to the 25 bytes. So there's also less latency in Matroska that transports streaming actually early. So basically we saw the difference between Matroska and Fragmented MP4 so the advantage of Matroska compared to this is a kind of smaller overhead from the network there's actually only like 400 bytes difference between the formats so it's not that much but for example if you buffer two seconds in adaptive streaming that's the latency that you're going to have in Fragmented MP4 but in Matroska you don't have the data you need to get to the end of the first frame is like two or three percent of data so that's data you could actually use for better encoding in your codec and the buffer latency so the amount of data you need to get to the end of the first frame is the same compared to MP4 you need to get the whole let's say 20 or 40 frames before you can actually start sending them and that's actually early so I hope you have a lot of questions and I have some stickers in my bag and you can also go on the video LAN booth in Building K there's a wheel of fortune you can win lots of stuff and there's also some stickers there and also thank you Hello How are you? I have a presentation just after you so I am a bit stressed but before that I am curious about how you compute the cluster size before sending the second and third frame do you know the size of the clusters when you need to write this size in the Bstream? So I am going to go back to show let's say here so yeah, as you said since it's inspired by XML actually each element in EBML has an idea and the size of the data that it contains and then the actual data but in live streaming there's a feature in Matroska and EBML actually where you can actually tell this element contains data but I don't know how much so I'm not telling you the size and you keep reading until you find the same cluster beginning and then it's a different one it's called, there's different ways to call it but usually we call it unknown lengths and so you just send that in your stream even though you don't know the size you don't need to wait until you know it you just send it and it's too bad and on the other side the parser knows it has to handle that size In that case, how do you know at which level of Matroska you are at the block level or at the cluster level it is with the key of Matroska it is a cluster so it is... that's how EBML works so to read an element the ID, you have to actually know what the parent is and then it's parent, etc so when you find data that are not for this parent you look in the other parent it's for this parent, etc and then you actually find where it belongs another question that's it so MPEG-2 transport streams are able to be cut off at the 188 byte boundary which means that if you've for contribution sources happening before what is distributed you are able to when you have multiple streams for resiliency going into your final endcode you can just mix and match if you've got network problems or other kinds of say encoder issues are you able or not able to do that with Matroska in a similar way given the format so I have two questions ok, so about live streaming have there been thoughts about program metadata yet? what do you mean program? such as EPG and things like that well in the case of adaptive streaming usually the metadata are not in the stream it can be in the manifest so in adaptive streaming a manifest is actually an external file which describes where to find these files for example with a general rule they're not rewriting the file every time a new segment comes and they can put all kinds of data in there it's not related to the actual data yes but in case of a single stream live stream like I said earlier you could actually put these tags so tags like the author or the date it was produced or you can put all kinds of stuff you could actually put it here and because it's a live stream you could actually put it here and here you can add a new one during the streaming to actually change the data well like if you play a song and then the song changes then you have new metadata that you can introduce in the middle it actually exists for mp3 streaming and you get id3 or I don't remember the format in the middle of the stream that you have to pass and then you get the new song that's playing it could be also done the same way with Matroska okay thank you I feel that you are mixing explanation on file format and what you put on the wire for doing the streaming and you are comparing Fragment at MPEG 4 which is a file format with the streaming which is maybe HLS or dash whatever and why don't you compare to the other ways people are doing low latency streaming like LTP or MGPEG which are the low latency solution nowadays well since ever okay compared to RTP RTP is a network thing and usually they actually there's all kinds of formats for RTP but usually there's not even a container that would be Matroska in this case and what I'm talking about here is actually either files for advancing streaming but that's how you can still write on the server while people read it which cannot be done in Fragment at MPEG 4 that's why it's mixed between file format and streaming format because in the case of Matroska it can be both in the case of Fragment at MPEG 4 it cannot but compared to RTP I mean it's not the same goal you can probably you do get better latency with RTP but then you get you don't scale that well or you have other issues and that's why people on the web at least they're not using that because that's not how technology works but for example RTP is used for example if you have Twitch or other stuff I think Twitter live the client is actually sending the server in RTP format but then for distribution that's not a good format for them they need something else Thank you Steve


Hello everyone, welcome back to another Apple's NTR's vlog. If you are new here, my name is Charlotte and I am a fourth grade science and social studies teacher in Northern Arizona. Today's vlog is going to be a tour of my brand new classroom at my brand new school. So I previously taught in the east valley of Arizona in the Queen Creek area and we recently just moved up to Prescott area which is a northern Arizona city and it is a very small town and so we're kind of like a quad city area. So I am going to be teaching in the quad city area. I won't tell you exactly which district I'm in just to protect my own privacy but I am very very excited to be here. So today I thought that we would come in and just take a look at the empty classroom. I did build like a digital version of my classroom on the Sims 4 a few months ago which was super fun and I think what I might end up doing in a couple weeks is like rebuilding that just based on the changes I'm making. Kind of just after coming in here and seeing everything, the furniture, the layout, what's available, things like that, I might end up changing the layout of the classroom on the Sims so keep an eye out for that. But I thought today that I would just come in and give you guys kind of like a blank canvas tour of the classroom. There's a lot of stuff in here that needs to come down and be removed and like change but overall I'm really excited the classroom is beautiful. There's a lot of really amazing things in here. There's also a few things that obviously I'm going to either change or remove. So what I might do today is just kind of take you guys through. There's only a couple things that are missing from my classroom that are mine. One of them is like a large bookshelf like that. It's the black one that has like the two-story and then there's a really big box that has like my lamps in it and like some bulletin board paper and like all of my taller items that I didn't want to just carry around. I wanted them in a box. So those are the only two items that are not currently here but everything else is here and it's good to go. So I will just give you a brief tour. So just keep in mind all the things that are currently up in the room that are like like physically you can see them on the walls and stuff. Just keep in mind those are not mine and they will be coming down. I think maybe I'll do that today. But anyways you come in the front door and then we pan this way and you can kind of see my huge wall of beautiful cabinets. Now I will have to repaint these. I'm planning on getting a bottle or a can of black chalk paint and just doing a nice thin overlay. I'll probably have to wash these down first and then completely paint them but I did get permission from my principal to do so. And then my teammate who's next door, Jocelyn, she is also going to be painting hers so we might just share a can of it since we won't need a ton. One thing that I have noticed is this beautiful thermostat which is adjustable by me. So right now it's 70 degrees in here and it is perfect. I already started putting some things in these cabinets and like I said a lot of stuff is not mine so I'll have to go through it and kind of dump. But a lot of my things are already in the cabinets in boxes, not organized at all. I will be coming in a few days during the summer and organizing those the weeks that I'm not working for my sister. So you have this really tall one which is great and all of these shelves can come out so if I want to make like a tall layer I can. But again, most of my stuff is in here. Some of the old teachers things are in here which I will probably get rid of like I'm not going to need this one because I'm not a reading teacher unless I want to use it for small group which I will be conducting small reading groups so I don't know for sure if I need to keep those or not. And then they left this huge box of Legos which I'm psyched about because I do a lot of earthquake stuff and also the Lincoln Logs so I'll be using those for science for sure. And then again I have a sink, it looks like the drinking fountain does not work. I hate that this is like taped up like that so I'll probably change that because I don't like that look it's kind of icky. And then I have these under cabinets as well. This is a computer cart. This is a computer cart, Chromebook cart. It looks like everybody gets one. They're like Dell which is really nice. And then you can kind of see the TV over here. Is this a microphone? That's sick. Okay so my big question was do I have to keep the TV? Oh well this I'll probably get rid of this. I don't really need it. And then this looks like just a regular computer which I probably don't need. So my big question was do I have to keep this TV here? Or can I move it around the room? So as I'm looking it doesn't appear to need like a special plug. It's just kind of plugged into the wall here. As far as I know I'm going to just kind of peek about and see but from what I'm looking at it looks like it doesn't need like a phone jack or anything. It looks like it just needs a plug. So my plan is to move it to that side of the room and have it sort of in the center of the two whiteboards because what I want to be able to do is like have like concept model space and then like vocabulary over here. So what I'll do is like I'll make my three vocabulary things and then on the way I can like draw on one side and have vocab on the other side and then over here I can have like my daily schedule and like things that never really go away. So yeah I was thinking I could put things that like don't really move on this whiteboard or see if they can um well no I'll probably just leave this here. Or I could put my small group table in front of it and use this for like small group stuff. And then I do have two beautiful windows that have a great view. I'm not going to really try to show you the view though just for privacy reasons. There's a lot of rolling chairs in here which I don't need like there's one two three and then mine four. I actually decided to bring my office chair here because in my new house I will not have an office. So I won't have a necessity for that chair. And then this big desk over here is probably going to be either one moved to a new location or two removed altogether. Cash be careful there's a mouse trap down there. Um and so yeah um so yeah this whole back wall is all bulletin board material. And as you can see there's a lot of like icky like stains. Um so what I'll probably do is cover most of this with like bulletin board paper. I actually considered doing like a better than paper wall but I think that's going to be a lot of work. I would want it to be this one and I just don't think that I have the funds to do that. I might be able to I mean I'd probably have to buy like five more rolls of it but we'll see. Um and then let's see what else. Okay so the other thing I can have six groups of five students at each group which is fine I'm used to that. I will probably see if I can either A get another table or I will create another table with these like hexagon tables that are around because I do have quite a few of them and make seven because I like having seven continents. And then this table is completely damaged as you can see someone's brag tag is like shoved under there. Um yep someone's brag tag. So this table is completely destroyed. What I'm asking I already kind of put in a work order to either get it removed, fixed or replaced which my principal has assured me they put in a work order and so it might get fixed. If not um I might try to ask them if I can repair it myself with like some like super good glue and like Mod Podge just so that I can remove this tape because I'm not interested in having that. I would rather like Mod Podge the table and just you know not have that or honestly like I'd rather even have desks that can form small groups. Um be careful please. So anyway these tables are really nice though they're very large I think they're bigger than the tables I had before and I like how they're gray and black. And then I also love how I have black chairs. Um these are great because A they're black and two um when you push them all the way in the kids can still stand like right here at the table because they do a lot of standing work when working on their um hands on stem activities so that's going to be great. But yeah there's a whole lot of stuff in here that I don't actually need like I'm not probably going to need many of these tables. I won't need whiteboards. Um I won't need this desk. I do love how there's a desktop computer though. Oh god there's another mouse trap under there that's scary. Um so I'll probably hook this up and use it for like class dojo like have class dojo pulled up on it all the time and then the kids can go over there and like take points and give their self points. I really like that. Alright so I forgot where I left off my sweet principal stopped by to say hello. Um another thing that's different is I have carpet instead of tile floors so I'm gonna have to be really strategic about some of the labs that I do involving like water and mud. Um but overall so yeah she just explained that yes I can keep this TV anywhere that I want so that's great and then she also mentioned that I can dump anything I don't want into the hallway and just I don't know and just mark it as trash or please remove. So I think what I'm gonna do today while I'm here is just take down all of the things that are on the wall oh and she also said that if I wanted to repaint the room that I I can um it just has to be like a neutral color so what I would probably end up doing is painting it like just plain white um or maybe like a really like lighter shade of beige um just because like the damages around are just kind of like like this kind of stuff I just don't want to deal with it like the paint's chipping and like she also said that this desk is going to be removed which is wonderful news so I will also spend some time today taking all of the stuff off of it okay that's fine and then she did confirm that there is going to be a replacement of this table and possibly giving me one more and I said that if I needed to I could keep this one and just fix it if she gives me permission I can just use like the mod podge um to fix this table and then I could have seven so that's great news okay so what I'm going to do is I'm going to spend a little bit of time here today maybe an hour just taking down all of the posters and things that are up on the wall and um try to get them either put somewhere for trash or for any other teacher like the parts of speech my reading teacher might want that and then like these math posters my math teacher might want that um same for up here so I'm gonna just kind of remove everything for now and then then possibly try to assess some of this like furniture like this thing I do not want that is trash she told me um these two tables I don't know if I'll keep them I might keep them and maybe paint the edges of them with like a black chalk paint because I hate like the smudges and then this one I might keep I don't know yet because I'm planning on like if you watched my classroom sims classroom video I'm gonna put my desk my my uh rise and fall desk over here like this is gonna be my little corner but I'll teach from here too like it won't just be like a desk because I have my black organizer and then my rising and falling desk will go there maybe my black table will go there in the corner just so I have more like space um and so all of this will kind of be put back there the tv is gonna go here and then I don't know about peach's cage I'm thinking maybe like maybe I could use one of these tables for peach's cage or put her on top of this counter top but I also think maybe I'll keep these for like stations because I do a lot of stations um so those could be really good and then they would be just like permanent station tables I'm not sure yet I don't want to waste them because they're great tables maybe just get rid of this one because it doesn't have the black legs because I'm so anal about that like see how this table has black legs and this table has like tan legs so yeah that's what I'm gonna do and then cash and I are gonna go have lunch and go for a walk around the lake so yeah let's do it also my principal just brought this to me and she said that every year one of our local churches sponsors the schools and so in here are a few gift cards for supplies do you want to open it it's okay to rip it whoa an amazon gift card it's for a hundred dollars wow that's amazing what does the card say that's so cool so we'll save this for like markers or notebooks or dry erase markers or anything else mommy needs for a classroom isn't that cool baby not for toys all right so we'll put it in there and keep it safe okay so I've done a little bit of taking things down I think I got all of it minus like but I did get permission from my principal to paint this room so what I might do is go look for like two cans of like discount paint like something that they don't want anymore something that's very neutral like very light maybe even like a creamy white or something and just come in here like the next couple weeks and just paint oh thank you so much but what I did think about was since I do have this sink area I will probably use it for like a mini kitchen if I'm allowed to so I'm thinking is I'll plug in my microwave on the top of the counter here like maybe over there and my coffee pot and then I can put my refrigerator right here with like an extension cord up to that plug or I can put the extension cord on the floor snaking around that way I'm not sure yet um because I do have this plug here so I could snake one down like around on the floorboards I don't know if I want to do that though so we'll see but I did get this wall cleared off and I did get a couple of pieces that I weren't sure I was gonna keep out into the hallway like that weird filing cabinet I'm probably gonna get rid of a few of these rolling chairs because I don't need them I will probably keep this one and maybe the one that's back there I don't like how they like recline that's kind of weird but I might keep them anyway just for like a small group table or like as a prize like they can sit in that chair for the day um so it's like a dojo prize they could roll this chair over to where they sit and sit in that instead of the black chairs or I don't know yet this is definitely gonna be like my desk chair I need to clean it it got destroyed in the move um oh you're putting that in my pocket thank you another one so I don't know we'll see about the whole fridge thing it's either gonna go here which would make the most sense because I'm thinking that this cabinet up above the sink is where I'm gonna keep like my snacks and my food and like things that I need to cook with like dishes and things like that and then above over here I'm thinking I'll do like cleaning supplies make this like a cleaning cabinet or I could put like he's got vinegar in here I could put all of my like liquid lab supplies in here like even paper cups things like that so we're gonna stay another hour because my teammates coming in and I just want to chat with her for a little bit so I think I want to do right now because this is what I'm going to do is I'm gonna be storing all of these crates up on the top of the cabinets so what I might do right now is just put all of them up there so that they're up and out of the way and then I can bring them down when necessary because they're all kind of on this countertop and I could get rid of them and just put them up and then that's done so I might do that all right you guys so I did a couple more things in the classroom just got some things up off the floor because I know the cleaning crew is coming in this week and then I ran into my teammate Jocelyn and we talked for quite a long time it is now two o'clock in the afternoon cash and I are starving so we're gonna go grab some lunch and then I'm sure the next time I go to school yeah they're flying their kite um I am sure that the next time I go into work it is going to be time for me to set up my classroom and I know a lot of you are going to be like why are you going in on your time off I am somebody that really likes to have my classroom looking nice and I don't like rushing I don't like being put under pressure to get my room done when everybody comes back to school I'm gonna have a lot of new to the district trainings that I'm gonna need to go to and so I just want to have my room completely ready before I have to go back to work so I will spend a few days hold on please I will spend a few days in my classroom during summer just getting things set up I do have to repaint the room so it is going to be quite the job but once that's all done like next summer I won't have to come in and spend as much time because I did just move classrooms and I feel like that setup kind of just takes longer than the average so anyway I will be sure to pick up the camera and vlog that for you guys the next time I'm at school but otherwise don't forget to follow me on Instagram and TikTok and of course hit the subscribe button so that you can see all the fun things I'll be posting this summer thank you guys so much for watching and I'll see you in the next one bye guys