What's up everybody? I'm EvilRyman. Today we're here to do something a little bit different. We're here to talk about this beast right here This R.C. Rural Drive Drift Guard that I have I've been you know poking hints at it on a couple videos prior as well As I've been poking at it on Instagram and Twitter So here today we're gonna do a full build breakdown on my one of my Rural Drive chassis Which is this one right here my Matt Teal Toyota 86 First and foremost it is a Yokomo Toyota 86 body It has been painted multiple times multiple different colors. It's been through a lot It's been through a couple competitions as well as a bunch of modifications in the back. It's had wings It's had things cut out and stuff like that, but it has done me very well and done very good for me It's been reinforced on the inside. So it it's been wall tapped a few times. It's been bashed in a few times It's it's done its fair share of Competitions and runs so we're gonna get into the full build breakdown of this chassis because I know a lot of you guys been Asking about it and I do do a lot of Rural Drive R.C. Drifting. So it's my fiance She's starting to get into it. So we're gonna get into it show you guys my one chassis I'm gonna get take this body off first and Give a little bit of a look at that first I do run pins just so that if I bashed into somebody the body doesn't fly off So it's like a chassis way for a moment So it has been reinforced on the inside with plenty of shugu and light buckets and stuff like that to keep it Nice and rigid and keep it from breaking of course new additives the usual suspects on the side so it's just a standard 86 body kind of a beater body, but We're gonna toss that aside and get down to the nitty-gritty that is my Eagle Racing R31 Custom built chassis. I say custom built because this is not exactly how it sits. So We are rocking some 3d printed 1552 wheels These were printed by a buddy of mine. We're just trying out to see how they see how they do in and hold up for drifting some really good book of bearings in the front so The carbon deck it is all carbon fiber and aluminum. This is an ego like I said Eagle Racing R31 But and I have changed it around and modified it a little bit Because originally the Eagle Racing R31 the motor sits up front So I have completely switched it with the help of my brother. We came away to make this happen They do sell kit for but this is kind of our own custom little look We have moved the motor to the back of the car as you can see the motor is put in the rear We have a belt drive system with a tensioner and everything that we have set up in the rear of the car We are running a hobby wings eight and a half turn motor, but we do have a speed passion 13 and a half Which I will probably switch into this car because it was a little bit too quick for what I wanted it to do hobby wings ESC and We're running a massive amount of steering angle as you can see this thing will just go pretty much 90 degrees lock and It's just an insane car to drive So it is set very soft in the front Which is another big modification if you haven't noticed this whole front end was designed and Kind of changed by my brother and I this is not how it's supposed to be with all the The way the shocks are mounted and the plates in the front and stuff like that we did change that all around to kind of Make it my own chassis in a sense with the motor in the back There was no real way to make with what I had to build it. So that's why we built it this way It is set pretty soft in the front very spongy set up for drifting which is definitely helps in the front There's a lot of weight in the rear. So it's kind of stiff. I do need to change that up a little bit But this is just one of the chassis I run I have a couple more chassis that I run just want to give you a little build breakdown on this one This is like I said all these parts are you go racing are 31 There's blue and gold obviously on it because I had you know, mixed max parts from other other other cars so but they are all from the Eagle racing. It was a gold one and a Blue one so we are running to me a belts with our own Tensioner setup in the back if you can't really see it back here Our own tensioner setup right here so that I can use different kinds of belts just my sliding attention is around so I have some clips from a Drifting and stuff like that that I want to bring to the channel I want to show you guys a little bit of the chassis that I run because I know a lot of people been asking So this is my Ruble Drive R31 Ruble rear engine converted Fully custom front shock towers and everything on those lines Very good chassis for me. I've used it a lot in Florida when we were in there making it You know one of my favorite chassis That I've had I did get off a buddy and we did modify it. It was originally an all-wheel drive chassis So this is what we're at on this one. So I do have another chassis that I want to show you guys It's actually my fiance's chassis. So I'm gonna show you guys that real quick Give you the breakdown of that chassis as well So this chassis is a little bit different as you can see it is a beater body basher RX 7 But it is because she's still learning how to drive so this is the other chassis that we run and Doesn't have wheels on this side because I just pulled it out of the case. So we're gonna pull this body off So this is a pretty much out of the box FXX MST Ruble Drive with a linear front motor setup as you can see with the sideways motor in the front unlike the traditional FXX which had the lateral motor In there. This is a basically a simple gear driven FXX drive shaft to rear wheel drive running a full speed passion Revington R with a 13 and a half Speed passion competition front motor and a simple servo nothing major. This car does not get as much lock as My equal racing does but it gets enough lock to get the job done I had a lot of fun driving this car the other day because mine was acting up a little bit so spend a little time driving this car getting it dialed in and Had a little fun with this one definitely got to do some suspension work on it make it a little bit softer because it's a Pretty stiff setup right now, but so I just want to give you guys a little look at my RC jasses here and If you guys want to keep seeing RC on the channel, let me know down in the comment section down below I did go the other the other day to a Friends basement where he has a track setup. So we will be running more So we'll have some content to bring you guys to the channel. So like if you guys want to keep seeing RC stuff Let me know down in the comment section now below. I did want to show you guys a little look into my chassis and stuff like that These are the ones that I run as I bash my camera all of them are rule drive all of them are gyro built-in That one's the FXX is running a basically a cheapy little gyro from RC Mart Whereas this one is running the gyro through the receiver or I run a Futaba helicopter gyro to keep this car from Snapping back and forth and a lot of RC cars Well, obviously all rule drive cars run gyros depends on what the settings are some of them Some people are running without a gyros those guys are definitely a high par with the rule drive cars for sure So just a little glimpse at my chassis here today for you guys a little something different on the channel Let me know if you guys like it if you guys want to keep seeing RC stuff on the channel Definitely would like to do some build on some new bodies that I got lying around as well as modifying these cars and well Of course videos of them shredding and grabbing doors and tandeming with some of my friends So as always make sure you guys follow me on Instagram and Twitter all but you found in the description box below as always I thank you guys for watching. I'm gonna leave you guys with a few clips from the other day I did mount a camera on top of my FXX chassis and ripped it around a little bit in the basement. So as always I will see you


What is up Marvel Squad and welcome back to our channel. If you guys are new, make sure to hit that subscribe button because you don't want to miss out on any of these videos because today I'm doing something that has never been done before on this channel and it's a little bit risky. It's super exciting and I'm so excited to see how good this challenge goes. Well I don't know if it's like a prank or, I think it's like a challenge. I'm going to be doing wearing a wig in public to see if my boyfriend notices and the wig that I have is very long and thick and it is blonde, but it's not really like my blonde. It's like, I'll show you guys, but guys, this is going to be a crazy journey because I'm going to get my camera guy, Danny. He's coming with me. We're going to be sneaking up on him while he's shopping because he told me he had to run a lot of errands today. We always have our locations on if anything bad were to happen to each other, which is perfect because then I can stalk him and go wherever he's at and sneak up on him and see if he, you know, recognizes me, which I don't think he will. Look, literally I am wearing all black, under disguised. I don't really know if this is really disguised, but you know, he's never seen me wear this before. So I think that'll be good. We're going to walk past him and go in the same aisles as him and like see if he can recognize me, which I don't think he will, but then I think at the end, I think I might do like some flirting and be like, hey, like you got a girlfriend. I don't know, we'll figure that out later. But anyways guys, make sure to give this video a big thumbs up and make sure you hit that subscribe button and turn your post notifications because you don't want to miss out. And any of our videos, because we are about to make some crazy content this summer and I'm super, super excited. More vlogs coming. I really, I kind of think this is more of like a vlog, but yeah. I do have to give the shout out to the person, to the couple that I saw do this challenge. Jake and Kenzie, just shout out to you guys. I saw the video and I was like, man, I want to test my boyfriend to see if he recognizes me. So it's like perfect. My friend Ally is going to come in and put this wig on. Should I get my hat from downstairs in my car? Let's see how it looks first. Okay. All right guys, so we're going to get onto happening on my wig and disguising myself because I'm super excited, but this wig is super thick and crazy and wild. So wish me luck. You guys want to see the wig looks like? Look at how thick and long it is. Like it's down to my butt. And it's like an ombré. I know this is like blonde like me, but the blonde, you can't really see it in through the camera, but in person it's more yellow and it's got like a dark, dark root. And I don't have long hair like this. I have pretty, I have like medium hair as you could say. So this, this is about to be a different story.  Jesus. It doesn't even fit in the camera. It doesn't even fit in the camera. I wasn't going to get like a pink wig on Amazon, but how would I do that? I could do this. Okay, that's probably as low as I'm going to get. Yeah. Okay. Oh my God. It is pretty odd that we have it on. The hair stylist, so. I've never really messed with the legs much, but. Okay, hold it. Like, well, I pushed it down. It makes my face look really oval-y. But guys, you guys. Or even like a headband. Like if you just, what is the winner? Oh my God, I have to wear this all night. This is, I might braid it honestly, because it's really thick. Yeah. I feel like a hat is just, a hat doesn't have to do. Yeah, probably. So I'll have to run downstairs and get a hat.  Yeah, you want me to go grab it? It's in my car in the back seats in the garage. Okay, okay.  I don't think he's gonna notice. Like my face is already really, really different. Like, throw it out. Ew. Ew, it makes me look so ugly. Me, I wish we could have like bangs. That would have been perfect. So we're gonna have to style this now.  And I would not have a big head. 12 seconds later. Can you see the bump in the back of my head? Kind of, not really. Can you tell it's away from the back?  You got me from the back? No, no, no, no, no, no. Yeah, everything. Straighten these bad boys. All right. They even look good down. Right, like if we straightened it down, maybe it would look better. Maybe it'll look better, yeah. Yeah, okay. This honestly doesn't really look like me. No, it doesn't. Look it, from the side in here? Yeah. Oh yeah, no, you can't even tell it to you.  We'll be back once we done straightening this hair because this is a while. It's a while. Guys, it's done. Like look how legit it is. Like, okay, you can't even tell from the side. Yeah, no. He gets off work actually in like a couple minutes. So I will see you guys when we go pick up Danny and we go into the store that he's at. And this is gonna be super, super fun and I'm so excited. So see you guys soon. All right guys, so just picked up Danny. This is our friend that's behind the camera. You know, we revealed him kind of early. We're gonna do like a suspense thing, but I already named his name in the beginning of the video. So you guys already knew. Bill just told me he was on his way to the store. I talked in a day and he said he had to get tools. I don't know why he's getting tools. But he told me that he was getting tools at Harbor Freight or something like that. I'd been there before. So I'm gonna kind of just track him and see where he is. I don't wanna pull in the same time he pulls in because I kind of want to park like a way. All right guys, so he's actually almost there and we're gonna sneak up on him basically like stalk him, you ready? Are you excited? Yeah.  Guys, I don't look like me. Like look it. Who are you? Not Mariah, this is, what should I name? What should be my new name today? Margarita. Stephanie. Stephanie, ooh, ooh. So my new name today is Stephanie. It's not Mariah, Bill anymore. It's Bill and Stephanie. So. Bill and Steph. Bill and Steph, yeah. Anyways guys, we'll see you guys when we get to our first destination and hopefully it doesn't fail the first time we go in but we're gonna do our best. So we will see you guys when we get there. All right guys, so we made it to our first destination. We're gonna walk in right now and we're gonna try to find him so we have to be very, very, very sneaky. So let's go. I'm nervous. So we're gonna try to find him right now. I don't know where he's at but it calls us to get here so far. So let's try.  Oh no, because you have to wait for the next door. Next door, okay? So. We're gonna run. It's good over there. Cause he's focused on the tools. It's a man in his dreams. My heart's beating. He was right next to me. Okay, come on, quick, quick, we gotta go. He passed me and goes, I'm sorry. I went like this, looked like this. And what, did you bump into him or what? No, because he knows what he did. He turned, I was walking behind him. Like he was going straight. I thought I was just gonna follow behind him and he turned around and I'm like, he's like, sorry, he almost bumped into me. Guys, that was a good one. Like that was good. That was a good first start. Cause he didn't see me, no spine on him. I'm really gonna show him all these clips. He's gonna be like, what the heck? He were by me the whole time. All right guys, I'm currently sweating right now because his wig is literally like 100 pounds of hair. I'm being a little dramatic but it literally feels like a hundred pounds of hair. We're gonna call Bill. Guys, that was a good start. He had no idea. He literally walked past me and said, sorry. That's what he said, sorry. But he almost bumped into Danny too. It was just crazy. That was a crazy experience. I don't know how much longer I'm gonna take this cause I'm sweating and my car doesn't have air conditioning right now, it sucks, but we're gonna give him, oh, put it on my face. We're gonna give him a call and see what he's doing. What are you doing? Well, when should I come over? Like what other shores are you going to? I was just wondering what you were doing. Danny said you wanted to hang with him. Did you want to hang with him instead? Well, how much longer are you gonna be at Harbor Freight? How about Don? I'll just bring you. Hurry up a little bit, geez. I know. Sorry, all right, you're done, okay? Oh, okay. All right, love you. Thank you. Okay, bye. And we'll have to see him show up. So we will see you guys when we get to our second destination. All right guys, so we made it to our second destination, which is Coles. So we're gonna go in there and be sneaky and find him. I'm gonna bump into him and be like, oh, this is about to be crazy. You're like, dang, who's that cute girl? Did you say that?   Yeah, it's the wig. What is this? It's the wig. And you're filming, Danny. It's the wig. When'd you guys read up? When'd you do all this? When did, what did? So I came over, put the wig on me and got the wig ready, and then I picked up the wig. You have never worn anything like that before in your life. Oh, the skirt. I saw your shoes, I would have known right away, but I did not look at it. I wasn't even looking around. I know, I know. I was purposely like, like... So you actually been, that's scary. I know, creepy, right? No, because anyone can do that.  I'm like, I'm tweaking right now. Like, this is actually, because I thought I was actually having a date with myself. Yeah, you don't have to do so. Maybe you were hanging out with you while you were hanging out. Oh my. No, no, no, don't do that, don't do that. Who's this cute girl? You should wear wigs more, you know that? Hey girl, isn't it cute? My name's Steph, my name's Danny. Oh, well, I'm happy to see you Danny. I guess we're not hanging out alone tonight or whatever. I guess now we're all hanging together. So you were texting me, were you with her already? What time did you go over there? He purposely texted you just to get like, to make it not seem like he was with me. Anything was going on. Why would I even think he was... Yeah, good point. How'd you check my locations or something? Yeah, I was checking. Obviously we had a location. I told you where I was going. I know what. I wonder why you're asking so many dumb-ass questions. They were like, why are you asking so many questions? No, literally. I don't know, I'm just curious. I was curious, I know. I see the videos wearing a wig in public, with my boyfriend on his face. And who made this? Us? Another couple. I shot him out the beginning of Jake and Ken doing this. Oh, you got it from Tom? Yeah. That's funny though. That's actually very funny. All right, so I had to make them turn back on the camera really quick. I just needed a moment to process what just happened and you guys are like, scary. That was scary. Just think, that could have been like, a guy plotting my death. Like a murderer. You could have been someone like, hated me. That's why you have to be cautious of your surroundings at all times. Guys, and girls, whoever you are watching this video, Marvel Squad, be careful of your surroundings because that just shows. I've been dating her for two years and I didn't even notice her next to me. You guys are horrible. Is this like a prank or something? Like a challenge? There's more like a challenge if you would say. This is like a prank slash challenge. Everyone should be doing this. This is like an awesome video. This is fun. I'm gonna put like, cutting my hair.  See if my girlfriend notices. I didn't even know we were dressed in for camera training. Yeah, you look cute today. Do I? I tried to style my hair with my pants. I think it came out pretty well. Or you can end this video because this is your video. So go ahead. Anyways guys, so I think this video went amazing, especially even after I bumped into him. And it was just funny because the reaction was priceless. Yeah, I'm sure it was. All right guys, so we're gonna end this video with our end of the video. Post of the video. Shout out. That goes to Natasha, period, Matthew Lee Ode. We love you so much. Thank you for bringing part of our channel to Natasha. I love the name. Anyways guys, so we're gonna end this video here. Yes, we are. So I can start getting followed. Yeah. Like a creep. You're classified as creep. I am. Creep. We will see you guys in the next video. But until then, peace.


So this is basically the first unit of more substantial content that we would like to share with you at this conference and as it is a keynote, it continues this theme of setting the town for the conference. So, yeah, it's more concrete. But it's still also a bit high level, so to speak. And what we would like to do in about the next hour with you is to take you on a journey together with our favorite mascot.  So, how have we worked with information historically, how can we contextualize the work with transcribers within history. And then also, after we have looked at how transcribers itself contributes to this journey. So those are basically the sites that we're going to see so the things that you can do in terms of information extraction with transcribers. We're also going to take a look into the future of how we work with information as humans. So, the keynote is entitled Unlocking Historical Value, Information Extraction with Transcribers. And yeah, will be delivered by Michałusz Toszewski, our head of R&D, and yours truly. So, packing our bags, let's get a few things straight. And it is good to start with some definitions that we all know what we're talking about that we're on the same page basically to use a historical documents metaphor. Information, memory, and culture, what role do they play for transcribers and for how we have been dealing with information throughout history. What is information anyway. I've already said a few words about it. It's when we structure things when we put them together in a meaningful way. But they also have a somewhat mechanical component, or we can define them in a pretty sharp way, but maybe also one that's general enough to be useful for all the different use cases that we're coming from. So, information, whoops, exited full screen. I don't know, do I need to snap going back to the, just a second? No, it's back. So, yeah, information is things that are distinguishable. Look at these two pictures, which one do you think contains more information? I would say it's the one on the left. So it has more complexity. It has more meaning than the one on the left where we have two identical instances of this coin, which is a historical coin. By the way, around 100 BC from Ephesus in the Roman Empire. Nowadays, Turkey. So things that we can distinguish between, I think that's a very general but useful definition for, yeah, talking about information. Memory, that's another important concept because many of you are part of memory institutions, museums, archives, libraries, parts of universities are also devoted to memory. So, remembering stuff. Take a look at this picture and try to use your memory. And then a couple of seconds. Yeah, we're going to do a little experiment. Everyone ready? So now the image is gone. And now let's see who can remember how many objects were in this image. So I'm not asking what the, there's always one. Yeah. Great, great memory and great capacity for taking a look at things in a very general way and a general purpose way, which is very important also for historians, by the way. So being able to take a step back and look at things from different angles and not just the angle that we're used to. What I wanted to show is basically that it's very important what we're focusing on when we extract information and when we try to remember stuff. So, yeah, here again, for those who want to count, if you didn't believe me, it's exactly 14. Well done. Well, memory has a cultural component. So it depends on what the lens is through which we're looking at history and how we're extracting information from data. There's always bias in there bias that we need to reduce and keep to a minimum, which is not easy. And especially culture is something. Yeah, that tries to do its best to prevent us from seeing things objectively, if there is such a thing as objectivity, which is not a given at least in the humanities. So the host ended this meeting it says here. And the clicker isn't working anymore. So let's see if we can fix this like this. Yeah. What is culture. Another little experiment. What color is each tractor wants to answer. What, what are these two colors. Anyone. How about Mr 14. No. I think most of us here would probably say blue and red. But that's sorry. Oh, yeah. Oh, yeah, yeah, you mean special like color technology that farmers use. Yeah. No, it's it's simpler than that. The thing is that not in all cultures, those would be as clear cut as blue and red and Russian for example, there are several different words from blue that are used a lot and not words like light blue, dark blue that are derived from color so that have a certain logic, but words that are completely different. For example, Goluboy and senior. So those are so the color scale and Russian. Yeah, is divided into basically, and you have to sort of larger groups of how you would call a certain shade of blue. And on the left, the right one I just produced myself just recolored it. The left one is called senior tractor. This is a very beloved children show on YouTube, that is about a dark blue tractor. As you can see, many of us will probably wouldn't call it dark blue, which is the closest translation of senior, because it's very close to this line, but Russians are very good at distinguishing between such very close lying shades of blue. So it depends very much on how we're trained how we're brought up. And yeah, what the goggles are on our faces that we're using to look at history and information. And to now to define those three terms information stored distinguishables basically stuff that we put somewhere. And that has meaning because we can distinguish between the various elements of them. Memory, that's how we store and retrieve it, which as you've seen has a strong cultural component. And in the end culture, which is exactly that component that determines how we look at things, and that we also need to try to take keep to a minimum as scholars. So yeah, let's take a quick look at what this all means. And how this. Yeah, how we have been dealing with information throughout history. So the first part of the journey. And you can see me as a bit of your tour guide is the sequence of information ages. I tried to make a small typology of, yeah, the basically major eras throughout history, where things changed in a major way regarding to how we deal with information. First, the zero earth age, basically so stepping back before we're even going to discuss the first age, the age of biological memory. So we are at least according to modern science animals. And before we, yeah, gained the things that we now they use to distinguish ourselves from the other animals. Yeah, yeah, we were pretty close and pretty similar to them. So how we remembered stuff and how we stored stuff was basically the same as your dog or cat cat does it in your biological brain. We used this memory through our senses and thought already so thought can also happen without higher functions as there are often called in biology. And the rules. Yeah, we're also determined by thought. So if we thought something was dangerous or not. Yeah, that happened all inside our brains. What was this going on until about 600 to 200,000 years ago. Yeah, historians linguists biologists aren't quite sure, often fighting about it a little bit. But we have a number of things that we can use to make this estimate. The final is that we share one of the most important things that determined what came next with our, yeah, cousins that aren't around anymore and unfortunately the Neanderthals, we share a biological trait. The next big thing which came around language must have evolved sometime before we and in the end the falls split at least from a biological perspective. Yeah, and this takes us to the first age of information the age of language. We still were using our brain, mainly senses and thought we're still used for access. Language developed and the rules, the cultural sort of rule set was determined by this new fancy tool. And along with it art as well if we look at cave paintings. For example, these make higher brain functions necessary. This happened after the point that I mentioned before so after 600 to 200,000 years ago, until about 8000 years ago, and the main thing that changed was our biology. We developed this insanely complicated and fine tunable and fine tuned vocal track that we all have in our throats and faces, and that we use our lungs for to basically play bagpipe in a very fancy way. Yeah, and also the genes were obviously important for that as I explained, and we shared those with the Neanderthals so they were probably able to speak in a way that was quite similar to ours. What happened next. Once we had figured out language. Next thing that we figured out was how to put language outside of our bodies and permanently. We invented writing. And the new thing that we now had was graphical science, with which we could store this linguistic information outside of our bodies. We access those signs through reading and writing, writing systems genres, catalogs indices. So basically everything that we later used for thousands of years was now possible. This started around 8000 years ago. So this Yahoo script was generally or is generally accepted as one of the first instances of human writing. Next thing. We were able to put with linguistic linguistic information and abstract thought outside of our bodies was to process this outside of our bodies as well, because a written page isn't able to process information it's just information that's there. This started roughly in the early 17th century. In good tradition, the Germans and the French are fighting over who actually invented this who invented the first calculating machines. The Germans say it was them because the first schematic that you can see here was made by a German will have chicken. The French say no it was us because we actually built it first. So, yeah, the age old fight between idea and implementation. Once we had that figured out. We went back a step and enabled the machines to process information in the way that we usually communicate information with other humans. So we learned to teach the computers to understand us again so that's something that they couldn't do yet. And to talk in them in a basically non formal or formalistic way. And the first major event here that we can quote is the development of baseball, a question answering system which was developed as early as the 1960s. That was able to answer questions about baseball. So pretty narrowed down field, but still, yeah, very powerful for its time. So yeah, that's basically how we got here we had different ages where we learned more and more things that were useful for processing information. And now let's take a look at how we process information that comes from historic historical documents. And I'm giving the floor to you, Michael. For the next chapters. Yeah. We aim to structure data, which means we want to extract information from data. And on the surface we are dealing with transcripts with text, and transcripts has many features that allow us analyzing text to get from raw data to information. And the core features that enables us to achieve this task has been included in transcripts from the early days, namely the manual. You can also use tagging of entities or whatever spans of text you would like to annotate in the editor. And in addition, you can also provide entity linking features for example by adding wiki data IDs to individual entities that you previously manually annotated in the text. Once there is manually annotated data or ground truth in other words, you can try to train machine learning models. And that's why we have recently started working on automatic tagging components, which are not yet implemented in transcripts. But it's, as you will hear later in the next lecture on our roadmap, and we have been already developing automatic entities entity tagging components within the framework of managed project so we work with it it's not there yet for the community. And when I hear people talking about text tagging they often refer to names entity recognition this is one of the most prominent use cases. But we are actually working on a generalization of names entity recognition we want to provide. We have trainable models that can deal with arbitrary user defined tag sets not only with names place names organization names or dates which is the typical tag set in the entity recognition. And the way to achieve this is for example by means of supervised models that are being fine tuned based on pre trained language models like Bert, for example. And we've provided here a few examples of automatic text tagging from a project that we are working on with the museum for nature kunde in Berlin, which they are also members of the cooperative. And together we are working on the workflow for the transcription and entity extraction of insects specimen labels this is a very specific tag set that is needed for those labels. For example, on the first label and name of an insect species followed by the symbol denominating the sex the biological sex of the animal plus the name of the researcher the biologist who described the species. We can also see the transcripts generated for those label cards and the tags highlighted by colors. And with those, yeah, structured tags we can actually extract this information put it for example into spreadsheet or ingested into a database. So this is one of the major steps that is necessary from going from raw data to structured information. Another component that is essential is entity linking. We are also working on automatic approaches to assign. Yeah, unique identifiers from external knowledge basis to entities recognized within spans of text. The entity here are again a few of those insect specimen labels on the first label on the top left on the entity for most has been identified by the entity tagger and by looking up the respective concept or entity in the geo names or wiki data database. The tagger came up with the unique identifier of this entity, and this is very interesting because for most eyes nowadays, Taiwan. So irrespective of the wording of the name used for an entity, you can still relate to mentions of text to the same concept in the real world. The third example is here in the second and the third label we have two types to name to and mentions I'm sorry of the same insect species that three boxes on front the corner. And the second on the third example only from the corner is mentioned but still the knowledgeable researcher knows that those two mentions refer to the same species which can be identified with this cryptic code in the catalog of life. As you can see here are examples of data normalization irrespective the surface form of the entity they refer to the same concept and now things are getting really interesting because you can take the step from information to knowledge by linking concepts to databases with which expert knowledge to describe entities and their characteristics in the real world. Here you can see, for example, a screenshot from the catalog of life where you can see not only the scientific name of the species three boxes on front the corner but also its taxonomic rank, and it's geographic distribution. So, with this information, for example, you can make more in depth analysis you can interpret data and, most importantly, thanks to those unique identifiers, you make your data interoperable you can connect it with other collections. And apart from extra analysis. Yeah, I just wanted to take over for a minute again as the tour guide, because I actually put the quotes there sorry for missing the first one. But I think that in the beginning was the word we all know that that was from the Bible. That was basically the introduction to what we can do with text in terms of information extraction. And the next sites that I wanted to introduce and to let our local expert show so you can basically pick picture me as the tour guide on the bus and Michael as the local who are going who is going to show you the pyramids of Giza. And the next chapter here is visual analysis because after picking apart text. Another thing that we can do aside from that is taking a look at the document and just gauging by the way it is structured visually. So what pieces of information are stored inside of that document. And the quote I think is a really good description of what we're trying to achieve with the new components that we're providing here, because the real voyage of discovery consists not in seeking new landscapes but in having new eyes and so new ways of looking at the documents is what we're trying to provide among other things with visual analysis. Back to you. Yeah, in order to understand how historical or modern documents can be visually analyzed we need a brief refresher of some major concepts in computer vision from the field of object detection. The task of semantic segmentation which is illustrated on this slide consists in determining for each pixel in the image to which class. The pixel belongs and in one image there can be multiple classes like in this image for example there is the class of beach shore mean shore sea sky and persons. And each individual instance of a person is assigned a specific ID denominating that person class so you cannot identify individual instances of persons you just know where all elements belonging to the person class can be located in this matrix of pixels which is an image. Instant segmentation takes a slightly different approach or yeah addresses a different challenge. In instant segmentation the goal is to identify individual objects of one particular class, for example here in this image we ignore all background background is any pixel is not a person. And within the person class we try to identify individual instances persons. And there is also an optic segmentation, which is basically a combination of the two previous tasks and optic segmentation tries to identify on classes. The pixel classes in an image plus in addition to assign unique IDs to all instances of a particular class so for example here we can have the shore to see the sky class. And within the person class we see there are three individuals. So, we basically merge an optic and instance and I'm sorry, instance and semantic segmentation. And what it has to do with transcribers can be explained easily. We have the field model now the field models now integrated in the beta platform, and they both support they support both panoptic and instance segmentation. A page processed with field models can be seen here, we see the raw image and now the overlay of three different fields, one identifying the name of a person in this index card one identifying the place, probably the birthplace and the year of birth. So, this structures this cloud of pixels by highlighting areas that have some semantic feature in common. Those field models are also trainable, which is major, yeah, major, yeah, feature of the whole platform that you can adopt your models to your materials to your specific tasks. And with those trainable capabilities, new ways of processing documents arise. Here is an example of segmenting the layout of newspapers by identifying heading fields versus paragraph fields versus separators. So, if you as a researcher interested only for example in the headings for a quick analysis then this tool helps you to pick those elements that are really relevant for your analysis. You can use field models to identify particular pieces of information in such complexity structure documents as this form or an index. Again, this is a quite complex tax set there around 15 different fields, all of which have a separate meaning and yeah the trainable component allows you to apply those structures to arbitrary large collection. Field models can be also used to handle a particularly difficult layout type multi column documents without field models probably standard baseline detection would just draw one line across the whole page, which of course is useless for many types of analysis and with field models you can first separate the layout into individual columns and then run baseline detection with each in column, thus preventing the model from merging undesiredly baselines. Taking field models one step further, we now have also made available table models, table models allow us to deal with complex highly condensed information in tabular structures by overlaying a grid over the image to highlight the rose and the columns. Table models are also trainable. And how does the training and recognition work well basically it's a two step field recognition in a first step individual rows of a table are identified. And in another step, the columns of a table are identified and by intersecting the rose and the columns model. You can come up come up with a grid that can be put over the tabular structure and the great feature or the great characteristic of this table models is that you do not necessarily need visual separators like horizontal or vertical lines. Just some gaps space between the text items can already serve for the model as a seclude to separate columns and rose from each other. Yeah, here we see a grid that is fairly straightforward, but there can be even more complex table layouts for example multi line rows where each row or each record consists of one to many rows of individual text. And yeah, with the help of this procedure, we can structure your document neatly into table and columns. The use case for field models is the labeling of very specific idiosyncratic particular types of content. In this example we see. Yeah, music note sheets were the field model was capable of detecting the area of image containing the musical notation, another field for the text tool part and another field for any illustrations or initials. Yeah, so everybody back on the bus. Hope we didn't lose anyone along the way. Yeah, one next component that is one of the sites on our tour today is large language models. And here I have found another very beautiful quote, the art and signs of asking questions is the source of all knowledge. And that's exactly what we can do with large language models we can ask them questions about our documents basically and it's about. Yeah, how we talk to them. Yeah, this determines how well they work. So prompt engineering plays an important role here. And let's take a look at, yeah, what we've been up to in terms of language, large language models and what our ideas around them are regarding transcripts. When I showed you the slides that demonstrated automatic text tagging. I demonstrated the use case of supervised. And, yeah, information extraction, but large language models, as they have become prominent with open eyes developments and so on, allow us to extract information from documents without training data so just by leveraging the linguistic and word knowledge contained in those large and heavily trained models. And the chart here highlights a workflow that we experimented with. We made experiments to use particularly designed prompts together with some input texts that were fed to chat GPT. The main was to extract structured information by annotating the documents in question with, yeah, the named entity or entity classes that we were interested in. And yeah, the most important part of this chart is actually that there is no training data contained in those symbols so it's really promising to work with documents without having the necessity of creating ground truth. So we made experiments with two types of entity schema definitions. On the left hand side, we made experiments with a user defined entity schema where the user tells the large language model which type of entities he or she would like to extract so you could just prompt the model to extract name, location, or date, for example, and also use more complex schemas that you first need to define to the large language model. And what we can see here is an example again from the MFN insect specimen labels. So, yeah, for example, Senulus Atratus was correctly tagged as specimen. And yeah, the, the collector of the specimen Blutgen leg, for example, was also correctly identified in the open-ended schema, however, that is shown on the right hand side. The user does not provide explicit definitions of each entity category but asks the model, the large language model to infer autonomously what kind of entities are contained in the collection based on a sample corpus fed into the model. And the results were quite astonishing. For example, the Senulus Atratus was identified by the model as species. So it is exactly what we were interested in. Some deviations from the user-defined schema, for example, in the second line, Blutgen leg was split up into two tags, person and collector suffix leg, which makes perfect sense. It was just the user-defined schema that wanted both words to be tagged within one span. So it's basically a viable approach if you want to explore your collection, if you want to get a basic understanding of what is contained in a set of documents. The sample below shows a slightly more straightforward application. We made some benchmarking tests on the Hype 2020 data set, historical newspaper, entity processing, shared task data set that contains named entity markup. And the model was also capable of understanding or extracting quite robustly those entities. We compared the supervised tagging approach, the unsupervised one, and on a very high level we found the following insights. Supervised models basically achieved best accuracy in terms of precision recall and F1 scores on entity extraction. And unsupervised models, well, it's hard to tell whether they are much better, whether they are worse or much worse. In some cases, they achieved almost similar performance as the supervised models. In some setups, we achieved far worse results with unsupervised models. So the lesson to be learned here is whether accuracy can be achieved as high as with supervised models depends really on the collection in question. The tag set that you would like to extract and most importantly the model. So GPT-4, for example, performed much better than GPT-3.5 and so on. Both types supervised and unsupervised entity extraction is very useful for information extraction. So when your goal is only to come up with a structured, for example, spreadsheet that mentions all entities within a text without highlighting the exact span, the exact location of a given entity in a text. This is the use case you're interested in. So annotation within a transcript, then the unsupervised approach might be not the best choice. Supervised approaches tend to perform better here according to our experiments. But the last column highlights the benefit of large language model unsupervised information extraction. You don't need training data. We just provide a few examples, a few definitions of categories, and you get more or less good results. What we have observed is that models, the large language models sometimes tend to hallucinate. We all know this phenomenon. They tend to have a behavior that is difficult to control for us as developers or prompt engineers. And that's the reason why, yeah, annotation where the exact position of attack within a span of text is important didn't work that well. So we, for example, we saw that the models tended to over normalize historical texts by modernizing the input, which makes the approach not so useful for annotation. But we had some we have some ideas how we could mitigate those. Yeah, difficult to control stochastic behavior of large language models. Our idea would be to combine the best of two words. We could use a large language model for the production of ground truth. And this hypothesis ground truth, let's call it that way generated by a large language model could be then reviewed by an expert on the domain. And this reviewed, yeah, entity extractions could be then used to train a supervised lightweight model, for example, a small bird model for entity extraction. And if the use case is just getting an extract, rather than a position exact transcript, you could basically also consider going directly from the large language model output to a spreadsheet format to a structured format. Yeah. Okay, so I hope everyone like this site. We're getting back on the bus again, picking everybody up again, and are moving to the next stop, which is multimodal approaches to information extraction. And here again, this favorite quote of mine, when you need to innovate you need collaboration and collaboration in this case means collaboration between machine components. So it's not just about collaborating as humans, but also putting the right machine approaches together. And this is basically what multimodal approaches are about, where you combine algorithms that can do different things well to achieve the best overall result. Back to you, Michel. Basically, we made some development tasks, development and experiments in the hybrid connection of textual and visual information, textual information gathered from transcripts and applying entity recognition, visual information gathered from field segmentation and field models, for example. In this particular example, we applied first table recognition. And on top of the table recognition we applied entity extraction with a custom trained model. The goal here is to identify in those columns where the patients where the doctor's name is. And to detect where the cause of death, for example, is located. So particularly tricky is the third column where cause of death and name of the doctor is mixed in an ideal table. This would be separate columns, but this is not often that sometimes not the case in historical documents. So we trained a model that was able to encode the position of a word within the table. So it's row and its column. And to detect within those individual cells, which string, yeah, denominated the doctor, the patient, the cause of death, for example. So this is an interesting case because usually entity recognition models require context, which is not really present in this table. So we included the context by the positional information within the table. Another use case of hybrid connection of visual and layout information is X classification rather than entity recognition applied on top of layout segmentation. So first we apply here a field model to detect the paragraphs of a newspaper layout. And then a text classification model can be used to categorize the semantic content of each paragraph, for example, the first paragraph that's a made up example could be labeled as economy, the second paragraph as culture. So a model that only deals that only processes visual features would be hardly able to make this fine grained semantic distinction so you really need to get into the content to analyze its semantics. But those examples that you have seen before are kind of a workaround we combined our tools that we already have for visual processing and our tools that we are developing for textual processing. But the future probably lies in approaches that merge both words. So in multimodal approaches that really genuinely try to jointly learn within one single model the visual and the textual features of documents. And on the left hand side of this slide you can see like the workflow that we apply currently we have an image we segment its layout. We run HDR on this document within this layout structures, and then we can try to extract entities from the transcript and the important part of this left hand side workflow is that the entity extraction only has access to the transcript so any layout information any typographic information whether we are dealing with bold or italic font, whether the whether where the position of a given word on the pages all this is neglected by the model. Multimodal approaches however they are able of focusing both on the text and the position typography layout and so on, of individual words on the page to come up with a more robust with a more informed representation of entities. And multimodal approaches even open up avenues to, yeah, to end to end processing where no individual layout recognition and no individual X recognition component would be required but that can come up directly from an image to the extracted list of entities. All this new possibilities of technology should also force us to rethink how we currently do things. For example, how we are used to represent documents in transcripts. Let me explain you this with two examples. We have seen an increasing number of projects or problems that actually are interested in processing not individual pages but entire documents. So here we would need to go from a page to a document and figure out how to correctly represented in a robust way. In this example we have two pages on the text region on the bottom of the first page is continued on that top in the first region of the second page. And yeah, with those new, especially multimodal approaches we could actually learn that those two text regions across two different pages are related to each other, but we need to think clearly about how to represent this properly in our data schema one simple idea would be for example to add a continued is true tag in the text region markup of the page XML file for the second page, but is this really the best approach. That's an open question and I think we have a nice opportunity with this conference to discuss such issues. Another rethinking is whether we should find new ways of interaction between users and documents and how information is presented to users. We are talking in this lecture about information extraction and when you're interested in getting a structured extract from a document you often do not really care about any word that is not to be put into a spreadsheet column. However, a transcript verbatim transcribes every single letter every single character on the page. So maybe a better representation for information extraction would be a table that shows you in columns, all entities of a given class that have been detected on the page. But actually, if you really want to, for example, correct or verify the results of an automatic procedure you also would need some link between an extracted entity in a table table structure and its position in the image so maybe we should rethink ways of representing data if we are to include more information extraction features in the future. Thanks a lot. Yeah, so we are reaching the end of this little trip that we have been taking together. And yeah, once you have gone on a trip, you often get hungry for more thinking of the future and where you're headed next. And this is exactly what we're going to do for the next couple of moments, because the story of the ages of dealing with information isn't over yet. So we have reached a chapter in this story, but there's more to come obviously. And yeah, things are really heating up at the moment. We are at an incredibly exciting juncture in the history of processing information. And one of the next big things that everybody's talking about is a cultural singularity. And this would mark the beginning of the transhuman age of information processing. And yeah, this would change things again. For example, the technology of how we store things is changing. The new forms of just the actual medium where we put information to use it later on is changing. So the biology is coming back here as well. We are basically putting together machine components or what we traditionally consider to be machines with biological components, which enables us to access this information, again with our senses and send source as well, and thought. So the whole paradigm of how we access stored information will change drastically. So the rules of how we do this, they're also coming back to this thought pattern. So culture, we don't know what role it's going to play, because if we access information directly in our brains, or in machines that are connected to our brains or located inside our brains, what effect are the rule sets and the cultural components going to have, if you think of what's happened with social media alone. So all of that is still outside of our bodies. What's going to happen when those things enter our bodies, how are we going to process information once culture sort of determines how we think. Yeah, when is this going to happen, probably soon. And what will be the major changes here. One thing is true machine autonomy and we are already pretty close to that. The first step is here to outsource the decision making process to machines as well, not just providing a basis for decisions. It's a very good practical example autonomous driving so self driving cars. So far, machine learning algorithms have provided the basis for the decisions that classical rule based algorithms take later on. For example, traffic signs, the machine learning algorithm identifies a traffic sign, and the classic algorithm decides okay. When I see this traffic sign, then I need to stop the the stage that we're entering now is that we're also outsourcing this decision making process to a machine learning algorithm. I consider that machine learning algorithms or AI algorithms are not as reliable as most classic algorithms, then that opens up a lot of questions I think. So machine autonomy, then machine knowledge, I said connecting things and really doing things in the real world. That's what's not what knowledge is all about so putting the information together and doing something with it creating something new from it. That's something that machines are also increasingly learning to do. Now it's probably more. Yeah, that they are good at pretending to do this, but in the future, they will be much better at doing the actual thing. And the merging of humans and machines that I already alluded to will also be a very big topic here. And the question is, I've put two scenarios here which one is more likely, or if they're going to happen one after the other, to go back again to history, a lot of humanities scholars here. Are we going to see a pygmalion scenario where we basically create our own playmate who's as intelligent or even more and more intelligent as ourselves. Or is it more Harari's homo deus scenario where we merge with the machines ourselves and sort of become gods. I think are very exciting thoughts, maybe a bit scary thoughts, but I think they concern us all. And we're going to try to keep it real, as they say with transcribers and do the next logical things, which is extract information and putting it to good use together before everything explodes. So thanks a lot.  And we have a couple of minutes for questions. So if you would like to know any more details about what we have shown you on our little journey today, or have any input then please speak up. I'm going around the mic so I'm giving you the mic in the meantime there's one question for Michael maybe what software are we using for an year and that was online. Thank you. So the question was what software we are using for named entity recognition. The implementation that we have currently developed is basically a wrapper around the Transformers library from hugging face where we can download pre trained models. In the examples that we have worked on so far we used the distilled bird model the multilingual one, because we were dealing with fairly modern varieties of language so quite modern German and English. So we didn't have the need to train or domain adopt them the models to historical language variants. So yeah, transformer based hugging face implementation. So if I may. Thank you very much for the presentation I would like to ask about the tables what about if the information is written knit, not correctly in the fields you know it starts in one field, and then it goes like to the other field. It's very typical for many older tables that they are not very well confined. Yeah, I mean, this is indeed a frequent case and we can also try to capture this with help of baseline models for example that you can train on top of field of the table model to learn this. I think the multimodeler in a first step, the hybrid approaches that we've shown could also provide highly useful for this because you're doing the same as a human you realize. Okay, this word doesn't belong in that cell. So, when you also process semantic information, along with the structural information then that could provide the necessary clues to put stuff in the right column. Basically, so this is something that we may work on in the future, I think anyone else. There's a question. Hi. Okay. Thanks for the trip. I have a methodological question in one of the last panel. You showed that the now the process of transcripts goes from the image and then to the fields then to the text and then to the final result. The next step is merging the two passage in the middle in one single process and future is that clockwork icon that you put on on screen. This is some kind of omnicomprensible step that makes all at once. Now, I was thinking about Interparis project of Luciana Durante in in the Canada University that is addressing the topic of losing losing control on what the AI do does to records. So I was thinking this is final step, maybe strongly recommended for our purposes, but doesn't put us at risk of losing control on what transcripts does to other records. I mean, this scenario that we pointed out is a potential avenue. It doesn't mean that we necessarily need to go there. But it could make workflow much easier. Of course, you do not have to train those individual models, but actually what I think would be the most meaningful approach here is to select to give users the choice which mode of modeling information they would like to use for their particular use case. It doesn't mean that if those end to end multimodal approaches work well, we would replace everything else. This would be just an additional possibility for particular use cases where the user should decide whether this is useful or not. And one other thing Gunther already stressed it in his presentation or his introduction. We still need to use our brains. So reviewing what the model output actually is. That's where the experts come in. So they need to be able to tell them if a model performs well and how reliable the the output is. The main thing here is that you have to consider how much more you can do with those models, even at the expense of a little bit of quality, what you can do quantitatively changes what you can do qualitatively because you can get out information from your documents that you couldn't have done in a hundred years, putting a hundred people to work on a certain collection for a hundred years, you couldn't have extracted as much information. As you can with, yeah, nowadays technology. And the question is how much quality or precision do you want to sacrifice for one thing. And the other thing is how reliable where the humans in the first place. That's also a good question. I think because especially in the historical disciplines. You read secondary literature and you quoted, then you're relying on basically, yeah, an algorithm as well that happened to be running in someone's head and not in on a server farm. Basically, so I think there are a lot of angles here and the most important thing is, yeah, look at all the fancy things that we can do, but be responsible about it. More questions for students and we need. Thank you. Thank you. Fantastic presentation. I'm, I'm just wondering whether, as you bring up, you know, the need for kind of human. Supervision interaction. You know, checking of these outputs, whether there is a kind of crucial user interface component to the whole cycle and ecosystem that you're imagining where. As the quantity and complexity of what is able to be produced by these models increases, we need increasingly powerful and adaptable kind of review tools that can take us quickly through and identify and help us to interactively kind of focus on the critical points that need to be checked by humans. So I wonder if that kind of UI is part of the plan. Very good point. And that's something that we're actually already working on. So quality control. That's one thing that's essential. And especially when it comes to training models. There are more and more studies surfacing that boil down to basically garbage in garbage out. And this is especially true for AI models. This is something that we may lose sight of a little bit with all the fancy things that LLMs, for example, are able to do. They provide or they produce very plausible looking output. But if the input isn't good, then the actual plausibility can be quite low and the output of the model can be bad. So being able to produce very good ground truth. That's something that we're focusing on strongly, which will then later make the output better just by that virtue. And on the other hand, sampling in general is something. Yeah, that's very important to us at the moment. So building better sampling tools for humans to review both output and ground truth as well. So quality is one of the main strategic topics for the cooperative this year as well. So we're very aware of this problem and we are actually building components that should help with this. Hi, I was thinking about your near experiments where you compared supervised to unsupervised methods.  Could you please repeat the last part of the question?  The only experiments we have done so far is with this historical newspaper collection, which spans from the late 18th century to the mid 20th century. And we have seen that, I mean, the biggest problem for us in this experiment was that, yes, I mentioned briefly the model attended to modernize old variants of spelling, for example. It also mod corrected OCR errors, which was kind of desirable. But all together the language variant, it was not that different from present day English. So, okay, for more historic for older variants, we have made no experiment so far. Yeah, yeah, that's an interesting case, definitely. And if you have any experiences, we would be glad to hear about your opinions on that. Okay. Thank you. But yeah, historical variants of languages do play a role. And we have some anecdotal evidence. For example, we've been working with Latin a little bit. And there are different variants of Latin throughout history. And it's, yeah, you can see that performance suffers when the model hasn't seen much of a certain variant. Yeah, so this comes down to fine tuning, basically, the model. Maybe one more question and then maybe a little break. Should do. Thanks for that. I wondered if you wanted to say anything about how we were thinking about environmental costs of the more compute that we're doing. I know we've got a plan for that, but I want to share that given we're talking about more compute. Yeah, there are actually different approaches that we're taking here. And one is we're trying to get hardware that's as efficient as possible, because there is quite a large margin of variation there, how efficient a GPU can run models. So this can be in the double digits percent region basically so 2030, even 40% of a difference, even of same generation models is possible. And the other thing is we are thinking of also working with workflow management a lot to string things together in the right way. So basically, as Michel has already shown, for example, using a large language model for creating ground truth, high quality ground truth pretty quickly. Because the large language model is very good at figuring out things, but it takes a lot of compute power. And then using the ground truth that you have produced with the smaller quantity of material with a more lightweight model. And this is both a good thing ecologically and for the users as well, because with the lightweight model, you can process things a lot faster. And this is both true for language models and handwritten text recognition, for example. So combining light weights and heavy weights in the right way is a very good approach here. And maybe if I might add the third point, and that is responsible usage. So we've seen that there were a lot of trainings that made maybe not too much sense. So we're also in the process of developing tools where you can see how much is my training actually doing. Because hitting a button also means that there's 500 hamsters running in your wheels. So we need to be aware of that actually costs something and that energy is consumed in that sense. So a responsible usage of that is also important. And we're also working on that because eventually it's us that's hitting those buttons, but those buttons have an effect. Yeah, we're also working.


or maybe you have to implement it. In this case, the pros are that you can, if you have the source code, you can potentially integrate it in any embedded device. And the cons are that you have to do it. I mean, it requires some integration work. And in some of the cases, the updates are limited to the application level. The examples here are the big guys, Azure, AWS, Google, and Hopbit. And, well, it has something common with the first one, and that is, well, as I was saying in the first group, that the cloud is strongly deciding what's running on the device. And we are going to try to avoid that. And then the third group is the agentless one, in which the server is in charge of communication with the device using SSH protocol. So the update manager here is the scripts that are sent from the cloud, and the cloud client is represented by the SSH server. Pros, the only dependency can be SSH. Maybe you need something else to do the installation itself. And the cons is that you don't get any feedback to the cloud because here the cloud is the master of the communication. Example, Ansible. So now, how can we make this more decoupled? First, I'm going to take a little bit to show how we are approaching the problem of reproducible states. For this, we use containers and a state JSON. The state JSON is just pointing to one to multiple containers, and each container has a root file system attached, so you are going to be able to know what's running on the device at any given time, and if you update it, you are going to know that it is going to be on the device after the update. Here you have the example of our state JSON. It has the VSP with just a bunch of binaries, the kernel image, the modules, the update manager, which is Pantaviser. And then we have one application, only one container.  So this is the diagram now, where I have divided the application level into a set of containers, and I have changed the name of the update manager to container orchestrator, because I think it's more appropriate for this example. So what's the container orchestrator? Well, it is the update manager, so it's in charge of installing the updates. But in this case, it's going to install and install containers and start and stop them. We have an implementation of this container orchestrator, which is Pantaviser. I have put the link in there, so you can check it out later. And now we are going to take advantage of the use of containers. Instead of having the Cloud Client and container orchestrator in the same base system with all their dependencies, we can move the Cloud Client to its home container. So this is what? Cloud Client. This Cloud Client has to communicate somehow with the container orchestrator. It has to control it. And for that, we are going to need an open update protocol. But first, I'm going to talk about a couple of examples that we have implemented. One is an Azure Device Update agent client. It supports updates from Azure IoT to devices that have the container orchestrator. You have the link in there too. And another example is our client, the Pantakor Hub client, which can be used as an example to other implementations because most of the Cloud providers offers an REST API. So you are going to have to implement a client for each one of them, but they are all going to look pretty much the same. And you have the source code also in there. I mean, if you... Because, well, I haven't prepared any demo for this presentation because there is no time. But if you want to see some demos, we are in a booth, booth 33 in level 4, so you can come later or tomorrow. I mean, we are going to be there all this week. We can show you a demo of the Pantakor Hub client, maybe Azure client, or we can talk about how it is implemented in more detail. So this also opens other possibilities, not using the Cloud, which can be convenient in some cases, for example, for development. Just switching the client with a local client that can be managed from your host computer. We have also an example of that, which is the PBRSBK container, and an example of the tool that managed the PBRSBK from the host computer, PBR. So as you can see, you can not only get the local experience here, it's all about being able to switch the Cloud container with an update. So you get the same base system always with the container orchestrator, which is a minimal implementation of an update manager. In our cases, pretty small, it's like less than two megabytes, so you can fit in those devices with 16 megabytes of storage. As I was saying, to allow control from a container, we need an update protocol. We do that by offering a small server that interprets HTTP requests with a unique socket, and it can be made and can only control the container orchestrator from some of the containers. We set it from the state JSON, so you cannot mess with the socket from the application level. This would be the minimal update flow. It's pretty basic. First, we check for new updates in the Cloud, then we download the update, then the Cloud Clients send the state JSON to the container orchestrator, then the Cloud Clients send the state artifacts related to that state JSON, and then the new update is run. So for that, we're going to need at least these four endpoints or requests, or steps to install JSONs, objects to install the artifacts, commands to tell the orchestrator to run a given update. This can open other possibilities like containers controlling any aspect of the device, rebooting or powering off or whatever. And then some metadata so the container can get feedback information for the Cloud. This is the conclusion. This was really fast, and I'm going to read this because I'm not going to read better if I improvise. Following the UNIX philosophy that teaches us to do one thing and do it right, we have decomposed the device site software to get the simplest possible container orchestrator, Pantaviser. More functionality can be added to this basic versatile setup in the form of containers with privileges that can control the orchestrator using an open protocol like we did with our Cloud Client container. So that pretty much sums up the idea. With this setup, you can pretty much use any cloud that offers a REST API. So you don't have to marry your Cloud provider. You can still do it, but it will be a healthier relationship because you have a way out. And then you can get all the advantages of a local experience. In our case, we have a CLI tool so you can update your device from your host, but we have also worked on some other examples like Web User Interface. So you can update a Web User Interface that is running on the device, and you can use that to update the device, too, and then course-like tool for the same... I mean, it's very powerful in our opinion. I have put here some references with the links that I mentioned before and some more like our documentation if you want to take a look. As I said before, you can come and visit us in booth 33. If you liked the idea, we can talk about it. If you hated it, we can talk about it also. Some heated discussion is not bad if you are talking about embedded devices. And, well, I have a lot of time, so we can do a lot of questions. Yes, well, I'm going to repeat the question because so the guys in the streaming can hear us. You have asked if we have some kind of mechanism to sign the updates with a certificate and check the integrity of the updates. Well, we have... As I explained before, we have the state JSON to make reproducible states, but it can also be expanded to sign the updates because in the end, a version of the device is only going to be a JSON and a bunch of artifacts. So we can use encryption, I mean, asymmetric encryption to sign the device with a certificate that can be in a secure place in the device, like TPM, TMP, TPM. And we can also check the integrity of the artifacts because I'm going to go back to the JSON state. Well, this is because, yeah, I mean, it's a good question because I forgot to explain some things. Well, the JSON is pointing to these hashes are just artifacts that come with the update. So before installing or before running the update from the container orchestrator from Panda Visor, we can check the integrity of the artifacts before starting the containers. I don't know if that answers your question. Yes, as we have a version history, the device can roll back if something goes wrong. It's continuously checking if the containers are running, if they should be running for that version. And then if something fails, it restarts the device. Then the boot loader has the information that it needs to roll back to a previous version that was marked as a good one because it could be started. And I forgot to repeat your question, sorry. But I think with the context, anyone can understand it. You had another one, I think? Well, he has asked if... I mean, how have we ended up using containers in a world with specifications so constrained? In my words, sorry. Well, as I said, the container orchestrator in the end is like less than two megabytes. When I say the words container orchestrator, I guess you might think about Kubernetes or Docker and we are trying to make something similar to what Kubernetes did with the cloud, which is that developers could forget about infrastructure. But we are doing it in small devices like 16 megabytes. There are people that have thought about using Kubernetes and Docker and all that stuff, but if you have to fit that in 16 megabytes, well, you are going to have a hard time because you have to leave space for the application level, which is the level that is going to take most of the storage for sure. So, I don't know where I was going, but we use containers for the... The short answer is that we use containers to get the reproducible states because we don't know any better way of doing it and because we could make it somewhat light. And it, of course, has some overhead, but it's not that bad. And we have tested examples for devices, small routers with 16 megabytes or more. It's LXC. He asked which container runtime we were using. Each application is a Squatch-a-first. In the state JSON, I have only added one because it didn't fit to have two, but each container is just these three things. LXC configuration file, the router Squatch-a-first, Docker digest, this is for PBR, the tool from the host, and a rendition that is going to be parsed by Pantaviser. So, yeah, we use LXC. Yep. Okay, how are the containers cleaned up after a successful update? You mean the containers from a previous? Yep. Well, you can do it if... I mean, it's configurable, but we have a garbage collector that cleans up the artifacts that are not being used in this current version, the artifacts that were used in the version where we roll back if something goes wrong. The logs, et cetera.  If it's configurable, you can set it to clean it always or do it only when you reach a threshold in the storage. How do we manage the list of versions? In the device side, well, it's just a bunch of JSONs and artifacts. On the cloud side, you get more visual stuff. I mean, it's not that I don't want to sell our cloud client, but the talk was about between the cloud providers. But we have a cloud UI that is amazing. That is called Pantacore Hub. And there you have all the, well, a version list. You have all the artifacts from all versions in the cloud.  I don't know if I'm answering your question. I think he... Sorry. Yeah, but I'm not sure if I know how to answer this question. Because, well, we just download the artifacts of the new... I can describe the cloud object storage. So, we download the... For a new update, we download the new artifacts or they are sent by the cloud container using the Unix socket. And once everything is already there, we check the integrity, the signature if there is any. And we try to progress to the new update with the new artifacts. Better now. They communicate through a file. We have in a file, in a text file, the version that worked... I mean, the version that we roll back to and the current version. So, in case the device is rebooted unexpectedly or because of an update that needs a reboot or for any other reason, Pantalizer writes in the bootloader file the revision that it has to be... That goes next. We support Uboot and Grapp. Which? I don't know that question because I wasn't there from the beginning. But I guess... Well, taking into account the current solutions, they wanted to get something for these routers, small routers with low storage. I think that was the starting point, the inspiration of the idea. Okay, so the question was, how does this scale when we have thousands of devices to manage, right? Sorry, what? Billions. Yes, I mean, my big cannot think that big. My mind. But yet, we have another web user interface which is called Pantaflit, that is more focused on that kind of tasks to manage big fleets of devices. And from the device point of view, well, we don't have any mechanism to not flood our cloud API, I guess, right now. But if we get to that point, we will implement it for sure. We will have the resources. We have a small team.  But that's also maybe your own Pantaflit-based system. And those things scale through the stuff of communications and the tasks that are being updated on the device, which is then controlled in the same order here. And that's how I'm going to answer that question. Okay. Anything else? Okay. I guess we can leave it here. Thank you for coming. And, well, as I said, we will be most probably all the time in our booth. So go there if you are interested for demos. Thank you. Thank you. Yeah. We know.


Griffin Bass with you here on Go Mustang Sports TV. I'm here with Coach Dave Webster after Dickinson's 11 to 8 win over York. Coach, you guys are 6-1 now. You're able to come back from the Lost in Newport yesterday. How does it feel to get the win over the Spartans? It feels great. Not the win, just competing and playing hard feels great. We didn't do that yesterday. Certainly not consistently. We're pretty flat, so really proud of the guys. We were without some key guys today, and others stepped up. And the effort was super. Just to go back to yesterday real quick, you guys did lose to Christopher Newport. I knew you were trying to probably enact a little bit of revenge after they knocked you out of the tournament last year. Are you hoping you get to see them again later down the line? You know what, we never talked actually about last year. And I'm a big believer that each season that's a new team, a new year. So revenge never came up. It was just about us and what we were trying to do, and didn't do it well yesterday, that's for sure. But they're a terrific game, and I thought they were really well coached, had a super game plan, and terrific yesterday. All right, so back to today, Matt Thurston. He led all scores with five points today. He had three goals to assist. You didn't have any assists coming in since today. And we thought that you guys, was that part of the game plan, moving the ball around a little bit more and getting more guys involved? Yeah, we were essentially without four of our top six offensive guys today. And so we knew that we had to share the ball a little bit more and get more guys involved and try to do a little bit more off-ball. And so that was part of it. And Matt makes some special plays every game, and it was great to see him contribute that way. Gotcha. And then Pat Scarborough and Gull made several big saves. How important has he been this season and today getting the win? We're all just thrilled. Patrick's super goalie, and he's been behind a terrific guy for the last two seasons. So we're all just thrilled that he has a chance to do his thing now, and he's been fantastic. I don't know exactly what year it is for you guys in the Mustang Classic, but how much do you enjoy this event and how big is it for you guys just getting prepared for conference play and then down the line playing in the May? Yeah, I think this is our ninth or 10th year. It's the premier event in College of Cross. We love it. It's taxing. It takes a lot out of you, but it certainly gets you prepared, and it's a lot of fun. We're here because it's fun, and the competition is so great. Every team's good, and every team's really well coached, and we just have a blast being here. Thanks a lot, Coach Webster. Good luck the rest of the way. Thank you.