I don't have a routine. I feel pressured now. This might be the most boring talk. I hope it's not, but I at least don't have a routine. I also don't have any entertaining gifts, unfortunately. Thank you. So thank you for the introduction. My name is Katherine West. My talk is called Using Rust for Game Development. And since, you know, probably a lot of you here aren't game developers, it's what you can learn from it, even if you're not a game developer. So the genesis of this talk is that, you know, on IRC or Discord, I keep getting these questions. You know, like, how do you make a game from scratch in Rust? I've tried, and maybe I run into, you know, the bar checker complaining. Or I try to make a game and I need internal mutability. Or maybe it's just like, they've tried this and they seem skeptical because they think that the bar checker is very limiting. And I think that these are all kind of, these three questions are kind of the same. This is like beginner fighting the bar checker. This is intermediate fighting the bar checker. And this is advanced fighting the bar checker. But, you know, obviously in case it's not clear, because I'm here at Rust conference, at RustConf talking about game development, I don't necessarily agree with these. But the trick is, is that, you know, you want to give an answer to these kinds of questions. And, you know, sometimes a small answer is not appropriate for a question that's about what happens when you grow something beyond the small. So this is my medium sized, oops, I've already messed up. This is my medium sized answer to this question. So before I go any further, I'm going to have a lot of code in these slides and I really am sorry if people in the back can't see all these slides will be available online. But even better than that, I actually didn't write this talk for RustConf. I actually wrote a long form blog post of this. This is way longer and probably way more interesting. So I'm going to make that blog post public at some point in the next week. So if you don't like my talk, you maybe will like my blog post. And if you can't see, then the blog post will be there. So I think the real reason that I get a lot of these questions is that Rust does make certain patterns more painful than others. This is a great thing. This is probably my favorite feature of Rust, is that it gives me this pressure not to do the easy bad thing. And what I found is, is that these things that Rust kind of guides me to do are oftentimes the easiest generally, whether or not I was written in Rust. And the reason I know this is because I had to learn it the hard way. And I didn't have Rust help. I'm going to talk a little bit about the game I worked on at a charcoal festival called Starbound. I'm going to pick on myself a lot. I made a bunch of mistakes, and I'm going to talk about them. They're all my fault. It's easy for me to pick on myself. And what I should have done, like one of the things I should have done is I should have used something like ECS design. And maybe if I'd used Rust, it would have given me the right guidance and pain to maybe pick these better designs faster. So if you don't know what ECS is, don't worry, because we're going to build one. So this talk is going to be working through a game engine from the simplest it can possibly be to something that I think is appropriate for like a mid-sized project. And it's going to have an ECS in it. And the reason that this works so well with Rust is that Rust tends to reward data-oriented design. And I might be using this term a little bit different than a lot of people use it. If you go look up on the Wikipedia page for data-oriented design, you'll see a lot of things about like cash locality and cash behavior and performance and things like that. But I don't actually think that that's like the most important part of it. And I think it has a lot more meaning for a language like Rust. But I also think that these are kind of two sides of the same coin, but we'll get into that. So what is a game? What is a video game? It's simplest possible form. You make a game state. So this is obviously very, this would be very complicated and large. And your entire game fits in this one structure, right? So players, entities, monsters, whatever goes in this one structure. And then you go into a loop. So a video game, you take input, right? Like from a controller.   Then you render what you got and you wait for the next frame. Repeat. Shoot. Sorry. Okay, I need to stop clicking that. So then you repeat. So this is the simplest possible game. You can't even call it an engine. You make a game state.  Repeat. So what can we learn from this ultra simplistic example? One, I don't necessarily recommend you make a game this way. But you could make a game this way. I promise you can make a game this way because I've seen them. I've seen these source code to unreleased games that are written this way with procedural, ball of mud, 12,000 line, world generation method. I want names, but I've seen them. And if you did it, you'd have minimal problems with the borrower tracker. And a little bit of thought will tell you why.  So if you have complete total mutable access, you probably aren't going to run into problems with the borrower tracker because you borrowed everything. And one of the things that the Rust compiler will do for you is it will allow you to split borrows.  You can borrow one field and then mutably and then borrow another field mutably. And as long as it's a public structure, Rust understands how to deal with that. But it has some downsides. Yet, everything is in one giant, morally global mutable mega-state, right? Everything is procedural. You could very easily end up with a procedural ball of mud where your input system, and this is supposed to make the player jump, but I could just lean over here and just mutate the physics state because nothing is stopping you since you have everything borrowed at once. So there are some downsides. But it's important to keep this in mind because we're going to try and improve this. But we know this could theoretically work, and we're going to improve on it. We're going to improve on it. First, we're going to improve on it in the wrong way. And we're going to do it with the example of Starbound because I did it. So we're going to try object-oriented design in order to improve our game engine. So what are the principles of object-oriented design?  Encapsulation says that the data that you bundle into a class should be private by default and you should lean towards privacy because somebody else might mess with your data and you might have to mess up invariance and then you should lean towards accessing that data through members. The abstraction principle says that rather than depending on having one class spawned on another, you should lean towards depending on some kind of abstract interface or like in C++ it would be like a pure virtual interface. And this is related to the last point, minimal coupling, which says that if you shouldn't couple the design of one class to the design of another and instead interact through these pure abstract interfaces. So let's try this and see how it works. Spoiler, it doesn't really work that well, but we'll get there. So this is kind of a, sorry if it's hard to see back there, again all these slides will be available. This is kind of like the smallest possible star bound game state. So you have three entities, and if you don't know what an entity is because I'm using all this game dev jargon, like if you have a world, everything that's static doesn't move, you can't interact with it, you can't do anything with it, but you can maybe stand on it, that's your static data and then anything you can interact with in games is usually they're all entities. So in this case, our entities are the player, the thing you control, monster, which may be like a little critter going around that you shouldn't touch because it'll damage you, and then maybe some NPCs that you can talk to. And then we have our game state, which is the struct that we made that's going to contain the entire state of our game. One thing to note here is that our game state is like a list of all the non-entity data that I talked about, and then this list of entities, and we've got this, if you don't know C++, I'm sorry, but hopefully it's not too distracting, we have the shared pointer to void because we don't know what to put there yet. So we have entities as an NPC, a monster, and a player, so the type system's kind of in our way, so we have a shared pointer to void, and we're going to improve this in a sec. Another thing to note from this example is that one of our internal states, I have this, I'm going to use this a lot, and this is going to end up being important, but there's this entity index type, and this is actually really interesting because this is super common in C++ game engines. In fact, I've never seen a systems programming language kind of game engine that doesn't have these, and this is a C++ example, so you might think like, okay, well this field is player ID, so server is a multiplayer game, you need to keep a list of all the entities that are a player, but why don't you use a pointer? It's C++, we can live dangerously, let's keep a pointer, and the thing is that nobody does this because it's wildly unsafe. If you keep internal pointers, they will become invalidated and your game will crash. So nobody in the systems game development environment that all game engines have this, it's either an ID or some kind of index or something to identify entities, and this is important because we have to do this in a lot of places in Rust, but it's the best idea, and it has other benefits like, from network serialization, it's hard to serialize a pointer, but this is very common in all game engines, and this is kind of an important thing. So we had this shared pointer to void, and it really sucks because we can't do anything with shared pointer to void, so let's make this, we're doing object oriented design, so let's make this interface called entity. So let's very quickly think of all the things that will be common to every single entity. Position? I'm done, I can't think of anything else. In our example, all of the player, the monster, and the NPC all have velocity, they all have physics, but maybe you have a stationary entity. It turns out that, maybe you have a stationary entity. It turns out that there's not very much in common, almost at all, so Starbound actually has an entity interface like this, and it's filled with a bunch of methods that all return basically option because some entities don't have them, and then you're kind of defeating the purpose of having an entity interface, but anyway, but one thing we can do is we can give them methods. So we're trying to do object oriented design, so let's say that every entity can handle input state, so like a player might start to jump up to space, and then every entity has to update themselves, and they have to take in the rest of the game state, so like if you're a monster and you're tracking a player, you have to know where the player is, and you have to maybe know where other monsters are, so you need the rest of the game state, and then they can render themselves somehow, and we're not really going to talk about that. So then the player, you know, implements this interface, handles input, handles update, handles render, monster and NPC are exactly the same, and now we can keep a shared pointer to entity instead of a shared pointer to void. Okay, this seems like it could work. New requirements start coming in, so you make a video game, you're constantly going to be experimenting, constantly going to have new requirements. Monsters can track players because they know the player's position, but they should track, because it's a multiplayer game, so that's okay. All right, well, health is private to you player because we're practicing encapsulation, so that's okay, we'll make an accessor. Not a big deal. More requirements start coming in. The monster should not go after players who are marked as admins, so start by on some multiplayer game, we have this concept of admin, it's not super important, but okay, we can make more accessors. Not a big deal. Well, admin is admin. No problem. Okay, well, we have to continue on with development, so let's, we need monsters to start damaging players, so if a player gets too close to a monster, you know, it has some kind of geometry around it, and if the player's geometry overlaps with the monster geometry, the player should be hurt. Well, okay, so let's think about this. Where do we put it? We're doing object-oriented programming, so we should take our methods and apply it to the place that it makes the most sense to go. So I guess the players damage themselves. But you could also say that monsters do the damaging. They're the thing that does it. There's not actually a great place to put it, so we can pick one. We'll just pick player, players damage themselves. The player will check its own bounding box with that of the monster, and it will decrement its own health. If we did it the other way, then monsters would have to have access to player's health, which, like, mutable access, that doesn't seem right either. So we'll do it this way, and we need more accessors because, you know, now the monster has this damaged region and the player has to check it, so one more accessor, it's not a big deal. So, okay, so you're making a stealth game, and if the player walks next to a monster and they're on the ground, the monster might react in some way. That's no problem. We'll add an on-ground accessor to player. Except we've been practicing good object-oriented design, and our player has this sub-object, right? You know, we know that we shouldn't use inheritance. We've been told that a lot. You know, inheritance is by wish-you-use composition. So we've composed our entities with this physics object, and only the physics system knows whether you're on the ground. Not a problem. So we'll add accessors to physics, and we'll add an accessor to player that just passes on to the accessor to physics. And as you can see, you kind of need more and more of these, and then the more you do these patterns like composition, the more accessors you need. How many could you possibly need, really? Well, this is the four real actual starbound player class copied from the starbound source code as of the latest version. You can tell that we've been following good object-oriented programming, and we've been using a whole bunch of these pure virtual interfaces, because, you know, you shouldn't have something dependent on the player, necessarily. You might have something else that can emote. So, you know, I think there's eight or nine of these. There's actually more like 35 of these interfaces inside starbound. There's a bunch of them. So let's just quickly go through all the accessors in this class.  What kind of game did I make? Okay, that's four-something. I know it's four. It's four like the player creator or something like that.  Keeps going and going and going. Oh, whether they're an admin or not. It was up there somewhere. I don't remember where it is. Is admin, yeah. Oh, whether they're dead. That sounds important. Okay. Okay. There's like 400 lines of these. The number of methods to the player class is like 200 lines longer than the number of members in the player class somehow. I don't know. This one's great because this is some like sub-object for like the ship's AI. And we just gave up. And now like, oh, you can get it, you know, immutably because it's an accessor and also we need to mutate it and I'm tired of writing methods. So we just, like, there is literally no difference between this and just a public member. So this is, don't do that. Don't make games that way. But we're going to try this in Rust anyway. And we're going to see what we can learn from it. And what you're going to find is that this kind of design fails quicker. And there's a reason for this. Okay. So we start with the simplest object-oriented C++ version. We have an entity interface. And, you know, we have our game state list of entities. We're going to translate it to Rust. And now we have an entity trait. And the list of entities. Very quickly this vector of option, you know, we have a box entity because it's a trait. So we can't keep a direct copy of it. But also just, you know, like there's this option here because I'm keeping a vector of entities. So, you know, you might remove an entity and the option becomes none. Then you allocate a new entity. You have to go find a none to put it in. And then the rest are going to. So, okay. Right away. This is already wrong. This is already broken. So an entity writes to update itself, right? Gets a mutable reference to self and a mutable reference to game state. Even if it got an immutable reference, this would still be a problem. Because an entity gets a mutable reference to self and it's also inside the game state. All of our data in the whole game is in this game state. That's mutable aliasing. You can't do that. So right away we try to make a method and it doesn't work.  You could do things like remove the entity and then call update and then put the entity back. It doesn't work. The best thing you can do, which is bad, is to try interior mutability. So we have ref cell. Now all of our methods take immutable references to self.  You go on IRC and you say, why do I need this?  I need ref cell. You're fighting the borrower checker. It's helpful, but not necessarily helpful. If you've ever found yourself in that situation, this talk is for you. So it gets worse. Say you have this entity trait that's pretty sparse. This is something that actually shows up in Starbound. We have a tag system. We have everything in Starbound. We have this tag system where you can get a list. Maybe in Starbound it's like a list of strings. You have this tag method. I've unallieded the lifetime, so it's clear. But this borrows the whole entity, no matter what, and returns a reference to tags. So already this is a problem. That's a lot of borrowing. And maybe if you want to get the tags from an entity and then do some operation that involves mutating the world, maybe that's a problem. And it's even worse because if you need internal mutability, you can't return a reference. You have to return a ref cell ref type. This is really hard. It gets worse. The larger your game state, the more you borrow. So we have this game state. In Starbound it's like a 2D block-based building game. So we have a method like this in our version of this where you get a block based on some integral index and it returns a block. Well, unallieded, this borrows the entire game state. This is a problem. You don't want to have to call a clone. You don't want to borrow that much. But the fact that everything is private is getting in your way. And this is worse with traits. So I mentioned this because Starbound has this as well. We have our world class, which has a million methods in it. And there's a world server and a world client. So if you're the server, you talk to a world server, you talk to a world client, and they both implement this interface. And I'm trying to improve the performance of Starbound. I've been doing a lot of perf work. I need the world to return a reference. Everything is terrifying. In C++, this is used after free. In Rust, it borrows absolutely the entire game state and then you can't do anything and you're forced to clone it. So it's not very helpful. So what are the takeaways for this? This is kind of common knowledge in the games industry, actually. But in case you haven't heard, object-oriented programming kind of hurts more than it helps. And this is kind of insidious because in a game, you have these objects and it calls out to you, right? What's an object? Well, player, that's like a physical object. Monster, block, right? And these sound superficially appealing, but if you keep going, it's actually quite harmful. Most of your concerns end up being cross-cutting concerns. Things like damage or collision, right? Multiple entities collide with each other. It doesn't make sense to have a collide method. And this bad design that we talked about fails quickly in Rust. You start getting pain points. It's like the Rust compiler is telling you that maybe this is not a good idea. And one of the takeaways is that I find it helpful to force myself to think about things as just data, like to think in a data-oriented fashion. And by that I mean just concentrate on the representation of my game state and don't conflate the methods that operate on the data with the representation of the game state or the state of whatever you're doing. Okay, so this is a bust in Rust. Let's go back to the beginning. So this is kind of a Rust version of our first data model for Starbound. You have a player entity, you have a monster entity, you have an NPC entity. They all have some physics. So we're not going to do the shared pointer to an interface thing. We're not going to do box entity. Let's instead use an enum. So this is all of the kinds of entities we have. And then in our game state, we just keep a vector of these entities, which are either player, monster, or NPC. Then we have this basic update structure that we had before. So cards on the table. This is fine. If you're making a game jam game, probably stop here. If you want to use an ECS, that's fine, but this is not so bad, especially for small things. Yeah, your entire game state is available to everything, but it's actually fine. So I wanted to mention this because I'm going to explain a couple of steps beyond this and each one of them might be a stopping point if you wanted to make a game engine home scratch. So one thing I wanted to point out, though, is that see this physics system? So the physics system needs to go up, go through every entity that has a physics member. But here, there's this enum here. So all entities have physics. Or maybe even most entities have physics. We add more entity types. But the physics system needs to understand every single entity type. It needs to match on the enum and then see, it needs to know which of them do have a physics type here. It won't compile. And you have this, like, it's less, it's not that easy to, like, deal with a subset of entities that have the same state. We can fix this, though. We can unify our entity type. So instead of having an entity be an enum, we can take every member state that an entity has and we'll just shove them all in one base structure.  And then entities might have health. And then we have all these, like, player, monster, NPC, like, types that were there before. But we're just, like, if you're a player, you have the set if you have a monster out of the set. So this is useful. But what's even more useful is if you generalize this a bit. But before, you know, a monster had a current target, right? Like, in this slide, you know, a monster can be aggressive towards a current, towards some kind of player. Well, here, we take this out, and we make it its own little state. So we've just kind of generalized our fields a little bit. And sometimes you can't do that, because, you know, we still have stuff that only makes sense for a player. But this is kind of interesting to think about because it's a give and take, but this allows you to express more states than the previous one. In this, you can express an aggressive NPC. And maybe that's good. Maybe you would want that. But you can also express an aggressive player by setting the player to, setting that, this field, and then also setting the aggression field. And maybe your engine just doesn't know how to handle that. Maybe that doesn't make any sense. Maybe aggression only makes sense. Like if you have an AI system and the player's not run by AI. So it's important to remember that this is powerful, and you can do this, but it's a give and take. So you can go further. And this is usually the part where, if you're reading an explanation of what an ECS system is, they focus on. Because they look at this as purely from the lens of performance. So this is kind of confusing. So let's go back. We have this entity structure with all these optional fields. And then we have this vector of these entity structure. We're taking it, and we're saying that instead of having a vector of structs, we're just going to have all of the vectors for each of those parts in one top level structure. This is kind of called the array of structs to struct of arrays, transform if you're into data-oriented programming. And we're going to do this transform. And we're going to do one more thing where we give all these parts a more informative name. We're going to call them what they are, which is components. So if you've been paying attention, we now have entities and components. And our systems are just these procedures that operate on a game state. So we're getting close. So let's see. What are the takeaways so far? Thinking about the structure of your state is really powerful, and not thinking necessarily always about the methods you've decided to apply to that state. Another takeaway is that you can do a lot with vex and indexes into vex. The reason I mention this is because I see this a lot, actually, not just in game programming, but you need to make a graph structure or something and you need to update it. When you go online, people give you advice about using arena allocators or RC and internal mutability. And those are great. Those tools have their place. But seriously, seriously, seriously, just put your nodes in a vex and use indexes. It is so easy. That should be the first tool you reach for. However, it still has some problems. So if you do that, and I do that, when you get an index to something in your vex of members or your vex of nodes or whatever it's a graph, and you delete one, that index is still valid. And if you think about this, this is kind of like use after free. If you can think of your indexes as pseudo pointers, if you free a node, like if we freed an entity and then used an entity index to get that entity, hopefully we would notice that it was gone. But if we allocated an entity right afterwards, then we might not notice that it's gone. We might get a random other entity. It's safe, but it's not great. We can solve it. So this is a pattern that is super common in the game dev community, and I don't think enough restations know about. So I included it in my talk. We're going to talk about generational indexes. And a generational index is just this index we were talking about, like entity index with one extra field, which is called a generation. So if we had an allocator for such a thing, we might allocate a generational index, and it would give back like a zero with a generation of zero. Allocate another one. It gives back one with a generation of zero. Deallocate the zeroth index. Allocate another one. We get back an index of zero. So all of our indexes are small, and we can store them in a VEC. But we never repeat, our invariant here is that our generational index allocator will never return an index that is ever equal to a previous one. I mean, unless you make this generation overflow 64-bit ins, but if you can do that, your game technology probably has cloud in the name, so you probably have bigger problems. But these are useful because they solve this use-after-free problem of indexing. And to go a little bit further, we just need one more thing, which is this generational index array, which is just like an associative array of these indexes. So instead of a VEC, it's just like an easier version to deal with than VEC of option of T. You can set values based on generational index, and if you get them out crucially, if the generation must be up to date. So if you have an older generation, you get back a none. So putting it all together, instead of having these VEC of options, we're going to have this entity map, which is just a type def of this generational index array, and then we're going to rename entity index to entity, because it's no longer... We don't have to worry about it anymore. There is no such thing as an entity anymore. So entity index, entity ID, we're going to call this index entity, and we're going to keep all these entity maps of all of our components. This is an ECS system. Takeaways. Generational indexes are awesome if you don't know about them. They solve a whole lot of the problems of regular indexing for doing this internal mutability. There's a crate for it. It's called Slotmap, and I couldn't use it in the talk, even though I really, really wanted to, because it's missing a key feature. You can't, in Slotmap, allocate these indexes separately from using them. You have to bundle them all up, and it gives you kind of a structure, but you can't allocate the indexes separately. If you're the author of the Slotmap crate and you're at Roscomp, I am super sorry for calling you out on stage. But, you know, I have another version of this. I can release that as a crate. That sounds like a threat. If you don't make this change, I'm going to release a competing version. I didn't mean to sound like a threat. So, okay. One more set of transformations, and we're done. So, dynamic typing. Everybody loves dynamic typing. You know, it's great in very tiny quantities. But first, we need a type. We need this structure called anyMap, which is great because there's a crate for it, and it's not missing any features, and I don't have to call anybody out on stage. So, an anyMap, very briefly, is a map. It's like a set of types where you can store exactly one of any type. So, you can put a T in, and then if you put a T in before, you can get it back out. It's just that simple. So, this is where we were before, and we can see every time we add a component, this list is going to grow, and what actually ended up happening is we're going to list all of our types a whole lot, and we're going to find a pattern to deal with that. But just in this one instance, all of these components, adding to this list can get tedious, and every time we add, we are potentially touching a dependency of every single system that we have. So, let's give them an anyMap. So, the type system stops being helpful when you use dynamic typing. So, we're going to have these entity components, but they're just these collection of entity maps of type T. And let's go even further, and we'll keep all of our non-NST data in an anyMap as well. And we'll call those resources, and we'll change our name from GameState to something more informative, where we'll just call it what it is, which is an ECS system. We have these anyMaps, the type system stops being helpful. So, you know, you might have an interface like this. So, you can get our component out based on some type T. You can get a resource out based on some type T. And we've added these component and resource traits, which I'll talk about in a second. But this is an ECS system. So, you might balk at this, like, sudden introduction of dynamic typing, and you would have a point, kind of. This is not vastly better than what came before, until we add one more pattern. And I don't... This is common, like, outside of Rust, but I actually don't know what to call it inside Rust, so I'm just going to call it the registry pattern. So, we're going to make a registry for all of our components. We're going to have this structure called a component registry, and we're going to call register component for every single component that we have in our whole game. And we're going to do the same for resources.  We have to keep a bunch of stuff in here, like, you know, box functions, and it gets kind of gnarly. But if your component has... If this component trait has methods on it that allow you to, like, load from a file, what you end up being able to do is being able to have these methods on your registry that says, hey, go take an ECS and put every component that we know about in our game in the ECS. And then you can also do this where this, like, load entity, say your component trait knows how to load from a JSON blob, you can say, hey, go load an entity into my ECS based on this config file. So this might be, like, some kind of array of all of the components, and then it gives back the entity after adding them all. And you can do the same exact thing for resources where you set up all your resources and then you can load any given resource. So then, because everybody loves globals, we can take all of our registries and put them in a big, top-level registry. I'm joking, but this is actually fine, because, I mean, this is kind of, like, enterprise-y, but, you know, you might have seen something like this even in, like, Java, where you have, like, a list of all your types, and it's super useful to be able to say, at any moment, like, you have this global registry that knows about all the types in your game. And then you can have these functions here, like load component registry and load resource registry. You can put those in, like, if you have a component slash lib.rs that lists all your components, put the component registry right below it and you can list all your components, right? So, this is a super useful pattern. So what are the takeaways here? Dynamic typing is powerful and useful, but obviously you need to be careful when you use it. But if you can use it in just the right place, you'll notice that this has kind of solved the last problem that we had with our super simple game design, right? So you declare all these systems, right? And they, you know, we had this problem where every time we added something to our ECS state, we would immediately be messing with the dependencies of every single system. But now a system can only know about a part of our game state if they know about that type. So you can just look at the import list for any given system and see whether it depends on something. And it breaks up this, everything is depending on everything else, not. If you need dynamic typing like this, usually you'll have something like a type registry to go along with it. This pattern is super common. In object-oriented programming, you might call this a component factory, but if I think if I said I component and component factory, I would rightfully be left to head to. But oftentimes these patterns in object-oriented programming are really complex, and any map kind of simplifies this. You know, if you're going to have dynamic typing and downcast, just kind of own it. Just put everything in an any map. And any map is awesome. So let's just wrap up here really fast. Let's see what time it is. A little over. I didn't talk much about the performance implications of things like ECS systems, and I kind of did this on purpose because I've read a lot of really confusing ECS explanations, and I think that they can kind of lose the larger point, which is that it's very useful and powerful to just stop thinking about objects sometimes. And I also think that, you know, you have things like data-oriented programming, which are classically just thought about just for the reasons of performance, that this is actually kind of more important for Rust because Rust has this focus on like ownership and borrowing, and then the data-oriented design can both make for better performance and also for an easier time with the borrower checker. And critically, these are kind of two sides of the same coin. I apologize if you've heard a lot of this before. I feel like I'm kind of retreading old ground, like especially in the gamedev community if you hang out there or are interested. You know, a lot of these things are kind of considered common knowledge like ECS design and the fact that object-oriented programming is overrated. If you have heard them, I hope that this is maybe a simpler explanation that you've gotten in the past. And I hope you've, you know, the very least found some good design ideas or some good features that you may not have known about or may not have thought about otherwise. And also just to finish up, there's a lot I didn't talk about. I didn't talk about all the great crates if you actually want to make a game in Rust.  Specs will do all this, and it'll schedule your systems and it'll have great performance, and you know, don't make your own ECS system. But also there's other ways to make games. You don't have to use an ECS system. You can use the actor model. There's lots of great ways to make games. I was not able to talk about all of them. And that's it. Thank you very much.


The total is at 192267 and that ties out to what's on The balance sheet if I go to the tab to the left and I go into my Expenses we can we could sort the bills now once again. I'm going to close up the hamburger and We can see the outstanding items here and Then we could we could pay them so let's actually go to the pay pay bills now So there's multiple ways we can get in there. We have the drop-down here The pay bills we have this item so we can go into the pay bills We might be able to select by the way if I close that out and I go back in here I'm going to go into my hamburger expenses If you were to group if I wanted to pay these two for example, and then I go into my pay bills Then yeah, I thought it might check them off. It doesn't check them off But we still have them within here as well So notice this is another screen that's basically sorting all of the bills that have been entered that have not yet been paid So when we look at this pay bills form This is basically a pay bill data input screen Which will actually create or could create multiple pay bill forms Because each one of these that we select are going to be another bill that we're going to be paying So let's let's say that we paid if we want to pay these two for example Then we can choose the method up top So oftentimes for many people it'd be coming out of the checking account But we could have the savings account or a credit card type of account that it might be paying Through let's say the date of payment for us. So we're going to put it in the future again. So sometime in After these dates, let's make it the 1 9 And then if there are check numbers then we're going to enter The check number here and we'll have to actually print the checks if there are no check numbers Then you might not have the check number if you needed the check number to be printed Or you can say print later and then we can go into that area where we saw earlier Which was to print which would be to print the checks now to physically print checks You have to actually buy the checks Physical checks you can't just print them on a blank piece of paper and put them into the system But oftentimes people might be doing Electronic transfers these days So you might be using an electronic transfer when we enter this. What's this going to do? It's going to be decreasing the checking account That's what a pay bill does and the other side will specifically not be going to any other account other than accounts payable The screen means the other side is going to accounts payable So it's like a check form or it's going to be multiple check forms or expense forms similar, right? But they're going to be the other side's not going to go to any other account other than accounts payable Decrease the checking account the other side goes to accounts payable. So if we did it here if I said this was a pay bill form It's basically a check form. What would that do from a journal entry standpoint? we'd say that it's going to be accounts payable going down and The cash is Going down and these are debits and credits. So if you if you and I'll just use the 120 in our example So the cash there's nothing in the cash. So in our case cash is going negative and The accounts payable I'm going to say f2 and then say let's do f2 and Counts have counts payable goes back down notice that there was no impact on the income statement any of these accounts in That second transaction so no impact the 120 has already been included and now we're just taking accounts payable back down and Cash is going down although it's a little bit more complex over here because we also have the sub ledger We have to deal with meaning tracking the sub ledger by vendor so let's save it and Check it check it. Well, hold on a second. Okay. I have saved it. Let's go ahead and close it now And let's see what happened. Let's see what happened K. Paso. What happened? Let's go to the balance sheet remember to run it to refresh it And then we're going to scroll on down and see what happened We're in the AP the accounts payable and we can see here. We entered these Bills, let's make that if I scroll down a little bit make this a little bit wider We entered these bills and then we paid off two of the bills this last one I didn't put a dollar amount in it. That's why it was weird I didn't add any dollar amount, but any case that notice how you can tick and tag these off, right? That's what the accounts payable account will generally look like although it can be quite complex when you have a whole lot of Transactions going in and out of the accounts payable. You can also see that pattern of it ticking and tying off when we go to the The detail account by vendor running to refresh it We can see Well here. We're only showing the outstanding bills in this particular account. We could customize this to see if we can get the All of the activity of it increasing and decreasing But let's not dive into that right now. Let's first go and see the activity by going to the first tab Open up the hamburger and then scrolling down to the expenses and then Here if I close up the hamburger It was in unpaid and now those items should have moved over to paid now, right? So now they've moved over to paid that makes sense if we go to The expenses tab note that you could search for this items before we searched for the bills and then the open bills They're no longer in the open bills if I clear that filter Then you're gonna now that are back here in all bills I could look at the drop-down and look at the pay bills form The ones that we paid and there's the pay bills there We can also look at it by vendor and we could sort for the Overdue and open items, but the ones that we have just paid off We have to paid in the last 30 days. It should be in there as well, but let's clear the filters and Let's look for our vendors here. We go. Let's just look at vendor five We could see the activity. So if we talked to the vendor, of course, we can say there's they're gonna say what did you pay us? We're gonna say yeah We entered the bill here and it was paid the pay bill happened at at this time, right? And if it was a if it was a check Then we'd have to see if it cleared or not to see if it cleared the bank feeds But if it was an electronic transfer, we could still see if it cleared, but it should clear a lot faster that way Now just know when you enter the bill another way that you might do this if I if I hit the plus button If we enter a bill here, you might think well, what happens with the bank feeds What if I what if I'm like pay the bill with the bank feeds and then I just match the the payment the payment That comes through the bank To the bill form so you could possibly be able to do that it wouldn't be it's not normally the full service bookkeeping system In other words in other words if you're using accounts payable Then usually you're gonna enter the bill and then pay the bill and use the bank feeds to double check help you with the Reconciliation process, but you might think it's possible to enter the bill and then you might search through here and say okay Which bills do I need to pay? The unpaid bills and pay them with an electronic transfer Right pay them electronically and then allow the electronic payment to go through the bank feeds Right and once it goes through the bank feeds, you'll see them in the bank transaction So we'll talk more about this later, but you can then go to the transactions And and if I went into my checking account, I might see the payment going through here money going out and When I see the money going out, I might be able to find a match and See if I can match it up basically to a bill so you can see the bills forms here Or one of the things that you can match it to and if you were to do that Then you would think QuickBooks would basically enter the transaction of doing the pay bill form So we'll talk more about that when we get to the the bank feed area But again if you're using accounts payable Then you're probably going to go through the whole process of bill pay the bill and then match the bill Do the bank feeds or use the bank reconciliation? Process so we'll continue on with more of our journey through the forms and following presentations


I'm back. Hi, beautiful. Hey. You are looking scrumptious as always. I missed you. Eric's back. Hi. Promotion it again. Today, we're doing some fun stuff. Fun stuff. Are you ready for this? I honestly don't really know what we're doing. What did I call you today and say? First, he yelled a lot. There was a lot of yelling. I think I dropped the call at least once.  That's it. We're gonna color some hair. We're some squirt guns. We're some squirt guns. I know you guys saw me. I reacted to that. This is what's going on with squirt gun hair color. If you haven't seen it yet, it is the new viral sensation. And we have them. We have squirt guns and we have color and we have color. And guess what? Because all you guys were yelling at me last time for using colors when they mixed together, they make gray, I decided not to do that today. I decided to only choose red, violet, blue, and pink because none of those colors will make disgusting colors together. You're welcome. Because I listen. Instead of clapping, we just... It's a silent clap. You're doing great, sweetie. Wow, you look great today. Oh my God, is that a new blouse? I'm really sick, so apologies on that one. I'm feeling a little crazy today. This video's going to be all over the place because I did no planning for this. And we're just hopping right into it because it's already 10 p.m. We need to go out in an hour. So it's going to be a quickie today. Hopefully the editing is great because I- Thanks Marcel, we love you so much. This video is going to be a f***ing mess. Train wreck. So if you want to see some messy queens do a messy video, like and subscribe. Then you've come to the right place. Also, follow Eric on Instagram and subscribe to his channel, literally below. All right, great. Why don't we just fill it up and see what happens? Oh, oh wow. What'd you do? Oh, you missed? No way. I just want to test this. Oh my God! Oh! It worked! I'm so shook right now that this worked. That did not work. I thought it was going to be too thick. Oh! Okay, it works. It works. Okay, but Eric, what's the plan? How are we gonna- Yeah, and we have pretty much the same color. Can we talk about how nice this pink cable is? Guys, what do you think about the pink table? Custom. Custom cut. Custom. Oh my God, whoa. Whoa, whoa, whoa. What is this?  Yeah, yeah, I'm ready to get my hair done. What do you want to do to it today? Something fun. Do you want square gun hair color? Yeah! Have you ever done it before? No. Oh, it's the newest, coolest, latest thing. All right, well, let's get started. Okay, let's clip up her hair. You want to do one side and I want to do the one side and I'll do the other? Sure. Okay, great. You can take that. Okay. You can clip over here. Of course. We're clipping it off. We're gonna do a clippy up. Do I leave it a little down so we can put the color in it? We're gonna take a little moment, like section out. A little bit. Eric, what is this? Can you section it? I'm trying. Everything he does is like- I didn't know the slowest possible. Listen, I didn't know the section goes all the way around. I just hit the clip.  You separate the hair. All right, great. She's beautiful hair though. Now I know how to section. All right, ready? Yeah. We're gonna take her- I'm gonna get to your monetize for this. How much you want to bet you guys? All right, ready? Yep. All right, well, I think we just start doing it, but please don't get on the floor. Just go right into it. You gotta do it far away. Far away. Far away. Oh, wow. Do I rub it in? No. What do you mean rub it in? Wait, it's supposed to go across? We're trying to make like a cool pattern. You know, he tells me nothing beforehand. Tells me. Why don't we look at squirt gun hair color first? This is the goal.  Like a little house in a prairie. Oh, God. I know what we're doing now. So guys, I realize that we're nodging this at all correctly, but actually we don't care. We're gonna have some fun with some hair. We're gonna leave it up to the artist. And make a little art project here.  Right. Just do a lot of lines. Wow, you're going all the way around the back. You know, he makes this look much easier than it really is. Brad, mine looks way cooler than yours. Incorrect. Look at it. No, look at it. Yours looks like someone just got murdered. It's kind of weird. Brad, look at it. It looks like a purple cheetah. All right, we're off here. Ready? Let's do it. This looks cool. This looks really cool. You guys, this is actually looking kind of sickening. This side is nice. Incorrect. Incorrecto. If the glove fits, wear it. We're gonna put some fricking things down. Very professional. Very professional here. Oh, always. You guys, I hope you know that when I do things on YouTube for hair, I'm really just trying to entertain you and don't take it seriously.  Brad, what is this for?  It's for fun. It's for fun. So like stop coming for me and telling me I'm doing things wrong. Because listen, when I'm doing real clients, like girl, I take it seriously. Seriously. Let me live my life, sis. So far Ariana said she really likes it. We didn't even name her Ariana. Yes, we did. We're gonna add more hair. Ariana has a lot of hair and Brad, what am I supposed to do with this as well? Color. Because I had, what was I supposed to put behind it? You didn't have to put anything behind it. Let's squirt on the next time. Let's squirt, squirt it up. We have to change the colors. I'm gonna put blue in mine. Wait, Brad, you didn't section this. Section it yourself. I don't know how to section it. Where's the clip? This is a competition now, girl. Oh, where's the clip? It's happened to us. She stole my clip. She stole my clip. Brad, it's right here. Oh my God. This is like probably the second time that I've touched hair this long. You grew up in a salon. Hey, on the wall, it's really not that hard. Just squirt it on the hair. All right, so what are you doing? Filling this up with blue. We're gonna do the other colors now. Pew, pew, go!  Yeah, really? And it kind of looks cool. Looks really cool. It's definitely debatable, how good it's gonna look when it's dry. I'm pretty into it. Kind of like an emo kid moment. Now you did that one. Is that blue? No. Is it blue? Did you hear it blue? Okay, click. There is some sabotage going on. I wish I could robot. I guess you can. She looks beautiful. She loves it. She said she's coming back for more. Go. And we're gonna section up more hair. We finished each other's hair. We do each other's hair. I'll do all those. Work. You can just do the laundry. What? We lost the entire crowd watching. We have used that joke for about three weeks. Same joke. Of course. He said it's his... Did we use red yet? I don't know what you did. We finished each other's hair. We give each other weaves. How many doobies are out there watching?  Real professional. Oh my God, Brad, look at the side.  Guys, Eric's gonna start his own YouTube channel. Yeah, what should it be about? And he's gonna just make videos. Most of them will be just me giggling. 10 minutes of Eric Mondo giggling. Why would you just hand that to me? Why? Because my hands are covered. Eric, what did you just do? Are you useless?  I've watched a lot of his videos. It looks really easy on the videos. It's not. Ariana is very upset. She's so mad. You may. Our client hates us. Oh, now I have color on my mic. But if you want me to do your hair, let me know. How many more layers do you have? This isn't the 100-layer challenge. He just got my tips. I'm not gonna share it, but I'm gonna put them in.  No, I'm not serious.  I'm waiting for another. There was a time when Brad, you know, he didn't know how to share. Still don't. We still work on it. Have you seen my side? Why don't we show the audience our work? That's mine. Yeah, yeah, yeah, yeah. This is Eric. Do you remember having your own color station nine years old in the basement? I don't know. The whole basement, thank God it was unfinished because it all happened. Color. Filled with color. Look at this. I don't want it. I still don't want it. Next. Thank you. Next. We finished each other's sentences. We're down to our last Gonzalez. I wanna do creativity. No creativity. Ariana asked for creativity. No creativity allowed in this house. What does this look like? Fun and games, too? This is my job.  Look at it. I'm ready to get it. Hahaha.  Oh! Did it get over there? I hope. Oh, it's fine. What? Leave me alone. I would for you to not. I just wanna give her everything she's ever dreamed of in the hair color. Ariana, what do you want? What do you want? I kinda want like a drip moment. Like, oh! This is squirt gun hair color mixed with drip color. What squirt gun means with drip? Squirt color. You start here first. Okay, we're done. Your final bid. Okay, now I'm done. Hahaha. This is my side. You can tell I want a very artsy on this side. And Eric's side. And this is my side. Looks like a dessert. Little dessert. All right, we're gonna go clean up. We're gonna wait for this process and then come blow-dry it, right? Definitely. And we're gonna come back and finish the other sentences. And then show you guys the result. The result. Okay, I'll see you. Okay, I'll see you then. Oh my God. Whoa! Whoa! It's so bright! I mean, that's Eric's side. Whose side is that? We're gonna see what's his side, but this is my side. Hahaha! All the colors kind of like washed over her hair and made it this kind of like interesting color. So we're just gonna start blow-drying and see what happens next. Wait, yours looks really cool. Mine looks really cool. I have some blow-drying. It looks like tie-dye. That's my side. Oh my God. It looks like a cheetah. It looks like an American flag cheetah. Okay, let's see. Okay, some of your techniques look good. So we begin in the front. And then we blow-dry. Rushed it. Make sure you follow the brush. This is pretty good. What? Yours looks really cool. Hahaha! Like really cool. Like if I did this again, I could make it look so cool. Now that I know the technique, what? You're welcome. Hahaha! Wow! This is my side. What was your look you were going for, sweetie? I was going for American flag leopard, Ariana Grande, Red Vines. Where's the Ariana Grande? She's right here. Okay, move. Let me talk about my side. Oh wait. There is no... Wait, what color is that? I was really feeling artsy and I was really feeling myself when I was doing this. It's definitely like an abstract piece. I would say you really went for it. I was really going for like ocean vibes with like a midnight sunset. Yeah, yeah, yeah. You see it? Yeah, yeah, yeah, yeah. See, that's the sunset. It was like a touch of sand from the beach. Yeah! And stand by for the high-pony. Hahaha! I would have Eric give her a high-pony. Oh no, no, no. I want to do space buns. Okay, we'll give her a braid. Space buns! Can you give her a braid right here on the other side? Okay, I actually don't know how to braid. That's embarrassing. No, I think I know. Are you my brother? I think I know how to do it. Grant, I'm braiding, I'm braiding, I'm braiding, I'm braiding, I'm braiding, I'm braiding, I'm braiding, I'm braiding, braiding, braiding, braiding. Did I do it? Yeah, you did it. You braided it. Space buns are lit. Lit. It's not just one of us all with hair, you know. Let's see his face, but... All right, your bun is a little wet. I'm not done!  Look what you did to her. Hold on. She will touch up. It's good. She looks good. Okay, stop! Get out of my life! I want my channel back. Grant, talk to the brush. They gave it to your face. Is that it? No, definitely not. I'm still not showing the ladies and gents. Oh my God. All right, the finish wasn't really that strong. I am four of five. Hold on. We really went for it today with this one. I'm giving you Space Bun Galaxy Livingness and you're giving us what? Low pony, really messy bun. That's definitely an abstract piece going on over here. I'm kind of impressed. I'm confused. Which side was mine? This one. This wasn't the right finish for my side. Are you? The best part was, and guys, just for the record, this brad side, styled by me, actually accentuates what he put together. So how did you, I like the experience. I thought it was very messy. Lots of fun. I do like working with colors.  Yeah. Are you gonna come to my gig? I already came for it. Basically, guys, I realized that this is hard to do. Really hard. I learned now that I shouldn't have done what I did and I regret it all. And next time, I do squirt gun hair color. I won't do it like the way I did it. How would you do it, Brad? I don't want to talk about it. Is that how you do it? It was definitely GTs, but I'm gonna have to GTFO. Oh, I get it. GTs, but GTFO. Uh-huh. Good timing.   Anyways, that's all for today. That's all we got. Derek Mondo, thank you for coming to my channel today. Found it. Mondo's never losing. Mondo's never losing. And thank you guys so much for watching. Thank you for watching. It was nonsense today. I appreciate you. I love you. And you keep coming back for more. All right? Now smash the like button and subscribe.  See you. Hey, sidebar. I totally crushed it. Hey, we're back. Derek, come over here. Come here. Hey, crushed it. Crushed it.


we'll do that when the recording starts. Okay the recording has started so welcome everybody to today's Eden NAP webinar about automatic transcriptions and very much about the EU accessibility directive and how we can comply with it in a good way. My name is Alistair Krillman from Linnaeus University in Sweden, welcome to everybody, to old friends and new friends, just a little bit of information before I hand over to the to the host and the guest for today that is that Eden will be issuing open badges to everyone participating in this webinar. If you have not registered you can actually register in retrospect I will put that in the chat there. So if you have come in without registering and you would like a badge just go in and register and we'll make sure you get one. Okay otherwise I would like to hand over to the host for today the moderator for today's session Wendy Chow and the floor is yours. Thank you very much Alistair and I also have to say a very formal thank you to Alistair for all his technical support because it's my first ever international webinar that I've delivered for Eden so welcome one and all good to see you there and it is great to see such a great high level of international participants here and also to see the support for Alistair and Forindu. My name is Wendy Chow I work for the London Institute of Banking and Finance and I am based here in London. We are an organisation that delivers education from schools right up to postgrad level and executive education and we are fairly well used to delivering e-learning but we can always learn more hence my involvement with Eden and the Nap Steering Committee. What I would like to do is very formally and with pleasure welcome Therindu Leanna Gungarwadina and I have probably mispronounced that so I do apologise and I did say to Therindu I would but I would like to extend an official welcome to Therindu and ask her just to tell us a little bit about herself please. Over to you. Thank you Wendy and you pronounce it perfectly. Hi all I'm Karindu Leanna Adhane I work for the University College of Estate Management here in Reading UK. I'm a learning technology researcher and also I chair the online learning research centre at UCM. My main research area of interest is technology and society so accessibility is basically in the middle. What we will be doing in this webinar is we'll be asking Therindu to present her research which won the Best Research Paper Award at the Eden Conference in Bruges this year and we would like to make it as interactive as possible so what you'll find is that there are two or three poll questions that we will be asking all participants to contribute to as we go through. So perhaps it's good to start with a poll and the question should appear in the left hand side of your screen and the question is do you or your institution use automatic transcription services and if so why? And I will give you a little bit of time to answer that. Oh no no no no no I know where that's going to go. I've got 26 participants so I can see them right so we've got one yes it's a time saver that's an interesting point. I'm a waitress but hopefully soon. Yeah absolutely makes our work easier. Okay there's some really good some noted advantages here. I'm interesting with the person who said we're hoping for a national solution could you explain a little bit more what you mean by that if that's possible that would be really interesting. I can see some people typing in the chat there. I'm also interested in knowing about who said that it's a time saver what they use it for whether it's webinar transcriptions. So Marcus thank you very much for responding to that so I can see that you said you've got a national infrastructure in the form of play service I understand so we do collaborate to get up. Okay and Stefan has responded to your query time time saver for captions and Jogun I apologize if I'm mispronouncing it but I can see that you're still typing so I will let you close out this poll. You tried to google sort of just yeah absolutely. I think Thirindu will be identifying with that as we go through. Okay thank you very much well without further ado I think what we need to do is hand over to Thirindu and let her tell us a little bit more about her research into automatic transcription services. So over to you Thirindu. Thank you Wendy. Good afternoon everyone so here I'm going to talk to you about and present the paper that I presented at Eden conference last June it was my first Eden conference and I'm really delighted that my work was recognized as the best winning the best paper award. So my plan for today is basically talk through these topics but before any of that I want to tell you a bit about where my interest in accessibility started. In October 2013 I was part of the University of Reading group who offered the begin programming build your first mobile game MOOC on the future learning platform. So it was a billiard like game and I was the lead facilitator with a team of facilitators facilitating the MOOC. One day there was a post from a lady who said that she could not see the ball on the screen and when she programmed the ball that she couldn't see she had to ask her partner to see whether it was moving properly the way she intended it to. In this instance it was a very easy solution for me to help her change the color of the ball and the background to make it accessible but actually the way we presented it was not accessible to her because she was colorblind. Later on I learned that green on red is a very bad combination for many colorblind people. So this is where my interest in accessibility started by meeting a virtual learner virtually who had an issue with the content that we presented made accessibility something real for me. So that's where my interest in accessibility started. Moving on here I have few statistics about the use of video in the contemporary world. You can see that the use of video has increased tremendously over the past couple of years with dramatic disc decrease in the production costs and also the ease of distribution. It is also the same when you consider the amount of video used in education. It has increased tremendously over the past couple of years. However using this multimedia presents a problem for learners with hearing disabilities unless we provide transcriptions or captions. However as you may already know creating captions or transcriptions manually is a very time consuming thing and it can it can it takes a lot of time as well as resources. Let me move on to the next slide. So what is accessibility? Accessibility can mean many different things in different contexts but here when I talk about accessibility what I mean is a person with a disability whether it's long term short term age related or any other can access the content what is being delivered. So as I said earlier when it is video and audio content captions and transcripts play a very big part in providing accessibility. However they are not cheap. On average my institution pay about 1 pound 30 per minute of transcriptions and that comes to about 78 pounds per hour and these are subjected to minimum fees as well as additional charges for more than one speaker in the video or audio. But at the same time we know that automatic speech recognition services such as Google Duplex, Apple Siri, Amazon Alexa has made tremendous strides in the past couple of years. So also both IBM and Microsoft have reached the point where their word error rate which is the mostly used measure of checking accuracy has reached almost the level of a professional transcriber. Let me move on to the next slide. There are accessibility laws such as the EU accessibility directive and the new UK regulations for a very good reason. We know that making things accessible is the right thing to do but then why don't we do it all the time? You will remember in March 2017 University of California Berkeley took down or removed over 20,000 videos and audio recordings which they have previously made available as open education resources or OERs when the Department of Justice ordered them to make these content accessible. The decision to take them down was based on the cost associated with making them accessible. But because of this none of the people who previously could access these resources were able to access it either. So it was completely lost to the public. But what if automatic transcription was an option? It would not have cost them so much to use automatic transcriptions. So this is where we need to look at new technology and see whether these could help making things available publicly and accessibly. Let me tell you a little bit about my institution. University College of Estate Management or UCEM is celebrating its centenary this year. It is a specialized institution offering supported online learning courses in the built environment sector. As many distance education providers UCEM courses also started as correspondence courses moving on to online recently. In late 2015 then College of Estate Management was recognized as a University College and we changed our branding. This time was taken as an exercise for looking at accessibility and you can see some of the work that we have done since then about making our content more accessible. At UCEM when we have recorded media audios and video we always provide transcriptions or captions. However we use lot of webinars. These are only transcribed only if they are easy student with hearing impairment registered in the course. But actually it is shown that captions help everybody not just the people having hearing impairments. So for example international learners we have almost quarter of our students as international learners. So we wanted to see whether there was a way of providing transcriptions for all our content including the webinars that we hold. So in my study I looked at six software which provide automatic transcriptions. I have given a detailed description of how I selected this software in my paper. I will not go into much detail because I don't have that much time. With these six transcripts I wanted to look at audio that is relevant to our subject area that is built environment education. So I asked our experts our tutors to provide me with subject specific text about 200 to 500 words. And out of the submissions I received I selected four topics consisting of 1000 words to create my text. This text consisted of four subject areas property management, construction management, property and contract law and building pathology. Then I invited a purposely selected sample of 14 participants with various accents native and non-native speakers to take part in this study. Out of these 14 10 people were interested in participating in the study and they were given access to this course that you can see on the screen transcription testing. Here I have set up a webinar setting where they could go and record their voices reading this experimental text. You can see here the data I have collected. I had seven recordings from both native and non-native speakers and you can see how they have described their accent. For the analysis I first got them transcribed using all these transcription software which meant that I had 42,000 words to be analyzed. First of all I checked it with the recording because sometimes when people read out something they can make mistakes and I didn't want those reading mistakes getting compounded. Then I compared the recorded transcripts with the original text using Microsoft Word Compare function. But finally I did a manual analysis of all the transcripts with marking substitutions, deletions and insertions with color codes. And then I calculated the word error rate. You can see the calculation how you calculate the word error rate. Let me show you how the word error rate is calculated. This is an example taken from Caption Accuracy Matrix project. So as you can see here each word is given the same importance in word error rate calculation. However in technical subjects keyword getting keywords correct may be more important than getting something like a or the wrong. However it is not considered in the word error rate. So ideally it is not the best way forward in comparing a transcript but this is the matrix that is used most in comparing transcripts. But there are other methods like weighted average word error rate and so on. But I used word error rate because it's simpler. You can see the results here in this graph. As you can see I had seven participants but I have eliminated participant one's recording. This was because only participant one recorded the voice recording not using the headset like this rather using the laptops built in microphone. So the quality of the recording was very low and I felt that I could not compare it against the other recordings which were done with a proper headset like this. And as you can see here this script the dark blue line shows the best word error rate that I have achieved in this experiment. However this is way below the 95% accuracy level that is normally required for a transcript. And as you can see here participant five participant five's recording had the least errors least number of errors and that participant is a non-native English speaker. Moving on I asked some of the subject experts to look at the descript transcript which was the one that had the best accuracy level and looking at the descript transcribed transcripts the experts said some of them were good enough and some of them were almost good enough. As you can see here even though participant five's recording received the had the fivest errors word error rate participant four who is a native English speaker's recording created the best or rather the good enough transcript transcription. Also another interesting point was that in property management I did not get any transcripts that were good enough as per the experts. This was because the property management talked about the paragraph talked about property and bonds and the word bond B O N D was transcribed as bombs, blonds, bumps, bones and even buttons. So basically it did not make any sense and it was not possible to make sense of what the conversation was about. So with all of this my little research experiment concluded that the quality of recording does matter and in a technical discipline word error rate may not be a sufficient predictor of the quality. As I said before I compared the participant four's recording and the participant five's recording but it's a good enough starting point for creating a transcript as an accessibility aid. I think I have described my research and I must say that there are a lot of limitations in this research and I have gone into great detail detailing them in my research paper so I welcome you to read the research paper about the limitations. Wendy do you think that's a good enough explanation? I think that's an excellent explanation thank you very much that was extremely thorough thank you for presenting that so well for us and making it so clear it's very hard to actually do this and not read from the slides that you've given us some great detail level of detail so thank you. At the moment it does seem that perhaps that the transcription services are not sufficiently robust for technical delivery so there's on a higher level of accuracy so what I think it would be good to do is to bring on the poll if we may and ask our participants so in the previous poll we saw that not many of them you use automatic transcription services currently but by virtue of the fact that you're here today that's suggested that actually you would like to but let's not let's actually challenge that assumption so could we please see your thoughts for the participants in which ways do you foresee a use for automatic transcription services in your delivery and I think it would be quite nice to broaden that out in particular in terms of what your technical expertise levels are thank you so and one way for me is that I currently work for higher education part of my organization and we talk about banking and finance but as part of that there's a lot of regulation involved in that and whenever we've used transcription services in the past it's come up with some a high word error rate so we have resisted for going that way so lovely actually so would you be happy to answer one of the questions Marcus for the moment while the poll is going on and please do still contribute to the poll but I see that Marcus is asking he's not sure how the different transcription services IBM zoom etc performed within your research is that something you're going to talk us through in a little while or can you answer that I can actually in my research experiment zoom came zoom had about 73 percent which I was going to talk about in a little bit but unfortunately IBM was the worst performing in my experiment so I must say that in this what I used was the free available service so if the service that's paid for is better than the free service that they are providing it would not be reflected in my research research results zoom had about 73 percent accuracy rate yes that's interesting some of my colleagues were saying that they would like to use zoom privately so we're looking into it from an institutional perspective and we're also looking at a different virtual learning environment provider and they're suggesting that their ATS would come up with a very low word error rate but we're yet to test it Marcus does that answer your your query hope it does is there anything else you'd like lovely thank you Marcus if we turn to the poll let's have a look at the reasons why people might be using ATS it's quite interesting because I'm glad to see that we're not just saying it's because we should be doing it we've seen to move beyond that um so level one adding a transcript level two creating closed captions okay and increased accessibility do we have any feeling in terms of our views on ATS of videos and movies serendu sorry um so on the poll people are talking about there's quite that to do the ATS for the lectures which is what you've been testing yes and coming up you might be talking about to have you investigated ATS in terms of videos and movies um movies and videos when we create videos yes we have done it um I have a slide do you want me to explain it with the slide I think maybe we'll go to that in a moment so if we can hold that thought would that be all right I'd like to have a look at the other the poll um okay sure yeah are there any points there that you would like to pick up on and I totally agree with as program director it's that's unnecessary for teachers and administrators I think it's something that is now going to be ingrained in our delivery and I found that some of my students really like to have the transcript just in terms of an opportunity to reflect on it I would like to pick up on this one we have tried both manual texting and automatic transcriptions but the latter one still requires a lot manual correction afterwards I agree with that if the there are so many parameters that makes a good automatic transcription the quality the the way the speaker speaks the accent there are a lot of variables so it's really difficult to get a very good transcription automatically I think we've come to the end of the poll so I'd like to say thank you very much for everybody for contributing there but I think if we can summarize that we see that there is a need for it and we're all interested in how it can be done better um but it's about giving people the opportunity to actually rely on an effective ATS without having to manually correct things as we're going along so I think this isn't now a good time for us to return to Thirindu's slides to see for um she can tell us a little bit more about further research that she's conducted in this area so thank you very much over to you Thirindu thank you Vandy so since my research UCM have now adopted Zoom as our video conferencing tool this research was done before we moved to Zoom so we were looking at Webinar software and Zoom scored highly because it had accessibility feature automatic captioning we have about 10 webinars in each module and as I said before whenever we create recorded videos we provide transcriptions and captions at UCM but for the webinars we did not provide them previously we introduced Zoom in spring 2019 that's this year the spring semester and actually it's not 100% accurate and my research has shown about 73.7% accuracy is there with the Zoom transcripts so we asked our students what they thought about their experience with Zoom and I put in put in some questions to get their feedback on the automatic transcripts so that we can see how they perceive the transcripts whether they are rubbish whether they are useful and to be honest I was quite surprised the question we asked was have you used the audio transcripts we had 283 students responding to the survey and out of these students 52.7% that was 144 students said that they have used the transcript then for the people who said they have used the transcript we asked a follow-up question to see whether it's useful so the question we asked as the follow-up question was did you find the transcript useful if so why we gave them the opportunity with an open-ended question so that they could say why they found it useful or why they found it not useful so this is what I found 104 students that is 80% of students agreed that the transcript was useful 14% said that it was not useful and where the student did not say yes or no and discussed both pros and cons of the automatic transcript I marked them as other that's the 6% so basically 80% of students found the transcripts which were about 73% accurate useful and then I wanted to look at the transcript usefulness of the transcripts and the student's demography we had 92 respondents from non-English speaking countries that is excluding respondents from UK Ireland Canada Australia New Zealand and so on out of these 92 respondents 40 responded to the free text question did you find the transcript useful and this is the result so 95% of our non-English speakers non-native English speakers find the transcript useful again this tells us by facilitating accessibility we also make it meet the experience better for all our students I'm going to share with you some of the different views that I saw in the questionnaire I'm still analyzing it there were some comments by native English speakers saying that the errors in transcripts will be detrimental for two non-native English speakers but from this big feedback we have received from the non-native speakers we can see that even with this level of accuracy it is useful to the students here I picked some comments from native non-native speakers a student declaring a disability to show they are different in views while for some students their transcript errors may make it feel like it's a useful useless thing for others the same transcript may be useful so it was also interesting to see how students use these transcripts overwhelmingly students use the automatic transcript as a search mechanism so in zoom if you search for a word that takes you to that point in the video so the students use the transcript mainly as a search mechanism they also use it as notes and there are students who said that when they are sitting in a webinar in their office where they can't use the audio they use the transcript and also as an aid for English as an additional language speakers and for learners with disabilities I think that's all from me Wendy lovely okay thank you very much and the rindu has been very modest there by not articulating it but there are the prizes and I was so honored because I was sitting next to her that night and when she was presented with the award for her research and you could not have had a more modest recipient and a delighted recipient that on the evening so well done to you the rindu for that and also for continuing the research it does look like there is a real scope for the research to continue and I personally would look forward to some further research publication what it would be perhaps lovely to do now is to ask the participants are any of you interested either in collaborative opportunities or in turn just are you inspired to actually explore new adoption of ATS so over to you for those opinions please thank you okay so switch to zoom says Marcus maybe is that something you'd recommend the rindu switch to zoom we are finding it very useful and as I told you earlier we had this workshop where David White was speaking and he wasn't wearing a mic or anything it just picked up from the laptop microphone and the session was recorded and I was amazed to see how good the quality of the transcript was so that sounds like a good route to go at least to try it out it very much depends on the speaker and the way it's picked up like I like Alastair's comment has been quite challenging there because I offered ATS we all have work to do I guess the thing to do is to actually make it as easy as possible so that it is literally automatic and so you don't actually have to to do any correction the point that I found most interesting is that for the native speakers we make this presumption that non-native speakers would find it problematic whereas you've got that 95 percent waiting in favor of it from non-native speakers I know my students really do appreciate the search function so they're actually so even if they're a native speaker they say well it's just so easy to use the search function I get exactly the right point that I want to so either for revision purposes or assignment preparation purposes can I interrupt you there Wendy another thing that I thought would be useful to mention here is that the fact that in my research I didn't have access to hearing impaired students so when you have when you have to when you have a disability or any impairment you would have you would come up with other ways of complimenting it so it could be that lip-reading that sort of things would also help students so maybe having a little bit of inaccuracy in the transcript may not be that much of a difficulty for them but unfortunately I could not go in that avenue explore that avenue more because I did not have access to students that's an interesting point because the participants won't be aware that Thuringdu is aware that I am hearing impaired and I think in something like this if you can have the little image of the lecturer actually that's great because the participants can have that opportunity to lip-read and that's far more effective in my opinion it's only limited but in my opinion that's far more effective than conducting signing because signing is obviously not an international language there is no international sign language capability whereas if you actually do the lip-reading you can lip-read in multiple languages and in some ways it's even better than when people have slightly different ways of pronouncing things when you can still pick it up and I'm just turning to the poll comments there yes I think that point about speaking slowly and more clearly to help is a key one particularly when we get quite passionate about our subjects sometimes it's very easy to start talking through okay I'm having the developments we'd like to see in this field but a good support for the small Swedish language so that's interesting so are we um saying that zoom perhaps wouldn't be appropriate in Sweden I think actually the uh it's Alistair here again uh noticing on the chat there I mean I think it's in the pipeline but of course uh everything gets done in English first and then the rest of the world's languages tend to come in a queue afterwards like Spanish Chinese etc they get done but then smaller languages uh but for us vital Swedish Finnish Norwegian Danish etc etc they are way down the priorities but I think there is work going on I think from Marcus he's mentioned those it's in the pipeline but that's the key really and uh all of these applications have to have to be flexible enough to to understand different accents and different intonations yeah absolutely and I also think here that the comment um the people that make it visible so that you've got the automatic subtitles during the dialogue that's a really useful thing for people who are hearing impaired um because if you can't quite catch up with the literally etc to have that that subtitle would be wonderful yes and uh Lotta I do agree with you it's um it's subtitles are also beneficial for those watching videos and for sound has to be off absolutely if we are turning around and saying that we can join courses and do our learnings on trains and buses and transport then that is exactly right all right I think we'll have a we've got multiple attendees typing so whilst we are very quiet we are definitely looking to see what has been mentioned and that's happening the Skype has for a long time been able to translate directly from Spanish and English um but again not all language combinations work but the obvious ones do to a certain amount of accuracy and when you get that and you can speak your native language and be translated but how do you know that you're being translated well yeah you could be yes you could be speaking very seriously and the audience starts laughing at you because the translation has made a real that reliability as well as accuracy and I would like to add something there I'm a native singular speaker and we normally have two languages like the the speaking language is very different to writing language it is the same language but it's very different styles so when I tried translating things it always translates the writing style so if you speak in the writing style people would laugh at you so yes I do agree I can remember being in France one year and listening to a an audio of an event and it was in received Queens or King's English and I think everybody in the English because they were the first to enter the University of Minnesota so I guess yeah not really there golfers can't today be a big analysis first uh we we wonder if um normally in these webinars we uh you are all very silent good to see you here Laura if you would like to uh wake up and uh get a microphone then it's perfectly okay what you do is if you want to if you want to contribute to the discussion orally we'll just put your hand up and we'll give you the mic and the audio if you and the video if you've got it it works it works perfectly well as long as people have got a microphone and it's set up but I know there we are there you've got a microphone you can even put your video on if you want but please speak hello do you hear me yep perfect hi okay thank you uh great um great listening to you uh I was about to comment the crowd edit api solutions comment I made in there I think it will take a while until we get a very good uh transformation from the Swedish language and also it has to be edited of different reasons not only to be correct but also to be um it's about to choose maybe another way of writing it sometimes and if we can hook up students uh or other groups in a crowdsource way to to help out I think it could be more financially possible earlier to solve some translations or transformations so and then it comes down to how are we going to solve that and you need some sort of edit function in a good way there are many ways to try to do it but it would be interesting to see how could we easily facilitate students to help out in this transformation work that's what I meant thank you for explaining that that is really an interesting thought surrender what do you think do you think that could work um if you know about side note uh it is something uh a crowd platform so for example uh it was uh it was created by the spin-off company from University of Southampton and in that one what they do is the the videos are transcribed automatically but then the students have the ability to change them and correct them if there are errors so as uh as uh Mark said right right now uh if people are contributing it is going to get better have the direct link that would be even better but it might take a little while for you to get it if you could just type in the chat anyone else with a question who would like your hands up can be a reflection let's go ahead and do um type the information no no okay thank you for that surrender I can see sign there so that's in the the box with the direct link that's brilliant okay maybe we if there are no so perhaps we can do some collaboration and I'll have some other people try it and see whether they have better luck with it they don't but I just want to see what A Williams is typing if we wait oh thank you A Williams I must admit I'm as guilty as many of the rest of us not doing this very much uh I need to do it more that'd be a while absolutely I think we certainly here um we're very interested because we work in schools um and also in executives and you find that they they find it to be a time saver that's very much what the participants were saying at the beginning is that it is a time saver for people um so thank you for your comment there um A Williams and uh look into fully consider and embed within UCEM so that's great to see as well right well it's without further ado what I need to do is to uh draw people's attention to the fact that we will be having a tweet chat um answer induced presentation later on which is at five o'clock GMT and six p.m tonight um central european time and obviously the topic is automatic transcription services that are they good enough for accessibility so it would be lovely um if yourselves or you can encourage colleagues as well to come and join us on that tweet chat and we look forward to seeing you there but before I go any further they're going to thank you so much for your time today it's been really great to see that and it's clearly a um a subject that has resonated I'd like to round off a little bit by pointing out that uh right down here you'll sign it uh you can just click on that link and thank you very much for giving me the opportunity welcome to do that um but if you go back to the recording you'll be able to click on the links anyway uh two web links at the bottom the eden events page to keep an eye on and on that you will find more information about next the next uh webinar which is called designing inclusive and engaging structures to facilitate creativity and learning that will be 11th of december more information on the eden web a reminder about the annual conference for eden is the 21st to 24th of june in timiswara in rumania uh information is not up yet but will be next week and we'll be opening for uh contributions then so uh get your uh get your pencils sharpened for your writing contributions for next year's conference and a way out there warning for next autumn october 2020 that sounds like science fiction but there's going to be another eden research workshop this time in lisbon portugal so i thank both of you uh thrindu wendy i think we are done for today and uh thank thank you all for being here uh you'll get the link you'll get your badges very soon if you haven't registered then maybe christina can put the link for the badges into the chat you get an automatic badge if you have registered but if you haven't registered then you have to register afterwards because that makes sense in that case we will say goodbye from here and turn off the recording