So, okay, a few seconds. So, before I start, I would like to ask you how many of you are graphic programmers? Cool. How many of you are programmers? Nice. I just prepared my talk and I expected that there won't be a lot of graphic programmers, so I think I have done my goal. So, I tried to explain how our graphics pipeline works without much graphics programming details. So, if you want to hear something about 3D linear math or related algorithms, there won't be that such slides. I have a few seconds before my team member helps me with the cable. By the way, do you know ZLED? Who knows ZLED? Nice. It's the game. You can play it if you want. There's no signal. Yeah, because it's normal because I'm waiting for Stan. Actually, I have 45 minute talk, but it won't be 45 because I plan to have something like 15 delay because of such strange accident that usually happens with presentations like with cables. So, I hope you won't be so boring. Graphics, you mean, programmers? Or... Images, pieces of work that... Actually, a lot of ZLED is 3D game. So, we need to have models, 3D models. We have to cover them with textures, with normal maps. We have to render UI. So, we need some art stuff for UI. Buttons, checkbox, scroll box, and other stuff. So, I think our compressed data is about art, it's about 2 gigabytes. It includes meshes, textures, materials. I think I can call shaders art stuff as well because it's something like math art. So... Yeah. It was... Lift. Yeah. That's nice. I'm going to talk about ZLED, how it graphics pipeline works. My name is Vladislav. I'm a graphic programmer at 2.5 Ames. We are open source community that developed the game for a long time, about 20 years, and it wasn't open source for the whole time. It became open source only 10 years ago in 2009. So, what I'm going to tell about, what is ZLED at all, what we have behind visual stuff, what players see, and most of my talk about pipeline, so how it works. I will show you how it works, how it looks like visually, and how it works for GPU side. And I will talk about debugging, what we use, I use our members contributors that help us to develop our game. And very important topic about artists because it's really interesting and I think artists are really important for games because it is that how it looks like for players. So, basically, ZLED is real-time strategy. It's typical skin shot from the game, like this is one is civil center when you start with. It's about ZLED from 500 BC to up to 1 AC. Player can control about 17 civilizations. Each civilization has different type of buildings, different type of units. All these units are visually different. Also, each unit, each building has different variation. What does it mean? It means that an artist can add an option in object definition. We store it in XML files. You can switch between different textures, between different measures, between different sounds. How does it look like for a player? When he plays a building, for example, just a simple house, random generator, generate a seed for new mesh, for new object, and by this seed game, we'll choose a mesh, a texture, a sound. You might have variation of variations. It helps to artist a lot, but that's the problem for graphics programmers because you need to render all this stuff. If you try to render a lot of objects with different textures, with different measures, for example, if all objects on visible scene are different, it's pretty hard. It needs to switch textures on GPU. You need to set measures on GPU. All this stuff slows down the process. Either way, if you need to draw only, for example, the same object multiple times, it's the best what can GPU do because it's mostly about parallel processes. So, a player can learn about each civilization, so it's absolutely historically based. They don't have any, at least at the moment, we don't have any imagined civilizations. They all are real, and they are all based on real historic facts, and all this stuff player can read in our game, so it's a bit educational. Beside main objects, 0D has flora and fauna, so different animals, different trees, different grass that you need to render. The same behavior for variations is applicable for trees and for grass, for example.  I mean, it looks differently, it depends on random generator, so it also helps to artists to not switch between different models, between different meshes, so you just put trees, for example, you might have one biological kind of tree, you select it by mesh, or by list, and then you just click on the terrain, and on each click, it will generate, for example, different meshes, for example, they will be different by a height, so small or big trees, that helps a lot to artists, but not for programmers. So, what's about technical side? Important for graphics. We have an entity component system who knows, what is it? The main idea of the entity component system that you don't use classical approach when you have a basic object like iObject, and you derive all your object from it. Idea that you have entity, and you can attach any possible components to that entity, for example, if you want to place an object, the entity on a terrain, you need to know a position, so you have ICMP position component. If you want to render it, for example, it would be interactive or not, just a result stuff, you add a result action. If you don't need something, you just don't add it. For example, if you want to add a sound, you just create, if it wasn't present before new component, like ICMP sound, and attach to a new entity. So, if you would try to open our maps, XML maps, then you will see a list of entities. Beside that, for each entity, we store a seed that was generated when we placed it to choose the right... What's from this scrolling? Okay, variations. That's what I'm talking about, that you don't need to take care about of different types of the same thing. The engine cares about instead of yourself. That's the problem for programmers, because let's look at such example. You have a box, and you have different textures, like it's wood box, metal box, or, for example, water box. And to render it properly on old jail, an old API, you need to bind texture wood, then draw box. You need to bind texture metal, draw. Mesh was binded before, so we don't need to rebind it. And then we need to bind water texture. It's not so slow as it can be if we rebind the old mesh, but it slows down the rendering process. So, for GPU, especially for modern GPU, it's much better to use things like texture arrays when you bind a lot of textures for the same object, and you will choose it one time in shader code. But it's more about modern API. I will talk about it a bit later. Another important thing for our game that we have a lot of materials, I just can modify different options of materials, so how it works internally. We have shaders, native JLSL or AIB shaders, and they all are combined in materials. So, shaders provides API, because it's not a real API, it's just a list of possible options that you can choose, you can select in XML, in shader XML. And artists can combine different shaders in the same material, and then tweak some options in material that then they will be passed in the code of the shader. So, for you as for an artist, you don't need to care about shaders, you just tweak values in materials. That's our platform ratio. I think it's pretty expected that we have a lot of windows, have a lot of Linux, and have some people on Mac, for example me, just because I don't have any other laptop, but yes, the game works on all these platforms. Also, we can run our game on Raspberry Pi, and we have tried to run it on a ARM, ARM process. So, I mean, it can... There was time when we have run on Android, but we haven't tried for a long time, like for a few years. I think mostly because we have... We don't have a lot of Android developers that can fix stuff that's special exactly for Android, but we support OpenGL years, so we can run on it technically. That's our ratio about vendors. All these statistics is from our game, so when you click on the button Enable Feedback, it sends information about your system. It has a description in privacy, so you can see what to be sent, and one of values is your vendor or GPU. It doesn't matter, because usually AMD and Nvidia mostly don't have a lot of problems. AMD may be more strict about shaders, and they don't have a lot of problems for, for example, Windows, but Intel drivers if you ask some graphic programmer and ask for Intel driver on Windows, you maybe will see an angry person. I can just show an example. Intel driver crash, your application, I mean our application, just because we tried to multiply metrics by a virtual, some special conditions that happened before, so it just crashed, your application, okay. We still have users with really old stuff, like GL-1, it's not so many as, for example, GL-3 or GL-4, but we still have it, so it does mean that it works with GL-1. It means that it's the maximum possible version that supports the hardware, so we still have users who have only GL-1. It means, usually it means fixed pipeline, fixed function pipeline if you remember, okay, but it's more about statistics. So basic stuff, we use our engine, our game engine, it's also responsible for our graphics, it's called pyogenesis, it's written in C++. We use different libraries, but for graphics, more important are SDL. It's responsible for setting GL context, for managing buffer, swap buffers, and related to OS dependent things stuff. We use OpenGL. Basically, we use OpenGL-1 and OpenGL-2 functions, but for some cases, when we detect that user has extension, not in core function, but extension, we can try to use it. So at the moment, we have three ways to render our game, fixed pipeline, it's the first possible version in the world, I mean, it's the first possible way to render when you can tell to your GPU what you have to do, what you want to do with your data. You just tell, like, you have such light, you have such extra, you have such option, render for me please. With shaders, you can write a code that runs completely on GPU side, so you have already control on some stuff where you might add some smart things. So for really old stuff like GL-1, we used fixed pipeline, or some really strange video cards, because not all vendors try to fix old cards. For example, we have a list of different video cards. For example, there was some GeForce from 2006 that have wrong behavior of shaders. So we have to switch to fixed pipeline for that card. But actually, in our plans, we try to remove this at all, because it's pretty old and we have only about 1% users, I think we can force them to update their computers instead of support our old stuff code. So let's talk more about visual stuff, how it looks like. It's a typical frame that user can see in his game. So pretty nice. You have border, you have reflections, how we usually render it. First of all, we need to enumerate all objects that you have in scene, and you need to call them. Why we need to do that? Because we don't want to send to GPU more data than we actually need. So for example, if an object is invisible on any camera, I will talk a bit later about cameras, if it's invisible on some camera, then we don't need to render it, and it saves time. Because for example, usually user have top-down camera, so player doesn't see a lot of stuff, like about only 5% of maps, so we can avoid rendering of 95% of all objects on scene. Render scene for each camera. We have four cameras. Four cameras in max graphics settings. Because what we need to render? We need to render shadow. So who knows what shadow map works? Usually to render shadows, you just can't use some magic functions. What you do? You have a light source. Usually it's a sun. It's directional light. So you have a vector. Now you need to render all objects that are present in invisible scene on 2D texture. It's called depth texture. And then you apply this texture on the final pass to calculate is it your pixel that you're rendering at the moment is occluded by any object. And it is occluded or not, is calculated from depth texture because you have a distance between a light source and nearest occluded object in this depth texture. Reflection and reflection. Camera because reflection might have a different direction. So we need to have different angles of camera. Shadow pass, calculate without any color. So we just need to know transparent or not transparent object. And reflection, reflection, they need colors, but they are different from main camera because when you look at the water, in usual case, you have a much bigger number of objects that you will see by different. And the final pass, you combine all texture rendering in the previous stuff. And now I want to show you post-processing. It's not so interesting. Just like effects like we can apply anti-layers thing. We can apply color grading. We can apply some brightness or contrast tweaking. But it's post-processing, so it's not so important. It how it works in code. We use one interface, shader program, and we have pre-implementation at the moment. So shader program is responsible for do all basic DL stuff like bin textures, like bin mesh, any attributes for your mesh, and then each implementation and each derived class, like pfp, imb, dlsl, have all different code based on GL version and how it should look like. And now more pictures, finally. I hope it would be more interesting. So that's how the frame before looks like from a side. So you have a camera view and that's all objects on scene, not all, but most of them. And you need two columns, so you don't need two radars. And after that, after cooling, you have such objects. You might notice that we have such trained tile. Why we have that? Because actually there is a small intersection here between tile. It's bounding box and camera frustum. So that's why it's rendered. That's okay because it's not performance hit and we can just render it because it's pretty cheap in compare with other objects like tree. For example, this tile, the wall tile has about 32 by 32, about 1,000 triangles. It's pretty equal to each tree. So it's cheap to render such tile. That's how it looks like from a shadow, from shadow side. So you have cold region and now you have to render all objects that can cast a shadow. In the top left corner, you might see a depth texture. White color means first point. The black color is nearest point and depends on a color, color value. Actually it's only gray values. But depending on color value, you can check occluded or not your object. So it's about debugging, but now I want to show you how it looks like I think in real time because it's less boring, I think. So, in steps and current step. So how it looks like. First of all, we need to render shadow map. It's slowed down a lot because actually this time need only about few milliseconds. And you won't notice if I try to play it in real time speed. So how it looks like. Reflection map. So it's actually mirrored about water level. So it has the same position X and Z, but white position is inverted around water position, water level. Now we render refraction. We have red color to detect leaks. For example, if shader tried to pick a color from wrong coordinates, some of you who play may notice sometimes we render terrain. And just iterate through all objects. Important note. We don't try to render in straightforward manner. We sort all objects by types, by materials, by textures because, as I said before, because of variations, we have to minimize number of context switching of state switching. So it's much more efficient to render things when you don't switch different states frequently. That's why we combine them by material, by shader, by textures. So it's not so visible, but they are combined. So let's first talk about material, about debugging. We use different software. Mostly we use epitrace. It's awesome software. It supports most platforms. So it works perfectly on Windows, not all, but most platforms. It works on Mac OS, Linux, and so on. The really good stuff about epitrace is it even supports all GL. For example, if you have old application, for example, old game, you might try to trace it through epitrace. Another one, like RenderDoc, or NVIDIA inside its RenderSpecific, but they have problem with it. For example, RenderDoc can run GL2, GL1. It requires at least GL 3.3. It's okay for modern, but for pretty old software, it's impossible to run. But you might try to debug, for example, some parts. So basically we use epitrace. It looks like epitrace. It's already prepared to race. Usually you have two steps for epitrace.  And then you got a trace file that you open with epitrace or any other UI built. For example, it's Qt-based. So you can look at all calls that your application has done. It's pretty helpful to understand what's going on, what does the application really complete, why it's helpful, because you don't want to lock all your low-level AP calls in your application, because it's a lot of work. Also, it's not so fast. Also, it might be threaded. So you might spend a lot of time to try to implement stuff. So just try to use epitrace, and that's a good application when you talk. I think it's better in terms of one-time usability, but it doesn't support OGL. Platform vendor-specific is good when you try to debug on this platform, because it has all information. It has all connection with drivers. So it might show you some additional information about your system that you don't know. So it might be useful for writing graphic applications. So the important part, artists. Why it's important to communicate with artists? Because artists create visual style of your game. So programmers have to understand what artist wants to present in your game. And artists have to understand restrictions. For example, you might have on a frame a lot of units. They are pretty small, for example, a few pixels. But artists can create a texture like 2000 by 2000, and it won't be noticed by any player, because it's too small. You have really crazy density of texture, but why do we need that? So you need to talk about that. And I have three problems that I want to mention about this. First of all, transparent object. It's pretty hard stuff about graphics to render a transparent object. We have modern hardware that can allow few tricks. For example, order independent transparency, but it's another history. If we talk about usual application that supports all systems, transparent object is a problem, because you can't just render it. You can't just render it randomly in different places. You need to save the order, because it's important, for example, if you have red, green, blue transparent object, it's really important in which order they are rendered. You could try to move layers in Photoshop or Gimp, and you will notice what's going on there. Another problem is parallax mapping. It's a simulation of displacement, how it works. It's flat, just with a normal map, and how it usually looks in the game. And if you want to add more 3D effect to your surface than you use parallax mapping, what it actually does with your material, it's like a ray trace. You do different steps to find nearest position of intersection with your height map, and then you take not this one with texture and normal map, but you take this one, and you get a correct view of your material. So do you see the difference between them? That one is much more flat, I think. So because of correct position of the object, you will have a better result. And what's the problem? We have, and we have that. That's how it looks like in our game, but with a different mode. You might notice that the roof isn't flat. And what's going on when you have something wrong? This one. So the roof should be like this, but it's like this. Why it's going on? Why it's going on in that way? And it's not really noticeable, but here, another artifact in the same style. So what's the problem? Problem of normals, because usually when you try to trace, you have an assumption that your triangle where you applied the material is flat. But in case you have normals of different vertices in your mesh, you might get interpolated version of your normal. And for example, if you have this pixel, and noble is looking up, but you're something like here, and you're looking here, what's going on? It looks like you try to look at this point from under the object. It shouldn't happen for the original algorithm. So in this case, to fix that, you need to talk with your artist and say that don't use parallax mapping at all for this mesh or that kind of mesh, or just, for example, switch, divide your mesh or more triangles to reduce such effect. And the last one, MIP map. Who knows what MIP map level is? Okay, let's me explain. To save whistle style, and when you have an object on a scene, you might render it on a different distances. If you try to render all objects with the same texture, you might got a flickering, because when you try to render a pixel, the coordinate in the disk texture will be calculated on each frame, depending on a position. The coordinate of such pixel on this texture will be different. Because of that, we will get flickering. To avoid that, we have such technique like MIP mapping. When you have such texture, so original one, this one, and all these squares that are smaller are MIP map levels. For example, it's zero level. It's one, two, three, and so on. So if you have a small object, then GPU will choose not this one texture, but this one. And you won't have a flickering. Why? Because when you calculate a MIP map level, you use average. It means that when you try to pick a color, it will have a smooth transition between coordinates. And you will have a stable color. Let's look at this example in our game. What's going on? This one is near distance, average, and very far. What's going on? You might notice that normal tree, almost normal, looks strange. And what's going on here? So you don't have any leaf on this picture. Why it happens? Because of algorithm how MIP map level calculated. It's just average. And on low level of MIP map, you will get just mostly transparent object, transparent texture. So to fix that, you need to talk with your artist who calculate MIP map levels and tweak patterns that are responsible for MIP map leveling. So we don't have any stable solution for it because each textures can have different behaviors in different situations. So it's important to know about that case and account that and tweak numbers for different textures. That's all for MIP map level. So that's all for now. And we are looking for an equation. Present? Yeah? What part of the pipeline are you currently working on? Are you just fixing problems? No. Actually, at the moment... Okay. The question was about what I'm working at the moment. Working on. Number one, please. Actually, at the moment, I'm working on sorting stuff when I told about material sorting to minimize switches. So at the moment I try to profile different scenes, different objects, and try to minimize time that we spent on this because at the moment we have CPU bound, not GPU. So in our game, in big scene, GPU isn't really loaded about 10% by CPU, one core that is used for sorting about 100%. So that's our goal, I mean, our graphics goal to reduce the time that we spent for sorting and related stuff. Yeah, I have the same question. Sorry? I have the same question. Yeah? Do we have plan about metal? We have thoughts about that. What we think? I think and we think that it might be more useful for us to use Vulkan and use something like molten VK for metal because we are open source. It means that we usually don't have a lot of hardware, a lot of graphics programmers who can spend time exactly on this API. So for us it's much better to use some small amount of API set of API. Okay? Thank you.


There we are. Okay, so hello. My name is Patrick. On GitHub, you know me as Raghvip. I'm one of the maintainers of the Backstage project, and... Yeah, I'm Copenhagen, and I'm working a lot on Backstage. Yeah, and also maintain. And we started the same day. Yes, we did. So today we're going to talk about how we are evolving the Backstage backend system. So we're going to look at what it looks like today for building backends. We're going to look at where we're headed and why we're headed in that direction. And we'll also dive a bit... That's better. That is much nicer. We'll also dive a bit deeper into how the new system works and end with a bit of a comparison. So first I want to give you a bit of history of the current back-end system and how it came about. So when open sourcing Backstage, we had a pretty well-built out plug-in system for the front-end. But on the back-end, there really wasn't much at all for plug-ins. And we had also taken the decision to rewrite all of our back-end systems into Node.js. So we basically started from a completely clean slate. And we started very simple, just with a pattern for how to set up an HTTP router to serve the back-end APIs. Then over time, we added more plugins. Things grew. We added more utility services for things like logging and configuration, database access. And as back-end plug-ins got even more elaborate, we started seeing extension patterns for plug-ins. So you can now add things like entity providers in your catalog or custom actions in the scaffolder. Now, all of these new things grew out while we maintainers were kind of frantically trying to keep things a bit consistent across the project. So that's led us to where we are today, where we're quite powerful tools for building elaborate back-end plug-ins. So what does a back-end plug-in look like today? At the heart of it is the create router. Every plug-in exports this function or something like it. And it's essentially a plug-in that gives you an HTTP router to serve external traffic. And then through the options for the create router function, the plug-in can accept any kind of options or dependencies that it needs to get started. Now, this design is extremely flexible because it's just a function. You can do anything with a function, really. But with this flexibility, there's a lot of cost. So when you have something that is this flexible, it's very hard to provide any utility. So from the framework's point of view, we can't really hook in and help out in any way because there's no points for us to do that. So let's zoom out and see how this create router method is used. So this code handles the setup of the plug-in, right, where you supply your dependencies from the environment and any other extra options. And it lives in a separate file in each back-end installation. And to be clear, there's a copy of this file in every single backstage instance that uses this plug-in. So this file is just copied over and over again. Well, at least it's pretty simple, right? So what's the problem? It's kind of what we thought in the beginning, too. So, yeah, as Patrick's leading on to here, it does bring a couple of headaches. So let's look at the installation of an existing back-end plug-in today. So these instructions seem to be very plug-in-specific. And in most cases, they're always available to read me. I'm sure this looks pretty familiar to people today. As you might be able to see from the instructions, we first need to install the package from NPM. Then we need to create a file which is named after the plug-in and then import that into the back-end package and then wire it all together in the HTTP router. Now, we're about evolving an existing plug-in. So let's say I'm a developer and I want to add access control to my plug-in. Now, luckily, we've got a permission service that's super simple. So to use it, I need to update my plug-in to accept it as an option to create a router. Now, requiring permissions is a breaking change, and that line that you can see on the slide needs to be added for every installation of this plug-in. This will continue to happen for every capability we add that requires a new service to be passed from the outside. What plug-in authors ended up doing is basically just making this optional. Now, that's kind of problematic because it means we need to deal with optional code from, like, as a producer of a plug-in, like as a developer and also as a client that's going to use the plug-in itself. So let's look at the catalog installation. So this is code that you will recognize, I think, today, because this is basically adding the scaffolder processor to the catalog. So the processor we're adding here is what we now call a module, which extends the catalog with new features. It's essentially a plug-in for plug-ins. The way that modules and plug-ins interact is completely up to each plug-in, and there's no real standard pattern for it. So let's imagine we want to evolve any part of this setup. So let's look at the API surface. So changes to how the builder is constructed can be breaking, changes to how our processor method and the processor interface that can also be breaking, changes to how the entity processor is constructed, that can also be breaking, you get the point. There's a lot of breaking changes that we could make. The API surface is huge. So with all this in mind, we tried to make the situation better for both end users and plug-in developers. So based on learnings like what we've just seen, we basically set out with the design for these higher-level lists of what we wanted to do. So first and foremost, we wanted to simplify the installation of a plug-in. So having to create multiple files and managing imports is just too much of a hurdle, and it's a hassle for people trying to integrate with Backstage. We wanted a much simpler process, both for plug-ins and modules. So secondly, we also wanted a few breaking changes. So by reducing the API surface exposed by these plug-ins, we want to get rid of the situation where everyone needs to update their back-end code when we introduce new features or new services for plug-ins. We wanted less code in the back-end setup and move all of this back-end wiring into the back-end system itself. Like we're going to build a framework, so let's hide away all the hurdles that people have got to jump over. Let's move a lot of that stuff to the framework. And we wanted the system to feel recognizable. So to anybody that's been working with front-end plug-ins, it should kind of just be a really simple switch to writing back-end plug-ins, right? We've got a lot of that framework already built in the front-end, so let's take advantage of that. And we also wanted to make it easy to evolve and extend. So users shouldn't have to invent or learn new pattern for every plug-in. Now, how did we go about building this new system? So we started sessions and design sessions and prototyping at the beginning of this year, using a top-down approach and focused really heavily on the back-end setup and installing new plug-ins and slimming that down to what you need to get up and running. We settled that and found a design that we were happy with and published it as an RFC for some time in the middle of May. And we're all pretty happy with our approach, so we got straight into building the new system. And that was around summertime, and then we've been shipping early versions of the systems for the last couple of months. Now, let's explore what this system looks like with a high-level overview. The base of the new system is the back-end instance. You can think of this as the unit of deployment. The back-end itself really has no functionality at all. It's responsible for wiring things together. Next, of course, we add plug-ins, which house the actual features just like in the existing system. So plug-ins operate completely independently. If they want to communicate with each other, they have to do so over the wire. So there can't be any direct access between plug-ins. Now, because of this constraint, a plug-in is very much like a microservice.  So you can take all of your plug-ins and put it in a single back-end, or you can go the other way and other extreme and split every single plug-in into its own back-end or something in between, all depending on your need to scale and isolate individual plug-ins. Next up are services. So we don't want plug-ins to have to implement everything from scratch, right? It wouldn't be a very good plug-in system. We provide access to services like the current ones, the database, task scheduling and configuration. Now, those are the built-in services, and there are many more, but you can also import existing services, and you can easily create your own custom services as well. Now, like the other building blocks we've seen so far here, services is nothing new. What is different, though, is that they're now provided from the back-end through dependency injection. So this means you no longer manually need to wire up your back-end to pass services to each plug-in. And a shout-out to Jussi from Rode for helping out with that earlier. We also wanted services to be a customization point. So just like in the front-end, where you can replace services. So you can completely replace a service, or you can also take existing service and customize it, like changing the middlewares of the HTTP service. We've also seen these plugins for extensions, or extending plugins themselves, like the processors in the catalog and the actions for the scaffolder. So we have now created a common pattern for these types of extensions that we call extension points. So they function a little bit like services, where you depend on them and they're registered into the back-end. But rather than being provided by the back-end, it's the plugins that provide the extension points. Based on what each individual plug-in wants to expose, what kind of customization they want to expose. And extension points are also decoupled from the plugins themselves and exported separately. So it's much easier to evolve and deprecate individual extension points, rather than dealing with one single huge API service. Now, the last piece of the puzzle are modules, which use the plug-in extension points to add the actual features. So a module might, for example, add a catalog processor or a scaffolder action. And modules are quite similar to plugins. They also have access to services. But one key difference is that they must be deployed together with the plug-in that they extend, and each module can only extend a single plug-in. So in many ways, a module is a plug-in for plug-ins. So let's take a look at what this new back-end system actually looks like in practice. So all of that set-up code that you used to have in the back-end has now been distilled down to a single function call, create back-end. By moving all of that set-up code into the system itself, it's a lot easier for all of you to maintain the back-end, and it's also a lot easier to evolve and extend over time. There's plenty happening under the hood, of course, like wiring up all the built-in services that everyone has to manually configure before. We also choose to split out the startup into a separate function call, so that the API is a little bit more flexible, and it also makes it easier to add custom logic. This code, obviously, is not going to get you much more than just like a bare-bones back-end. So let's add the catalog plug-in. So rather than having to manually create and copy the contents from the readme that we saw before, we now have everything in one single call. So note that there's no arguments being passed into this catalog plug-in, and all of its required services have been provided from the back-end through dependency injection. This means there's much less code needed if you stick with the defaults, but it's also possible to make decustomizations. So you might notice that the catalog plug-in is a function call. That's because we give you the ability to pass in additional options for making lightweight customizations. So now let's look at the example of installing a module. So this is probably conceptually the biggest change that we've made from the previous system. As we mentioned, modules now are first-class citizens that you can create and install rather than some loose pattern invented by plugins themselves. So in this case, you can see that the scaffolder module extends the catalog by adding a processor for software templates. So let's see how this thing works under the hood. So I just want to make this clear that this is not code that you have to write to integrate with plugins. This is something that you would make as a plug-in author. So to create modules, for instance. So this is how it's implemented in the scaffolder, for instance. So we'll start with the scaffolder module because it's a bit simpler. You can see that we can create a new module.  And as implied, that only means that a module can extend a single plug-in. So in register, we add the init function for this module. We declare a dependency on the log of service and the process and extension point from the catalog. If you're familiar with the front-end plugins in Backstage, this is the same system that we use for utility APIs. As all of our declared dependencies then get passed to the init callback. Finally, in that callback, you can see how we use the catalog extension point to add a new processor. Modules are always initialized before plugins, so therefore there's no race condition between each other. Now, looking at the catalog plugin, you can see there's a lot of similarities from the previous module example. Again, this code is housed in the catalog plugin itself. So the main difference between... The main difference compared to the scaffolder module is that we register an extension point to be used by the modules. The reason we settled for this particular sort of nested in callbacks is so the extension point implementations can be accessed directly from the closure and the init closure itself. This makes it much easier to build extension points since you don't need to juggle internal and external services and dependencies and interfaces. This is a slimmed-down version of the catalog implementation, but in reality, the catalog plugin is configured exactly like this, today. The upside is that this code can evolve without having to get users to update their code themselves. And lastly, for people that want to go a little bit more extreme and do a little bit more customizations that we talked about, you can see how we can replace service APIs just like we do in the front-end. So, for instance, here we're replacing the logo with a cloud logger instead. We also added the ability to supply default implementations for new custom services, which means that you'll be able to import and use them right away without having to wire them up beforehand. All right. Let's look at some comparisons between the current system and this new one. So, in this example, I have a Kubernetes client service in an external package that I want to import and use in my plugin. So, if you want to do this today in the current system, there are a lot of hoops that you need to jump through. So, obviously, you need to import it into your own plugin and wire it up and use it. But there are also many changes that you need to ask users of your plugins to make to their backend installations. So, first off, you need to ask them to do what you see on this slide and add the packages to dependency. Then they also need to update the plugin environment type to include this new service. And they also need to update the backend wiring to create a manager and a plugin-specific instance of this service and forward it to the environment. Lastly, they also need to forward the service from the environment to your plugin. Now, that's a lot of wiring that everyone has to do for something that really should be simple. So, in the new backend system, this is a lot simpler. Thanks to the introduction of default service factories, all you need to do is import and declare dependency on the service. As long as the service has a default factory, that factory will be used to create an instance of the implementation for the service at runtime. Now, this means that users of your plugin won't need to make any changes to their backends at all, unless they explicitly want to override that service implementation. One of the more visible changes between the new system and the old one is the reduction in code size of a typical backend installation. So, comparing the setup of a newly created project, there's a quite massive difference. So, rather than nine separate files with 348 lines of code, we now just have a single file with 26 lines of code. That's more than order of magnitude difference. So, overall, this gives an indication of how much easier it will be to manage a backend installation once we've rolled this new system out.  And... Yeah, there will be some paint upgrade. But, as we saw on the last slide, it's mostly about deleting a bunch of code, especially if you have a somewhat standard setup. So, you delete the plugin-specific file and you replace it with a one-liner in the index file. Now, any customizations through the backend wiring, if you've done custom services, that would need to be replaced with a custom service implementation. And any plugin customizations would be either options or migrated over to be modules instead. Now, you'll be able to find both general and plugin-specific guides for this in our upcoming migration documentation. So, overall, this is going to lead to a much slimmer code base and that is going to be a lot easier to keep up to date. So, what about migrating the plugins themselves to the new system? It might get quite complicated, right? Nope.  You depend on whatever services the plugin needed before and you also depend on the new HTTP service where you register the router and then you're good to go. Now, the plugin can accept options for any of the more lightweight customizations but if you do have a plugin with more complex patterns, that probably needs to be migrated over to extension points instead. So, in summary, the backend system simplifies the installation and it will be a lot less code for you to manage and there's going to be way fewer breaking changes. If you use to the frontend APIs and even the current backend system, these patterns are going to be recognizable and it is going to be a lot easier to evolve and extend this new system compared to the current one. All right, so what we've shown today is available for testing so if you like what you've seen, you can give it a spin. We'd love for you to share any findings or feedback either directly with us here or in the backend system channel on Discord which I just created an hour ago. Now, you can also read more in-depth documentation that will be evolving as we build the system out at the link at the top and we don't have a specific date in mind for a stable release. We really want to give this time to settle and respond to feedback but we're hoping for some time early next year. So, hopefully we have time for some questions but you can also find us here, of course, and later on at CUBE, come. Thank you. Martina will run around. So, if there's any questions, raise your hand. Objections about migrating again. What are you expecting that to be available for us to expect? Available. Can't promise anything. So, we're hoping next year but it really takes time, especially from the experience with the frontend building out the frontend APIs as well. We discovered things that we didn't think about that need to be addressed so it does take time to build this out but the more feedback we get and the more people try it out the quicker it's going to be and also the question was when do we expect this to be available? So, question is what about running this system in parallel with our current one going to be deprecated? I hope so. It's basically the answer. So, we haven't talked a lot about running them in parallel in the same backend. So, as mentioned, plugins, they are quite a lot like microservices. You can really split them apart and move them over one by one from two different installations. If you have a big and complex setup sort of a related question if you're running multiple replicas for the same deployment for pods, for example how how friendly is this migration going to be for not killing your three other pods as one of them updates? So, I guess how much testing of that has gone on? So, maybe struggling for the answer. So, I guess if you mean you want to replace the migration for running a new system. I guess the APIs have the contract. The APIs of your public-facing APIs like the catalog API, whatever you're using that's the contract, that's not going to change. It's just the wiring together of the backend system that surfaces these plugins that's going to change. So, I guess in terms of downtime I don't think you should have any I don't think there's any migration from an end-user point of view it's just integration point of view a plugin or a point of view. Does that answer your question? Sorry, I'm unsure if that was the answer. I guess for clarifications' sake database migrations I'm assuming this comes with database changes. No, I don't think it does. No, it's nothing like that. So, this is basically just all JavaScript code that does all the wiring of the backend together. It's been 19 much happier. The plugins are all the same still. It's just the actual export of things from the plugins. So, the database migrations are contract somewhere else. This is just what the plugin exports and how we wire everything together. We got there in the end. Thanks. I'm curious if that's the right forum. What's the roadmap of data model evolution for Backstage? So, data model evolution and Backstage. I think the catalog SIG is the best place to show up and discuss that. Yeah, I would say so too. In terms of roadmap and what we have planned kind of hard to say. I don't really know yet. I'm going to defer this to product at some point. But yeah, not sure. I think SIG is probably the best place to start. We can start driving that roadmap and driving our next priorities, I guess. It's a catalog SIG. So, you can find information about the catalog SIG in Backstage slash community, I believe? Yeah, good. Charles has a question. This is for Charles. So, this is Charles's question. What are your thoughts on being able to unmount a module once it's been activated? Like at runtime? Dynamic plugins. I remember that discussion. We're looking at most likely introducing some form of lifecycle service to kind of hook into events of start-up and tear-down. Just especially for start-up where you want to get HTTP serving listening kind of at some specific point when the backend is ready. No concrete plans yet. Work with us to implement it. Sounds good. I think that we have no more question. Well, one last question is how do guides and as well as some video to show, because if I'm kind of a little bit looking at how much effort we need to put in migrating the current one to the new. Or maybe what I was asking automated tool or maybe you can build some automated plug and you can discover like, okay, you know this is the change within this backstage I guess maybe it's possible the pattern of code mods is what you're essentially talking about. So the question is can we make migration easier? I think it's hard because there's quite a lot of hundred and code already that's kind of very specific. When we mentioned in the presentation that there's no real standard for it already so we're going to set these patterns and make it easier. There's going to be some effort but mostly hopefully going to be deleting code like Patrick said, which is always good. We always want to do that. Of course with our documentation of course. Do we have some documentation already? Some documentation is up. Migration documentation is not up. So also where the backend system is kind of coming from is the pain to migrate or upgrade. So that's a good address as well with this new system. Great. So give it up.