Hi everyone, I am Matteo Collina and today I'm going to talk to you about community frameworks because yes, in the JavaScript community we really like frameworks and maybe a little bit of my personal history, well probably not, you don't care, but I think that I have a story to tell, anyway, if you have some time please follow me on Twitter, it's, you know, I tweet about open source things and JavaScript, so yeah, maybe you would find it interesting. So let's jump into it. Long time ago I was, I wanted to be an open source developer and you know, it was, and I tried very hard and I built things with Ruby, with CoffeeScript, with Node.js, JavaScript, I, you know, I was, I tried very much and I tried very much to become an open source developer. I really wanted, I really, really wanted to do it. And yeah, I don't know, I don't know, it didn't go, it didn't go really well. You know, it's one of the key things that I wanted to do was to build an open source to help other developers build, build amazing things. And yeah, this was one of the greatest stuff that, you know, I wanted to, I really felt I wanted, I wanted to do this. And I wanted to help others, other developers building, building software. And probably don't feel as miserable as I felt when I started and so on. Well, I don't know, something like that. A little bit of the, of success came when I built Mosca in more or less 2013. At that point in time, I was, I was doing my, I was doing my PhD and essentially it's, as part of my PhD, I was researching things on the internet of things. And I needed a very flexible MQTT broker. MQTT is, you know, fantastic project, check out Node Red as well. So hey, we are part of the same kind of family. And when, when I was developing this broker, you know, I made a few mistakes. The first one was, it was a completely unmaintainable, really unmaintainable architecture internally. I made some mistakes. I didn't know how to structure a large and successful Node.js application back then. It's also, I never spent enough time in, in automation. And it making sure CI was stable. These was actually really problematic, especially if you're building a distributed system, working in distributed systems, this is really, really, really good. I didn't spend any effort in building a community of maintainers. I wanted to retain ownership of the project. And however, it's, it really got some nice traction. And it's, and with it, there was a, I don't know, maybe I can, I thought, well, you know, even though I did this as part of my PhD, this is part of my PhD, maybe I can make some money out of it. You know, it can help me, it can help me grow my career. However, I had no real funds. I didn't want to raise much, was not much of venture capitalist in Italy back then. Not that there is much now, but a little bit, things a little bit better. And the project essentially failed. It was a galactic failure for a few reasons. Well, the main reason was that, you know, I built a monster.  It was doing too many things. It was not modular. It was not, it had no community. And I thought it was, it was an era. I was an era and I could actually develop and maintain the software and make it great, which turns out the case. That's not our open source. That's not how open source work. And, you know, that's it. So what can we say? I also add a few different things in this. One is some company, which I'm not going to name on this talk, took my software, paired it with some other software and of some really, really good developer living in London and some other software from another great developer living in the US. And they cook it map together. They change the names, reach the color. So it moved from being a front color to be of another color. And they packaged it up and sold it to a very, very large company for tens of millions. And none of the three people that I mentioned actually made any money about this. Yeah, this didn't really go well for my, for myself, really. In fact, whenever this niche was developing and when it was starting to develop some of the, some of, you know, the project was growing a little bit and it had a community. It started to have the beginning of a community, maybe people, you know, started complaining and they started saying, well, I'm going to stop using this, your library, because you're not fixing this book. This was actually really, really bad. And it was like, it felt really bad. I don't know. I felt bad. I was like, I was completely and utterly destroyed by that, by that sentence, which is very strange, right? Why? Well, you know, why nobody is appreciating what I'm doing, why they're not, they're not, you know, I try to help the community, I try to help people, why I'm not, you know, being successful at all. Well, and then I started following in a bad trap, which is mostly called the imposter syndrome, you know, in, in many challenges, we can be personal, professional, professional. We are held back by the crippling thought that people like us could not be successful.  And, you know, this kind of what happened. And in fact, the reality is that, you know, any of that, that's not, there's not such a thing. You can know something, you can have something to say in the word, even if, and that's probably not what other people have something to say in the word. So there's no, there's not a thing as imposter syndrome. So, well, there's such thing as imposter syndrome, but, you know, you're not an imposter. Believe in yourself. In fact, you know, the key thought that resonated well with me was, and I started asking myself the question if I was just building software for myself. And this was, you know, who am I doing this software for? But I'm not, you know, I want to improve the experience for other people, others. I want others to enjoy my software. So it's, it's really, really tricky. And it comes for, with a lot greater responsibility, to be honest. Because there's a lot of other things that, you know, are required for open source to be successful. You need a product strategy, you need documentation, you need to have a developer experience on the product. For developers that are contributing to the project itself, you need to have a release strategy, you need to have long-term support, semantic versioning, a lot of things. People really like 1.0.0, by the way, and 2.0.0. You need to have licensing. You need to have reliable tooling, tooling, including CI. You need to have, you know, some form of outreach and promoting the project. And you need to provide some support for your user. So, yeah. The problem is that there was no chance, there's no chances that you can do all of that alone. Okay. I don't want to be delusional, but it's essentially too much work if somebody wants to do it only, and I say only, on their own. So, and with no money.   I don't know what to say, but that's the reality. So, one of the things that you want to do is you might want to think, well, I can just do a startup. I want to do a startup to convert, to make sure that my project is safe. And by creating an economical company behind it, I can make sure that it's safe and successful. Well, you know, having a successful startup is hard. Doing a successful open source project is hard. Having a successful startup around open source is even harder. And remember that any cloud provider can compete with you tomorrow. And that's actually a very, very, very, very big problem. So, because they have unlimited resources, essentially. So, well, the question that you want to ask is that if there is another way, is there another way where you can target when you can be successful and create something?  And an open source project is as good as it is its community, which is really, really, really, really critical. So, the only thing you need to, if you are, if you care about your open source project, and you want to build an open source project, you only need to care about your community. Who is your community? What are you talking to? That's what you should care about and where you should talk to. So, well, let's see, let's try again. Let's tell another story. And let's see what can do, we can do differently now. Okay. So, back in 2013, at this exact same time, when I was developing Mosca, I also built another product, another library called Levergraph. I don't know, you probably don't remember this. You shouldn't. Okay. The type of research that they've done in the, for developing this thing ended up spread around the world for other interesting things. So, I don't care. But what happened is that this is Levergraph was a nice graph database built on top of Leverapp, which is a wrapper for LeverDB. By the way, Leverapp is phenomenal. You should check it out. Okay. And this is, all of this is developed in the Lever community, which is a community first, and then a collection of module seconds for creating transparent databases. Note that in November 2022, which is sounds are really long time ago at this point, Rod, Rod Vag wrote about open open source. And, and he stated that individual making significant and valuable contributions are giving the commit access to the project to contribute as they see fit. This is critical. This is how you grow community. This is how you open it up and you enable others to join the project. In fact, it's that has put in the seed for open governance and the Node.js way. In fact, in Node.js, our collaborator is responsible as commit access to Node.js node and it's, they maintain, they collectively maintain Node.js. However, every collaborator can nominate somebody else. So, and nominee and the nominees should have significant and valuable contributions across the Node.js organization. Those are two excerpts from the node governance. The node governance is such a big file. So, you should, you might want to, you might want to check it out and read a little bit more, but it's actually very an interesting read and it's also really, really fascinating. So, let's keep going and see where we are. Now, if I am going to be asked to receive this question now after seeing all of this, and after knowing what a good community of users is, when somebody tells me that they're going to stop using my library because they're not, I'm not fixing a bug, well, there is only one answer to them and it's thank you. Please stop. It's really, really better if you use something else. And the reason is, is that they don't, you don't want to be filled to felt pressures in knocking them. And they don't want to be part of a community. They just want to free be, they just want free be, really. And well, everybody likes free beer, but there's no such thing as free beer to some extent. Somebody is, is paying for it. So, you don't want, you really, really need to, to do this. And they're not really useful.  Okay.  Let me, let me, let me repeat this. Don't be the person that asks for a fix on a library without being willing to contribute any time into fixing that bug yourself. It might be that you need to contribute a very well written script to reproduce the problem or, you know, spend time to actually fix the bug yourself, but you need to help. Okay. Be a very good citizen of open source. So, let's keep going. And, and, and, you know, it's, even though I got involved in, in several successful open source projects, it's still sticking to my hem, into my mind on how can I build a successful and launch a successful of a sort of open source project myself.  You need to be able, you need, you know, you need to build something that you have, I use some unique perspective and you want to need to be something to say on that specific field. The second bit is that you want to be able to attract people that are interesting in solving the same problem and then build something together and share ownership. And, you know, it's, it's all about community folks. So, you know, that's it. Sorry, folks, you know, so let's keep going and talk about my experience. So, you know, in, in, in 2016, I was, in 2016, I was, I was in London and it was Valentine's Day. I was on a business trip. My fiance was not really happy about it, to be honest, but you know, it was, it was a business trip and I was, I had dinner with my colleagues and after that we spent the night coding a Flanger tool. Hi, Dave. Hey, miss you. So, in fact, and I found my niche. Okay. And I call it the, the adapter problem. Now, this is a piece of paper, right?  Okay. Like, I don't know, I keep folding my piece of paper, but you know, there is only a limit of time to time where I can fold it because after a while, I cannot fold it anymore. You see, it's too hard. And the problem with this is that you can only have so many abstractions before your code becomes extremely slow. And I got focused, I focused myself and I focused my career on improving performance of Node.js and Node.js applications. So, one of the first things that I built is Pino, which is a really nice logger. And wow, it's, you know, it took the, I presented it at a nice conference in San Francisco, then at a bunch of other events, somebody dedicated a poem to me and Dave, which is the other author of Pino. Thank you, Emily. It was a great moment and of Node.js lore. And in fact, in the key moment where things change was when I started to ask you know, a stranger, would you like to help maintaining Pino? This was one of the greatest moments because they said yes. And first of all, I met a friend, which was, you know, something that probably a life, hopefully a lifelong friend. But, you know, we have been chatting and working together for the last few years. And he, you know, he has been thanks to Pino and thanks to thanks to Pino, he went thanks to working in open source. He was able to change jobs, change jobs, change life. He increases his salary significantly, so hey, but it was kind of big surprise. And I have seen it firsthand how helping people, helping open source and building a community and being part of a community can help somebody career and can change somebody's life. So this is one of the greatest things that I think I am proud of. So, you know, Pino is now very successful, is the fastest, longer for Node.js. And it has very minimal features, so don't expect much, but it has a great community. And it's right now downloaded maybe four million times per month, which is, you know, pretty successful to some extent for my point of view. It has four active collaborators and it has now reached version six. Okay, successful library. So next, I started asking myself a nice question, is how can I build a new web framework for Node.js? Well, why do I want to build a web framework? Because I was not really happy about the performance of Node.js applications. And I was doing some benchmarking and a lot of consulting and turns out that there was some lot of time spent in Node.js framework and inside Node modules and inside libraries in there. And I wanted to fix that because most of the time, the CPU time should be spent in running a business code, is running the developer's application. The framework should not be in the way to gather good performance.  And in fact, I didn't want to, I knew I couldn't do that all alone. I actually, I've learned my lessons. So I started, I wanted, I really needed to build a community because this was, it's a Gargantuan work, like shipping a new web framework is Gargantuan. And I really wanted to build and I really wanted to build a community. So the first thing that I started is I didn't want to start developing the framework up until I could have find another human being that would have been interested in the journey, in sharing the journey with me and building with me together. Now, this has been one of the founding moments of, for me, because I was saying, I basically said, well, the open source, the code matters so little in what I want to build that I'm going to focusing on convincing somebody else. Right about that time, I ended up, I was doing a Node.js workshop, a Node training in Italy. And one of the attendees of my class asked, hey, Mateo, I would like to get started in open source. And I said, well, where do I start? What do I want to do? And then how can I do it? And then I said, well, I don't know. I would like to start working on a new web framework for Node.js to solve all these problems that I'm seeing on the others. And he said, oh, interesting. And then we started working together. And we started developing software together. And we were chatting every few days. We started tinkering. We spent long nights maybe doing stuff. And well, all remotely, by the way, all on GitHub and pushing things whenever we could. I helped him find his next two jobs. So again, I'm feeling very, very proud. And yeah. So it went, it went really, really well. So anyway, let's go and start about the genesis of a framework. And it's really, that's what led to Fastify. And Fastify is the framework with myself and Thomas, Delavedola, built together. And what we have started. First of all, it's based on open governance. In fact, it was, from the start, it was never to be my own thingy. It's a shared thingy. It's not my project. It's our project. I probably had the idea and some of the funding moments, but it's not mine. I'm not the first contributor of the project, okay? To be clear. Thomas is, I am not. And it's community first. We welcome first time contributors. In fact, we want to grow our community of collaborators. So if you want to join and want to contribute to a nice open source project, please come. We have a lot of things to do, really, a lot of open issues. It's also based on the concept of shared ownership. So if we don't want to be in a position of you asking us for a fix, for fixing a bug, we want you to fix your own bugs, okay? And that's how we grow our community. So this is everybody's frameworks. So everybody needs to contribute. It's also based on some few very important technical details. So first of all, we didn't want to have much overhead in production. Now, you can have, you need to have some because it's clear. It's not rust, okay? Where it has real overhead abstractions. But you can do pretty good things with Node.js. So you can actually have very little overhead. We want to have a good developer experience. We wanted to work both for small and big projects.  And which should work well for both. And so it can be something that I can learn on my own and then use it at work whenever I get the chance. And it should be able to easy to migrate to microservices or serverless and back. So we needed to support both monoliths and microservices and be able to use both to migrate to both models very quickly without needing to rewrite the full code base essentially. You need to have a good security model and have a good security reporting. And by the way, thanks for providing this. It's based on the concept of plugins. So if something could be a plugin, it likely should. So we created an open core system where it's highly extensible, even though it's, you know, cost free. Those mechanisms are cost free. And it's highly extensible. You can plug in and add things in several places of the framework and customize it the way you want so that you can develop your own plugins and tune it to be the what you need on your project. It's also easily testable. And so code bitten will fastifies easily testable. It has a nice dot inject function, which is fantastic. And then it's, we didn't want to do any monkey patching off core of Node.js core. That is one of the most of my main critics for a lot of frameworks. So no monkey patching. Sorry. And it supports semantic versioning and have long term support strategy. And it should, you know, add there to HTTP one dot one. So fastify was then born. And it's right now one of the fastest web framework for Node.js. It's, it has 10 active contribute collaborators. And it has, you know, now rich version trees. No, it's in release candidate stage as a time of this talk. So, okay. And it has now an ecosystem of 140 plugins. And it's actually very easy and fun to use. So please, you know, check it out. I just want to say a few parting words, which is people use software I built. And there have been some, and I'm part of some great communities, Node.js, Pino, Fastify.  And with this, I want to say thank you. Thank you for your time. And please check us out. Nearform is a professional services company based in Ireland. We do all things JavaScript. So if you need any help with Node or React or whatever, just hook us in and we can probably help you. And I just want to say thank you for coming to my talk.  And see you next time. Bye-bye.


Can Wing Chun really be learned in just a short time? Yes, you can learn Wing Chun in a short time. The way that I've designed the course is to use methods and training and teaching methods that have been proven to teach people in the fastest way. So instead of focusing a lot on the academic and the theory and the concept which are important, I focus more on doing. And that's how babies learn. They learn how to do something by doing, not by reading a dictionary. That's how they learn to talk. That being said, Wing Chun is actually designed backwards. Now if you look at the first two forms in Wing Chun, Xidun Tao and Chang Qiu, they're actually designed to help you win the fight. And the later forms, Biu Ji and the Wing Chun Dami Form of Gan Zhuang is actually designed to help you when you're losing the fight. So you start learning Wing Chun how to win fights as fast as possible. That's how Wing Chun is designed. Now with our Accelerator Wing Chun program, I've actually taken that system and then refined it even further to increase the speed in which someone can start from without any experience to become someone who can use the techniques in the fight in a stressful situation in a matter of just a few months. So one of the coolest things about our Accelerator Wing Chun program is that what you learn in that class, you'll be able to use that night in the fight. It's not a program where you have to train for three years and maybe after three years, you may be able to use it with a low success rate. So the techniques that you learn in that class, we test it even after the class and then we see if you can use it in a stressful situation. And then from our experience, people can use it anywhere from 70% to 80% success rate. So that's how fast people learn in our class.


Today we did a dynamic display with a T-72 and a Leopold II. It was a joint mobility display for the NATO Day and the Stop. Speaking with the Czech soldiers and the German soldiers, we're bonding really, really well. It's almost as if we're working together in the same army. As an armored battalion, us working with, say, the Czech or the German military with their tanks and their version of strikers would be very beneficial. I think the U.S. plays a special role in NATO. Obviously, the 64 is one of the higher assets of our helicopter side, at least. So, you know, bringing it out here and showing some of the capabilities of what we can do and letting people crawl on their cockpits and then letting some of the kids get up in there is just great to see how happy they are. We've been working with the Czech army and a couple of German soldiers, so it's really nice to keep that continuity between us and them, and it's pretty exciting to see another military branch. So, being the 100th anniversary, to be a special partner, it's pretty cool to show them what we got.


Hello boys and girls, welcome to my tiny channel. I've been working on this O8-09. O9 Lexus 250 IS 250 with the 2.5 V6 engine and I'm doing spark plugs. Easy job, you know, six spark plugs. 30 minutes, you're done. Well, not really. The passenger side is, it took 10-15 minutes, but the driver's side is taking me much longer and I decided to make a quick video on it. Okay, so kind of reverse, if you're doing this job, kind of a reverse engineering, I guess. Why it's a bit more difficult than usual, because the intake is in the way, mainly this bracket right here blocks this Mir Arnavot numbers, how they, the cylinders are numbered, but anyways, this driver's side middle spark plug, that's why you cannot do this without taking this off. This one is somewhat easy, accessible. This one you could probably get away with, this is the problem right here. Anyways, let me show you. It's not very difficult if you know where the bolts are, so there are two kind of hidden ones. There's one right here and it connects right over here to the throttle body and there is one here, which connects to one of these. You can access this one right from here. There is a bunch of harnesses you got unplugged and a fuel line right here that's connected to here and this is the bolt for it. Don't lose these washers. Look at this oil that came out out of the throttle body. By the way, every time you got your, you have access to the inside of the throttle body, clean it up.  So I'm going to, I actually did it. I'm gonna do it again. Yeah, all that oil is coming from inside the intake manifold. Check this out. Look at the carbon buildup on that valve. Hopefully you can see what I'm seeing. That's quite a bit. That's direct injection for you. Yep, for those who may not know, the difference between port and direct injection. It's mainly where the fuel injectors are located. In port injection, the fuel injectors would be located inside the intake somewhere along the way before the cylinders. Fuel would enter the combustion chamber with air together. The fuel will wash up the carbon buildup on the intake valve we just saw. Actually, they would never get dirty. They would be pretty clean if that was, if this was port injection. Direct injection, the fuel injectors are inside the combustion chamber. So basically it sprays fuel directly into the cylinder, pretty much, bypassing the intake valves. Now, some cars have both. I think the 350IS has both direct and port injection. This one only has direct, but I'm not sure what this fuel line is all about here. It is connected to the intake manifold, like I said before, which is right over here. I guess it somewhat cleans the intake. This is kind of soft dirt. This buildup is soft. There's just still just a lot of oil everywhere. So with the spark plugs, get yourself intake manifold gaskets as well. The black ones here, the rubber ones go here and the blue ones go right underneath this kind of an upper intake. I guess I'm not sure what to call this part, but let's say let's call the upper intake or the lower intake. I'm sorry, lower intake. Pretty simple at this point to do this. This car actually has 101,000 miles on it. It has kind of trouble starting when cold. I won't be replacing this. There is no air leaks. I am just doing that gasket, the rubber ones, and the spark plugs. Now these are old. I'm pretty sure these are original spark plugs.  You can see these are burned up a bit, especially this one here. I'm just going to replace these three spark plugs. I already did the other side. Replace these gaskets here. Clean this up a little bit. I already actually wiped this off. This was head dirt over here.   So when you take these off, you don't get dirt inside here or here. I mean, this will still get dirty once you take this off, because you won't be able to get with your compressed air underneath here. Just carefully take this off. Put it aside here. The reason why this is like that, because the throttle body has coolant going through it, you don't need to take this hose off.  You'll still gonna have some coolant spill, but it's not gonna be a lot. But you can just get away with, this is the only thing that I did not disconnect, these two hoses here. Okay, you're in an outlet. So there's plenty of room for you to put it aside. Unplug the coils first. Careful, see how this one came out with the seal. This one didn't. Have a look inside. Look for oil. If you do find oil inside where the spark plugs sit, that means you gotta replace the valve cover gasket. Check this out. This is this last one here. The gap is 51, 41, and 50. This is a new one. Let's see. 42, 43. These are pre-gapped. Don't put cheap spark plugs in here. These are iridium. NGK is what I'm putting in. About 20 bucks a piece. I'm gonna torque these down to 15. 15, 18 should be all right. You could probably get away without replacing these. The rubber is, well, it's not soft, super soft anymore. But why risk it? I don't remember how much these seals wear. So I've just noted these actually matter. The way they are shaped, see this is kind of almost like an egg. This part does not have, this is more round. This is kind of more flat. Okay, so this fitted perfectly. Let's see what happens if I flip it. See, this one seems too big for this side and this I need to stretch. So pay attention. Okay, I did unplug this. So first plug this in right there. This guy here, we're plugging right in the back. So I gotta get this closer to the firewall. And now plug in that white plug. There we go. Click and take a look. Last look underneath. Check the seals. They're all still in there. There we go. Easy. Here, this vacuum line here. You can squeeze the clip with your fingers. Don't need to use pliers. Plug this in. Another vacuum line here. Here harness in. Now there was a plug underneath this guy here. Okay, that's all I'm gonna do on this side. Before I put this fuel line back in, I'm gonna finish mounting the intake manifold. So this is what you're working with. We got two 10 mil nuts, one 10 mil bolt and four or six allen bolts. So obviously these two, the nuts go here on the studs. This will go, actually didn't need to take this off, I just noticed. This just holds the cover, the engine cover, the plastic cover. Now there's one here, one in there. The allen ones are five mil. The fuel line here, it's actually got a mounting point in the back of the intake. Okay, torque specs for the intake manifold is 15 foot pounds of torque. My socket is stuck. Okay, I was gonna start from the middle but since that one is stuck, I'm gonna get it, actually know what I can do. I have another five mil. Yep, I'm slowly but surely stripping the inside of that bolt. I'm gonna finish the rest of them. This one's kind of sketchy. I'm gonna leave it at 14. It almost feels like it's stripping. So we did one, two, three, four, five, six allen ones and two 10 mil nuts. Intake is bolted down. Probably won't be able to see it but remember that bracket in the back. This 12 mil goes there. Now that fuel line, could not find any torque specs. I'm gonna do 15 and see where it takes me. Okay, need a little bit more, about 18. Okay, that's more like it. Probably even goes up to 20. If it's gonna leak, basically tighten it up a little bit more. Okay, one more bolt, another 12 mil. That bracket that I was telling you about right behind this harness here. Now that I look at it, you would not be able to do the last spark plug with the intake on. All right, this side is officially done. Okay, this little 10 mil. Now what I did was, this actually moves back and forth. I was able to bend it back ever so slightly. So there is this, I think it's a fuel line, blocking the throttle body.  This will be in the way. So pull this back. Don't be afraid. Probably about half an inch or so. Nothing is gonna break. I'm gonna need two hands for this. So on this side, as you can see, all I had to take off was the intake pipe, the box, the intake box that's just the top and this line here. And just, I took this off just to make my life easier. So these two 10 mil nuts off to pull away the harness and then you have access to all the spark plugs. It's smoking because oil spilled. The intake was flipped over down the belt cover and some of it got on the exhaust manifold. Should have sprayed it off before I started it. All right, that's it. All is well. New spark plugs, new intake manifold gasket. Not very a difficult job, just requires some time. Anyways, I hope this helped you out. If it did, give me a like, comment if I missed something or there is a better way. Let me know. Consider subscribing. Also, check out my Subaru STi EJ25 build videos. So thanks for watching. I'll see you soon. I gotta get out of here.


Hi Yes, I'm sorry. I'm not very well actually so I would have brought some demos It's just been a bit of a rough couple of days just got cold But I just not feeling the best and I traveled on Friday, so I'll apologize. I'm from the UK So it was a pretty miserable day on Friday I like to I think myself a fairly inclusive and open person and I'm yeah sad times at the moment But anyway, we'll move on for that and so I'm gonna talk about and I would have had some demos and played some music for you but yeah About a project that I've been working on for a while. I've at the University of West of England with some PhD students of mine Some of them musicians and composers some of them are computer scientists and we've kind of have this mix Collaboration group and in particular We've got this music project where you look is looking at kind materializations of Musical controllers in particular but also DSP code and everything and stuff Although that's more my area rather than some of my students. They're much more interested in composition and in interaction, so HCI stuff and We've kind of built up this infrastructure that is around Physical computing so a lot of stuff some of the stuff I'll show you in pictures is actually building up Physical instruments, but the idea is rather than the musician having to go and buy something off the shelf or more expensive We want to make them so they could potentially build it now, of course, that's not for everyone I'm not saying that but the idea of trying to break down that barrier of having non-engineers build build instruments and things and you know we see software like that some things like PD or max MSP Visual programming is supposed to intense like that, but it's still very much in the programming environments rather than in actual physical computing and of course the maker community has broken that down quite a lot for I mean still a lot of People who are you know tech savvy and things like that, but they're starting to break down those barriers and some things and so we're we're looking at that intersection really interested and In a minute, I'll just give you a flick to that. This is the website where you can find out lots of stuff There's the github links for all the code you'll see that it's not just rust code and that's what I'll talk a little bit about that in a minute but So I mean my motto is over the years. I've worked in industry for a long time It's just used the right thing to solve the problem of course what you think is the right thing is not always well Other people's things the right thing, but that's a separate story and you know I'm no more right than anyone else's so I'm not going to kind of advocate one thing or another I'll just show you the kind of things that I'm interested in and and have been using rust for But in particular, I'm really keen on rust as an intersection with other languages I don't want it to be the solution for everything and I don't want the other things I use to be a solution for everything because I don't think they are so I just Won't make sense is what I do so There's a long history of musical instruments obviously and I'm mostly thinking from a Western perspective I don't at all want to say there aren't many other kind of ways and Ways of looking from an entomological point of view about musical instruments I just don't have that much experience in it So I wouldn't want to talk about it and present it here But from a Western place, you know We've got the kind of classical periods particularly from kind of around Shakespearean time in my history You know where musical instruments started but particularly since we had the printing press You know and music became be able to distribute the kind of musical form of the instrument became quite fixed You know the piano didn't emerge first It was the printing press that emerged and the ability to distribute music made wanted to standardize Things and I'm sure we can all agree that standardization is good in some ways But it also does hold us back in other ways, you know and And so of course we've got to the modern times Electronics, you know the the 19th century had lots of bad 20th century story had lots of bad things Caused by many of the innovations, but also had many incredible things and it's I'm sure as we work in Technology, it's a kind of difficult balancing act, you know Sometimes I feel that we should just throw it all away not by the next phone or whatever and other times I'm thinking wow, this is amazing, you know, so anyway in the UK we had this thing called the BBC Radio workshop was where they did the Dalek soundtrack, you know for Doctor Who and all of that sort of stuff Which is famous and so they started to build these studios I even managed to get a picture of a woman rather than a guy, which was great actually that the Radiophonic workshop was a real place where that ragi women had quite a lot of very innovative Delia Derbyshire is the famous one But there's a lot of others as well and of course in the 80s We had people starting to break away from the more conventional instruments and finding and being able to make instruments out of other things You know, particularly that obviously one is the record deck and making that into an instrument in its own right and Today we have many people live coding is a popular thing And we've got this kind of world of where people are sitting in front of their laptops And you know, I've been to lots of musical events. It's not that exciting to see People sitting in front of their laptops, but the music they make and the stuff they're using is incredible Right, you know, I'm not a live coder myself But I love the kind of philosophy behind what they're trying really that's sorry Yes, you think it wouldn't go to sleep when I got in full screaming Chrome, but it still does anyway I can probably turn it off, but I'm not gonna worry about that now, but One of the things we're really interested in so I'm really interested in digital using digital Sympathies and digital generation of musical instruments And so how we can connect to these digital devices But in maybe a less conventional way not sitting in front of a laptop But also not sitting in front of a keyboard and so we're interested in building things So there's been a lot of work particular There's this out of Queen Mary and the UK this thing called the Bella board Which is very much inspired by the Arduino kind of maker environment, but it's got a very high-end You know DSP and you know particularly the audio the op-amps and all that I you know, they're not in cheap, you know It's for actually it's a very simple board But you still pay 150 pounds for it because the op-amps they're using our expensive not cheap You know and it's a full 24-bit back and so forth and stuff and things It's a really nice little board and it can support it it multiplexes the channels So you can have full 96 kilohertz kind of speed But if you want to have 16 channels you get down to like 22 kilo so depending on what you'll do But it's a really nice and it's you know It's a full Linux stack based on a big old bone and they modify the kernels So they get real time and so forth and of course It's a Linux stack so we can build and compile using our favorite languages Whatever that may be whether it's C++ or Rust or anything else for that matter They do a lot of stuff with PD which is a visual programming language and things because a lot of musicians aren't gonna be further than that So these are kind of conventional interfaces. I won't spend too long on these stuff. Hopefully your things This is the kind of field the maker field These are actually all Bella based instruments that people have built and most of these are not You know not all of them are engineers most of them aren't engineers most of the musicians who have just ended up programming from an For a necessity Looking at some of their code is you know interesting let's say but this guy up here He's built this a complete retro drum machine He completed it and as far as I know before he started this project took him about two years He'd never had any programming experience at all and he managed to do it You know he you know stack overflow helped him a lot and something it's great talking to him It's amazing project, you know to think that this guy is just a musician I spent all of his time mostly playing analog instruments like guitars and pianos and then he just decided he saw a talk on Bella I don't know even though he probably have kind of conservatory or someone someone talked about me I was inspired to build it and it's pretty amazing that we've made that accessible, you know now He first programmed it in PD Then he programmed it in C so you can imagine the experience he had it was not a good experience And he's great. You can find his blogs if you go off my website you can find his blogs about it And it's really great to read and yeah, I think he now complains that he didn't have enough time making music But you know anyway, all right. These are the core sort of things We are making I'm not really going to talk about this much down more going to talk about the kind of language Side of things and everything but these are the sort of things so this is a pressure sensor So the rather than being Compasitive, you know, we have a lot of iPads and stuff and things people doing capacity But it doesn't have any kind of haptic kind of feedback and so that's the kind of a real issue So we took this pressure sensor and we've worked out different ways of Generating Musical instruments or interfaces in particular, you know, I'm not with them very much I'll show you in a second a separation for it and up here for example, these are free These are some 3d printed designs that I've designed that they're connectable They can be a horizontal or vertical and they connect together with just Lego pieces But the key thing about it, of course if I'd brought a demo I could have shown you is that they're all based on magnets So originally I designed them to be using a hall sensor so you could just detect the magnetic field when they move So they but these are actually don't have a whole sensor They rely on the pressure and so but the only reason we're using the magnets is to give the haptic feedback So the button pops or the slider you get just a nice little feel of the actually being resistive You know rather than just sticking and this is a little 2d one Which just drags it centers back in the center if the magnet was in so obviously this picture is taking about the magnet in it But if the magnet was there it would be centered and then you push it so you get so it's very simple and just literally underneath They just felt and things So the bit that this talk is more about is how do we describe these interfaces in software such that they can be materialized in Hardware and in particular this is a made by a company called sensual in Silicon Valley And I think they make touch pads, you know for things like Mac books and stuff and things and They have a really nice Very, you know, it's a you art They have a little a dongle which you can either plug straight into a USB Get a USB connection to it or they have a little dongle which takes the USB and just turns it into a straight to a you Art so then you can connect directly to a Sternal piece of hardware like a embedded STM or anything that supports, you know serial I mean it's quite trivial so all of the software that we've written that connects to this device Not the stuff that necessary generates the interface is all written in rust and so that's what I mean I'm not going to go into a lot of depth about that today, but I mean the punchline which I have at the end is That for me the killer thing about using rust for this project other than I really like the abstractions Which is what I'll talk a little bit about in a minute is that I developed it all on here for about a year and a half Nothing else didn't test anything anywhere else It was all just driving on here because you know, I don't use anything else So I used some Linux VMs for my students, but to be honest, I quite like my Mac I don't use Xcode just you know just at the prompt with Clang and stuff and With rust and when I ported it to the Raspberry Pi to Windows That's the only two platforms so far. We haven't done it because sound is not so great on Linux Two days, I think maybe overall with you know, it just it just I really didn't have to do very much You know, there's a few things that didn't work, you know, particularly the audio drivers You have to build by hand and it's all you've asked you to point it to it and stuff and things But it was pretty amazing. I was really impressed and You know, the greatest thing for me about rust is is cargo and the package manager is just amazing That being said I went to build the demo yesterday and of course I've installed the latest version of rust for other reasons and nothing builds anymore because of all the packages are out a day And so that is still a pain but it's a lot lot better than using C++ without a package manager and building everything by hand and I mean, it's crap right. It's still crap and even I was just working with someone just the other day looking at building Helping them with a C++ app that they got me started to use the Microsoft C++ package manager, which I didn't know about and It would only install 32-bit binaries on the 64-bit Windows machine So I still haven't you know, it's still not great. Whereas the rust ones amazing So I said and I use Haskell a lot and so I use the phone called stack I know people use there's another one called Cabal and I'm not going to get into the political arguments of that But it's still pretty amazing right and in the way that it puts everything silos. It's just great I just want to whoever I've heard the person's here develops it, but thanks because it's really great So cool. Anyway, so we're going to go back in history just for a little bit Just to sort because this is more a talk about how I benefit and what I find I like about rust and why I use it so hopefully you can guess where I started programming I'm quite old now, but you know, I haven't had my BBC micro for a long time, but my brother and I have used to fight over it and He makes films now. So he doesn't do any of this stuff, you know and But I did a lot of this I loved it just typing in code for magazines, you know I'm sure we can all say, you know the history and obviously at some point Typing in basic was not enough. And so you could do inline assembler inside the basic interpreter pretty cool and 6502 it's great. You know, this is where I all started. That's why I went to university not towards the older but it was and Of course, actually the first programming language I use the universal was lisp, which I still love to this day because of its lovely Prem, you know braces, but anyway, it was our but of course we did this. I did a lot of this It was a you know, a computer science and master degree. We had salarious machines We did a lot of C, you know programming at sea and I'm not going to do not I still love see today You know, it's an amazing language. It has lots of flaws and we wouldn't design it the same today And it's Russ. I suppose but I still love it Okay, and then I went and left after I finished my piece. Obviously. I did some scripting. I'm still mostly do bash I'm not really a Python person, but anyway, I Did my PhD and I did Haskell completely and so this is I just want to come a little bit about this the reason I put this up and I still use Haskell to the day I still love it and You know, it's I suppose the alternative to Python or whatever I use it for those sorts of things but of course I worked in industry for 15 years and Yeah, I use this language and you know for a long time. I really thought it was amazing and I really liked it and You know, I'm not gonna deny that. You know, I was a bit naive. I don't know I just you know, there's some good things about it. I felt called I could do template metaprogramming and you know Outsmart people was stupid, but you know, I did But I also did some of this stuff. I mostly worked in a graphics group in in MD for 15 years And I was involved with this which has its own things. I'm not going to talk about these things And then of course I discovered NPM. I met that discovered NPM before I discovered and Kabul and stack I couldn't find a stack logo stuff at the Kabul one. This is the Haskell package manager and I hadn't done any rust at this point but I was back at university and I was working in this domain and I was pretty Frustrated to be honest. It's you know, it's a bit like fitting a peg into a round hole, right? Apparently this is I actually use this one because apparently this was from a brexit website This is how some people perceive the UK and the European Union Think lots of expletives about now and swear words and what you yeah, anyway But this is a bit like C++ and Haskell, right? They're two things are counter to each other They don't many of the abstractions. I think in Haskell are really important. They're really interesting I don't you know, there's lots of extensions and GDATs if you're TDA GADT's which generalize abstract which are all great But they're a bit of an intellectual exercise in in showing how smart you are rather than whether they're actually that useful I know there's debates about putting them into rust and stuff, but whatever At the same time Haskell does provide really nice abstractions is very quick to work with just like people tell me about Python I'm not going to make any stances against it. I like the static type system versus like it, but so In 2015 I came to my first ever postum. I just back the friend of mine who's still here today A colleague is brought me then with some students and this is why I heard someone talking about rust I just overheard it in the Mozilla Devarum knew nothing about it And so this has been my journey since then and actually I really really think that I've been able to kind of fit that square peg into a round hole in some sense rust does that really well And that's why I really wanted to say thing and It's not the borough checker, you know, I'm to be honest, I'm not that interested in the borough checker I can see that it's great for security or that but to be honest if it didn't have it I'd still use rust. I don't really care I've spent three years actually learning how it all works and fighting it and reading papers to understand it So that was a pain, but I don't really care that much about it to be honest But I admit that I don't write software for many other people than myself So it probably doesn't matter that much to me, you know where I can see that it's great applications It's really useful the thing I would say and I wanted to make clear about that is that it's the thing that means I can't teach rust to first years I Don't in my university. I think it's just too much. I can't teach rust to first years because of It's a lot to take in Particularly if you haven't had a lot of computer science and programming experience and most of your experiences in Python or something I personally feel that it's too much, you know people they find it hard my students find it hard They don't even read the error messages, you know, you're just trying to get to look in if they start to read those complicated You know be right reading the messages template metaprogramming. They're not for the faint-hearted and you know It's not for the faint-hearted. I mean, it's just a comment and so I don't think I use the borough checker to anywhere the kind of Power that you could use it and it's just because it's not interested the and the abstractions I'm interested in are things like traits And the you know the parametric polymorphism that you can do over those traits and things like that and build it up the The kind of additions of kind of you know, obviously the range loops, but moving those more into having nice Where you can return things and get the nice build-ups of functional kind of style expression and things So that for me has been the bit that was missing from C++. They course they're putting it in Well, you know, it's like this again, isn't it? They're putting it into C++ some somehow And I find myself mighting more C code than I do C++ now because of of that problem, right? It's just so complicated and it doesn't really seem to fit that well Anyway, so that's what I was gonna do. That's an aside. I want to talk about it. So I ended up using Rust so back to the main part of the talk and so here is a diagram that comes from this guy called Thor Magnuson Who's a famous musicologist stroke computer scientist stroke musician who's a professor at Sussex University In the UK and he describes an instrument To look somewhat like this so you got this kind of the sound engines here the sound production Things you've got obviously got your synthesis whatever over here whatever is creating the sound and and you've got this gestural control Here, which is the interface that I showed you the bit where you're performing interaction with the instrument itself And obviously there's feedback. There's audio feedback coming from here and there's hopefully haptic feedback coming from the interface itself and And Really I've kind of completely separate. I don't really think they are separate I'm not trying to argue that separate, but it's much easier just like discretizing examples And it's if we separate those two things out and a lot of my work has been looking at that Divide in that line and of course the mapping then becomes a serious problem And it's not one that's that easily solved at this point So you'll see I have a whole and I'll talk about just at the end a whole rust based system Which is purely on this side of the world, which is just doing synthesis DSP as he integrates this language called Faust Which is a purely functional language for DSP just it allows you to Write DSP graphs in a functional expression. It's just a very mathematical. It's really nice we've got this little what of it generates C++ I've modified it so it generates rust now and It's just you know generates effectively a trait with some simple methods for initializing it And then a function to ask for n number of samples. It's not particularly difficult All the smarts is done in the Faust compiler, which I don't have anything to do with they're all based in France and super smart mathematicians Working at a conservatory, which is kind of slightly odd Anyway, but and a lot of this here has been building physical interfaces But using rust to interact with the physical world okay, and I'm not going to talk about it today, but we've also got an embedded platform Which is all hardware which is based on STM or ARM based microcontrollers mostly M4s and stuff that we use to do to connect to the physical world for actuation and sensing and I Did develop some of that initially start doing name rust I started working on that writing You know from bare machine writing it up, but I gotta be honest after about a year I kind of gave up and I've just write it all in C now just because of the code is really simple it's not very complicated and We use ST Gyps, and they provide reasonably good libraries And so there was just a lot of work and I know some brilliant there's some brilliant work going on And I guess when that's got to maturity That'll be great just I can just start off and I won't have to do all that work of getting the register's to map and all that So I you know I'm gonna put my hands up here I wish I hadn't had to go back, but I chose just for practicality reasons to go back to using C For for speed of use and I think that's really my message about My use of rust is that where it's been amazing is that you know You've got a credible ecosystem even though it has not that old and it works mostly across most platforms I've tried and you know can be packaged up. It's really great and that is a bit more tricky on the embedded world today Okay, so I probably won't talk too much about this But if you imagine that we've got an interface like this So I've got this whole system where you describe them in terms of Extended SVGs and you can you can draw them or you can write them in a little domain specific language Which I'll show you in a second and you can mark them up to say whether these are generating What kind of gesture you can do whether they're generating something called MIDI or OSC, which is open sound control Just a way of communicating standard communications and you take these SVGs and then you run it through this system Up here So this is it's a domain specific language It's actually an embedded domain specific language although you wouldn't know because I've embedded it in Haskell So this is the point where you can see the different things with rust coming in and I think using it where it makes best One of the things I found limiting at the moment Well, one of the things that Haskell is incredibly expressive with that you can embed a Domain specific language in Haskell and you and kind of rework the syntax You know, it's not racket. So you can't completely rewrite the syntax But using this thing called template Haskell which brings in template metaprogramming, but we won't go there But it's quite nice of being able to define your own it know own Syntax not completely you have to borrow some Haskell syntax and rust I've still found that because of it still looks see-ish in some ways You just still end up with a syntax that looks quite programming language Even though you can definitely embed it So I think that's a trade-off and that's why that top one up there is Haskell rather than rust and you know It's not performance is not critical at that top one. It's not doing anything. This one down here is doing path tracing and Rasterization it's because I'll show you the pipeline in a minute So performance is is important here and it wouldn't be so simple in Haskell I mean, it's still offline So we're not doing real-time rendering or anything like that And so you could probably get away writing in Haskell, but just didn't feel natural to me much better fit with Rust and there's also this amazing library called lion only online yet lion I think it is which is a 2d path rendering library, which I've modified slightly But mostly I've used it out the box. It's just a wonderful library developed by this guy Nicholas and And again depending where we're going This is all offline and generating this this generates things for it's called the rolly light pad Which I'll show a picture of a minute, which is a little embedded Controller which has a touch sensitive interface with 15 by 15 LEDs and you can upload little C programs onto it They're not see but they see like it's like a little embedded language that you can upload onto it So here we're actually generating code, you know, just text files that are then uploaded on to the thing So we're not doing any real-time processing or handling so speed again is not the critical thing Symbolic processing of what's outputted here, which is I'll show you that Jason found a second over here though This is the thing that connects to touch sensor that I was talking about This is real-time. So this is basically a driver, you know, it's driving It's talking with a very low-level USB driver and this is doing all the mapping from the rasterize it well I'll show you what I mean by rasterizing the second but effectively its rasterization and Here so performance is critical because you know We're actually this is running live with a musician playing it all the processing of when you touch something Mapping it to the semantic meaning of you know, are you in the right place on the sense? What does it mean? What's the action you want to perform? It's all happening in real time and effectively Hopefully you all know that USB USB Jitter is pretty awful on particularly on Windows and you know plus or minus three milliseconds is good and Even on the Mac, it's around two milliseconds, but it doesn't jitter which is quite nice And and then you've got the latency of evolved as well of taking that and generally on the Mac Actually, it's around recent upgrades around seven milliseconds on average without jitter, which is pretty good They say that you want to be between You know, but when you perform an action to hearing a sound you kind of want to be within ten milliseconds But if you think that it's taken seven milliseconds to guarantee that we're going to get thing You haven't got that much space to do any computation on the other side, you know, they're still doing synthesis I mean, obviously you're creating more than the same. So this is what performance was really important here Initially, I started in C++ and it was just you know C++ and you know I'm fortunate enough to be in a research group And so I get to explore things and play around with stuff and so and I've been to foster them And so Rust is where we are here. They work. It's worked brilliantly. I got you know, as I say, it's that whole stack Has just been great Okay, so I won't go into a lot of languages here But just to give you an idea these are the kind of things that people write in the language If we actually have a visual editor as well, so you can just use it in the web browser and draw, but I'm quite interested in what you can't do in a visual editor to particularly photograph Programmatic, you know expression. So for example, I've got this little example where you have an interface of Sliders that is drawn in a sine wave and that would be pretty hard to do I mean you could do it in something like illustrator in our little thing But you wouldn't be exactly following the sine wave. You'd probably be off and you know It would be a pain. Maybe you could be using bees flying or something but and so we have a little language that you can write this is like a stop button and You know, it's got size as a square one for this case And you can have something like a pad and then you can start talking about the kind of address this so this is the An OSC address so this is effectively the message. I'm doing right for time on it. Yeah Okay, so and I'm not going to go into a lot of details about this But you can compose them together and build up more complex interfaces. I mean, it's not rocket science. You know, it's the standard stuff this does, you know Use some stuff in Haskell like dependent types and stuff to make this all kind of fall out and it uses this Special extended use dependent type we can overload the syntax which gives us these special operators which allow us to have Arguments that aren't ordered, you know, you can actually write these arguments in any way And so there's it. I mean, it's more of a if you're interested in type systems. It's quite fun But it doesn't really matter to the user. You can build up interfaces and there's a little Domain specific controllers. What's interesting here is that we have a representation. These all just map To standard SPG types, right? You know, that's a really nice thing about SPGs is that of course They're just XML and if you extend it Illustrator will still display it or your web browser was today and it will just ignore any metadata that it doesn't care about I mean, it's great, you know, and obviously nothing to do with me. People are doing this for years It's just really handy particularly because this is kind of a graphical thing that users can see their interfaces and things like that so it works quite well and Obviously we have things like this What I didn't realize when I started off doing it, I should mention that I mean, this is nothing to do with Rust or anything was that initially I just used these three standard SPG types which aren't using Paths right they're not doing beast blinds or anything like that or any kind of thing but it and it's really easy to map if you imagine well, particularly if you know anything about rasterization and Desolation once you take this shape you try and glass it. I'll show it looks like a minute You can then rasterize it into effectively an array and you just get a bitmap of where people touch Right because it gives you a kind of mapping from a position on the on the screen to a point in an array And once you've got a point in the ray course, you've got a pointer to a class and you know or whatever a Data type and then you can perform an action and you get effectively Order one look up, you know for your touch, which is really important, but it turns out That actually because we're doing tessellation, which I'll show you what that means people We don't know what that means a minute is that you can take arbitrary paths and so people could start to draw weird You know, there's still vector drawings, so they're not completely weird shapes for their interfaces That all works brilliantly the course the difficult part is then to recognize gestures because you know if you've got some weird curve like this How do they want to interact with it? Is it following the path or do they want to interact it with you know multiple fingers things like that? And so I still haven't solved that problem yet I'm working on that and I I've kind of got a bit of a mapping, but it's still something to work out So this is the SVG that's created, you know, we won't worry about that So this is where rust really plays its key role in this pipeline is that so I don't know how many people are familiar with the graphics pipeline. Yeah. Oh, so quite few is great So hopefully you'll realize that this is effectively a graphics pipeline, you know my background's in graphics And so I do this so effectively we take these shapes up here is And what you do if you're rendering a 3d game or a 2d game It's particularly if you're going to run it on the GPU is you take some primitives some shapes up there and they're Normally, they'll be effectively they'll be described in terms of Triangles so this phase would have already happened you'd have a mesh and you just you just render that to the GPU And if you haven't you'll do this phase. So this phase is basically shape tessellation, which basically takes an arbitrary shape I mean it can't be completely arbitrary But as long as it's described mathematically either as a B spline or as a standard geometric shape like that and you divide it into triangles Okay, so modern GPUs of course support this actually in hardware as well So you can do it normally you'd send it a triangle and then maybe you divide that into more triangles So you can increase the resolution without having to send too many triangles in the first place, but we're not interested in that What we really do is we have this shape assembly phase. We label it with a number Here because we want to have a unique ID when it comes into the system itself We tessellate it and this is where lion comes in. So I'll just give you the little thing here He's done all the work hard work for this. There's some really smart work This is all the stuff they're doing for the web GPU stuff and things like that and relate to you know in the fire I don't know if he works for Mozilla, but quite a few of the Mozilla team are working on this and related things so we tessellate we generate basically a set of triangles and Then you have this phase of rasterization So this is the bit in the GPU where they're actually generating the pixels, you know They've gone from these geometric shapes infectively a triangle and they generate a bunch of colors that sit on the thing The observation here is that we don't really care about the colors because all we want to do is generate a lookup table Front of thing. So we just write we just write out our number that we generated in that shape Assembly and I'm using shape assembly because they call it primitive assembly in the Graphics pipeline. So it's just a bit of a play on that. It's just really assigning a unique number Okay, and so you can see that we've just generated this lookup table and clearly this will be the size of our Device that we want to target whatever that might be. Okay, so I'll move on quickly This is what you might look like when it's tessellating obviously tessellating squares and rectangles or rectangles and triangles with the trivial it only becomes complicated when you're doing Non square shapes or rectangle shapes actually circles pretty straightforward It only really becomes complicated if you have a squiggly thing like that because you might have a you want to draw a line And it will go outside of the thing and so you have to start chopping that up and that that's where it becomes complicated That's where a few bugs have come up in line where they miss a point and you end up with a fat line rather than the thin line things like that, but anyway, that's most of that's been resolved now and This is literally what happens, you know with your thing and if we are rasterizing it This is what you look like, you know, you've gone When you think about the size of the sensual so this is you know looks like an 8-bit retro image You know, but it's it's like that, okay? All right So then finally we get to the end bit. We're just really That would have been a nice bit of Jason which would have shown you the Wow I have to fix that another time up here. You can see there's the buffer up here So this is the buffer which is that is the one-to-one mapping with the is the array basically the maps That's been rasterized for all possible shapes. And so all you'll see is a 250 by 180 array of Shape ID numbers so you know if it was a circle it would have five in it if it was going round and things like that so There is one drawback with this whole system and which I will mention and Is that hopefully you can see here? I'm not going to have aliased all these points Okay, because this is nice the alias here And so you kind of get this when you look at it from far away it will look like it's nice and round I have to make a decide whether this is the in point and this is the out and so it's potential if you're doing very Fine grained interfaces. You might get behaviors that you're not expecting So and I don't really have a good solution for that moment Anyway, I'll move on quickly because I haven't got that long. This is what these two interfaces look like This is the really light path this little company from London who make these Overpriced box so finally I'm just going to finish how long I've got left about five minutes. Yeah, cool So this is a little application. I've developed it's completely written in rust about 25,000 lines of rust it's a hundred percent rust and It doesn't build anymore because I upgraded the compiler but anyway, but it's actually but it's really nice So it uses that line so it's all 2d based rendering path rendering to generate all the graphics Step sequencers it uses the simple threading library, you know the and cues and everything and but and it runs Really really well the performance has been amazing on my Mac and something really great Because you know it was a question when we started this project wherever rust was quite ready for doing real-time audio and I know a few people from Ableton have been exploring it They're a big audio company, but it's just been great and it's been a real pleasure to write it and and Everything and what's been really nice, and I'll just finish up on this Let's just group as I said in in France from Paris in France who have developed this functional programming for DSP programming language for sound synthesis and audio processing and It's a really nice. It's very mathematical So if you're interested if you're good at DSP and understand it It's brilliant place to start because you're almost writing the DSP equations and stuff and things you know and has delay lines and all that sort of stuff and as I said that they You write something that looks a bit like this and so very simple code if the stupid thing wasn't wrong you can see that this is a simple little oscillator you can set the frequency and The only thing that I don't like about it is that they've kind of assumed that you have this little kind of web Base interface or little GUIs that they generate for interacting with it And so that's okay, and there's ways of linking that with C++ software or whatever Which if I went to the next slide and it was working well, you'd be able to see but C++ code But I don't know why it's done that sorry about that But you can see that if I can show yeah up here So this one here they generates these codes and that comes from the fact of course I just want to go back one slide that they have this H slider up here Which describes a kind of an interaction with the outside world, you know it's a side effect effectively, but it's all functional and So they need a way of connecting that and to do that they introduce these variables when they generate the C++ code or when I generate the rust code it generates something as well and Then they have a way that you have the head like this UI that you pass in which is the thing that's controlling it Which actually has been a real pain for us because when you build up an audio application That's not really how you want to pass around your dead You really want to separate it as you know the UI wants to run in a completely separate thread There isn't real-time dependent whereas the audio thread is got to be you know real-time doesn't want to be and so he can miss control signals Because it might not look at the queue because of time things and everything whereas this is kind of all quite it fixed in So what I ended up having to do was Sorry about that it's really annoying, but basically if I slide over to here You'll see that there's these functions here. So when I generate the The rust code you basically describe a JSON file which describes the The site the variables that you want to change and then it generates the rust code looking at that and will generate just a bunch of Setters and getters. I mean fairly straightforward. There's nothing smart about it. It's really very simple code It just has a simple Strap plus a simple implementation the implementation is initialize the audio process that you know the seek the synthesizer or Whatever variables need to be initialized and then it has a compute function that you call For how many frames you want so maybe you've got a 64 frames that you need and you just call it And it's a simple loop the really nice thing about the fast compiler It's because it's the way it compiles is that there's only a single outer loop There's no nested even though you might write it with recursion inside it It can propagate it all the way to the top so you end up with a single loop and so it's incredibly performant and of course, you know LLVM can rip through it and optimize it really well There's no aliasing. There's nothing so it just the code it generates is it's great and so it's really nice interaction and It's benefited that rust was written in LLVM because I now just generate from LLVM from because the files goes to LLVM as well so that's been really great and You know another great thing about rust is that it chose to use LLVM which I love you know GCC is the haunt of my life and Anyway, so that's it really just wanted to say and I just yeah, I suppose the punchline. I already gave it to you was that It was really brilliant. I just the package. That's the thing for me about it. Everything else. I love him rust I've used it all the time, but cargo is just it's just been like the best thing You know an MPM, you know, well JavaScript. Hey, I still quite like JavaScript. I can't deny it. Anyway, that's it Okay, yes So the question is in rust there are Some other work going on we're particularly in video and audio and image based Control systems is there an overlap with this work? I guess there probably is I must admit I'm not that familiar with them and so but I don't see any reason why there wouldn't be I don't I There's some novel work in this but the rust part of it probably isn't that novel other than I think it's a great application For rust, you know, we wanted performance, but I really care about having some of those high-level Extractions and that for me is the main that's why I would encourage people to use rust again. I know a lot of People and I hear people at my university Particularly people in the security groups talking about rust because Microsoft now talking about it, you know, and I absolutely that's great I completely see that it's just I fall asleep when they start talking about it So it's just not me, but that's just a personal thing and not a criticism of rust at all when the borracher I mean, it's amazing. It's just not my interest. So but yeah, okay Yeah, if my cold that's you feel a bit better now, I think that's got a sweat on so it's good, but yes. Yeah, definitely


Sorry for the slight delay. So let me give you a heads up. This is not going to be a conventional tech talk. This is my first tech talk, not yet. All right, please, please come in the front in that case I'm running out of time. There's any kind of feedback or comment, please find me on Facebook, Twitter or Bharat matrimony or whatever. Yeah, that's cool. So yes, I am Rohit Nayar. Hello. Yeah, so I'll be speaking about Suru today. And let's get on with the problem statement. So let's think of this. We go into a grocery store. And what if the grocery store has this rule wherein you cannot buy groceries without having an assistant with you. You always want an assistant with you to assist you to buy groceries. You cannot do it yourself. So what happens in the process?  He doesn't, he's not going to leave you or he's not going to give you all responsibilities of buying your own stuff.  OK, what happens after that? He gives attention to the customer. What happens next? You are left stranded and you're waiting for another executive to come by and help you buy groceries. So you are dependent on the executive. And in the process, as a consequence, your dinner gets delayed. So what could you do possibly? That's a possibility, but we're not going to go into that. So yes, get rid of the executive and do it yourself. All right. So yes, what I want to stress right now is about self-service. So developers always need resources. And resources could mean you or, you know, resources in terms of your physical boxes or your time and stuff like that. So what generally happens is you are always, as a sysadmin, you are always interrupted with a lot of attractive work. And then the developer is now waiting for some other executive or a sysadmin, basically, to come and help him out. So the developer keeps waiting and then keeps waiting. So yeah, so what if the developer had the ability to gather resources himself without really waiting for the sysadmin to, you know, get him acclimatized with everything. Yes, we would never doubt our attractiveness as something else. And things would definitely be on schedule. So yes, we have a similar scenario, basically. So we have an array of web servers. And we always, our developers are always trying new stuff because of which we always have to be with them, assisting, deploying code, and we have to stick around with them. So as to check if everything is OK. Because most of our testing runs on production servers. And so we have to be there, along with them, to see if things are properly scheduled and that there are no mess ups or anything of that sort. So yes, that's what we deploy, we test, and we repeat the procedure every single time. So we realize that when it comes to testing and staging, we need not invest a lot of time along with the developers. We could just give them the handle and let them navigate throughout. And we could just be there, do something else, and invest our time in something more productive. So we thought, why not you, Suru, an application which helps them take care of their own resources. Let's them play around with the stuff they want. So yeah, so what do we primarily manage? We manage web servers, CDNs, Redis cache servers, and stuff like that. And we basically scale horizontally. Our web servers are always scaled horizontally. So I'll let you know how Suru helps us manage a staging environment. So yes, these are just basic stuff about Suru. So Suru was developed by Globo. It's an open source pass. And we can write apps in any programming language of our choice. And we deploy it using Git. So yes, this is the architecture. I think it's not very clear. So let me tell you how it works. So yeah, is it visible to all of you guys?  OK, that's fine. I'll just walk you through it. So the laptop you see here right now is your application developer. So the application developer basically contacts the Suru server. He makes a core deployment. Basically, he pushes his core through Git. And the Suru API, or the Suru server rather, will now contact Docker. All right, will now contact Docker. And Docker will now spawn as many instances as the application developer requires. So basically, so here we are using Docker as a provisioner. So basically, you can use Suru with Juju, which manages your AWS instances as a provisioner, and Docker also. But we went with Docker because we did not want to use AWS in the first place. Yeah, again, on the top right beneath the cloud, we could see an HTTP router. So that's a Hippachi router. It's basically a load balancer, software load balancer. It's a software load balancer. So all the requests will now hit the HTTP router, and it will be load balanced among all your Docker nodes. So that is how it works. And we also have a Gandalf server which manages all your Git repos. OK, is everybody well-versed with Docker? OK, so let me just explain what Docker is all about. So Docker is basically, you can consider it as a wrapper around LXCs. And LXC is basically a kernel feature available from 2.6.24. So yeah, it doesn't have a hypervisor. All right, it has C groups. So a hypervisor basically resides between your kernel and your hardware. We don't have a hypervisor. We have C groups. It's a built-in functionality. So yes, silly stuff. That's OK. Yeah, that's us. And so we are also to create a kernel. We are also to create a container which shares certain characteristics which are provided by the kernel. So yeah, that's what. There's no hypervisor, and it makes it really fast. Yeah, that's about it. That's Docker in a nutshell. Just in case you guys want to know what Docker is, please Google for Mithun Brain Transplant, and you'll probably understand that. Yeah, so the required entities here are Hippache that I mentioned. It's an HTTP software load balancer. Gandalf for managing JTrippers. Redis is used by Hippache for mapping. The thing is the Docker nodes that come up, they are all ephemeral in nature. So basically they have different IPs that get spawned as soon as you spawn a new Docker instance. It's allotted a new IP. So there has to be some kind of mapping, and Redis basically provides it. MongoDB basically it's used by Suru for a normal bookmarking. Demo I, let me see if I have. OK, just hold on. So we are just trying to deploy a basic WordPress application via Suru. And as you can see on the right hand side, this is created by my friend Kalyan. He's, you can see Facebook and other tabs open. That's a little window. Yeah, so we are going to try and create a simple Suru application. So yeah, we are creating a WordPress application. So we created an application. Now we are unzipping everything into a root directory, initializing the Git repo. And after this is done, we will push all of this to the Suru server. OK, happens to everyone. Go on. So as soon as you commit and push, Suru will contact Docker. Docker will start spawning instances. Yeah, this is when the magic starts. Exactly, so Suru has now contacted Docker. Docker has created some instances. Now we list applications. And as you can see, there's this one application that has been set. And now we see there is an error because it doesn't have a MySQL application running. So we will bind the Suru, the Docker instances, to a MySQL application. So we require a requirements.apt file wherein we mention all that is required. And then we push it. Now what happens is it will delete the old Docker instances and respond the new ones. Yes, so now we will try and export the database settings to the Docker web servers. And this is how we do it. So basically we are binding a service to the WordPress application. This is all done via Docker instances. So as we can see, our site is up now. And these are your environment variables that get exported into the web servers. So there are times when we want to scale now, there is only one instance of Docker. So what we'll do is we'll add more instances to this setup. There's only one unit, and we are adding two more. Now we can see there are three Docker instances that are running. That's about it. And so there are other tools like Flynn, which is also does similar stuff. But then the difference between Flynn and Suru is the importance given to service, importance given to scaling services. Now we spoke to a couple of engineers at Globo, and they said that it's all about the design principle. They wanted to basically scale web apps and not services because we don't know how services can be scaled. They can easily either go horizontal or vertical. So the only way they could come up with adding services was binding proprietary services or stuff like that to Suru. So that's just the basic in Flynn and Suru. So yeah, the challenge is less documentation. Suru, the only place where you can find Suru documentation is on the website, nowhere else. There are two dependencies.  And it's only built for web apps. Now that's the way it could be considered as a merit also because we will only focus on scaling stuff horizontally. And yes, there are buggy scripts. With Git, there are a lot of pre-received hooks that has to go in, but they were all buggy. And my friend Kalyan, he actually fixed all of them before we could go on with this. Yes, so do we have any questions? Why do you think it is better than or what do you have? What's the opinion about heat and open stack? Basically heat, by heat you can decide your templates and put all this thing over open stack. So why do you think that this is not good or? I think this is easily manageable. It's very easily manageable.  I mean, you can just create an application, give it to them, and they'll manage it. You don't have to really interfere into anything. And we thought Docker was the way to go. Right now, how do you handle multiple servers in this case? So basically currently talking about one server, let's say I want to have 10,000 or hundreds of instances running, how do you handle that in this? As of now, we really didn't have a scenario wherein we had to look into 1,000 instances. But we can easily manage 20, 30. That works for us. OK, less on production, but. Yeah, it's a staging setup. Hey, hi. So you have done something like you committed the code and then it committed. So after that, a trigger has been done to Suru to start the Docker and then to initialize it. Yes. So how do you do that sort of a workflow with? Like, you have to have a predefined installation of Suru on that system. And any code post hook trigger, so you keep that Suru based configuration, something like that. What do you do that? Yeah, there are a lot of files basically. So you mentioned all of those where you want to deploy your code. All of that is mentioned in your Suru Conf. And you have a requirements.apt file. And there is a deploy script also. So once that is read, your deploy script will deploy. All of that is mentioned in your requirements.apt and just create everything. You really don't have to look into anything else. OK, does it work at a branch level? Or does it work at a multiple Git level? No, it doesn't, I guess. I'll have to check that because it's a post-receive hook. OK, so I mean like staging, development. So these are all different different branches, right? So then if I want to check it, then how Docker understands it and how Suru does it in different configurations? No, I didn't get the second part. So I was just asking like the Suru-based configuration on different branches. How exactly the Docker knows it, actually? I can take it, we'll take it. Yeah, I'll take it off, I'll take it off, I guess. Hello, am I audible at the back? So it's a Friday evening, you are sitting with your friends that you are having your favorite beverage, whatever you like, and suddenly this happens. A server goes down. And you were not even responsible for it. You should not even have received that alert, but you received it because a new guy misconfigured the notification system. And that ruins all your mood. And believe me, it has happened with every one of you, right? So as you make your next multi-million dollar company, you add more people, you add more hardware, you add more software, these problems will go from bad to worse. And at DirectEye, we have faced the same issue. We have over 1,000 servers, physical boxes, 100 VMs, distributed across 10 colocation centers, and over 25,000 and growing checks every day. You can imagine the amount of noise being created and the number of people being disturbed when they should not be. So this is what happens. This is the monitoring system. It explodes alerts. And there's one single guy, the L1 engineer, staring at the dashboard and thinking, oh my god, now what to do? Same like him. He's surrounded by alerts, and it's a flood. And there's a team lead on fire. And he doesn't know what to do next. He has no information. He has just a million of events running down to him. So my friend, Sathe, as we call him, has wrote this. We have made an app, an app suit, that meets the need of every guy in the chain, from an L1 engineer to the head of, say, the operations, to the Sysad guy, to the new joining, to the team lead, and the end customer, the customer to whom you're serving your services. So one app to rule them all. One app to find them. One app to bring them all in, and obscureness, combine them. So what do we have? We have events. We have alerts generated from monitoring systems. You have Nagio, Springdom, any kind of monitoring systems you might have. You might have hooks from LogStash, raising events when there are anomalies. You might have metric events. You just have events. That's it. And what do we want? We want SLA. We want to make sure that our product is meeting the uptime. We have promised our customer that we'll provide you 99.99% uptime. We are better than the other company. But how do we figure out at our own? We want statuses. We want status dashboards. We want progress reports on an incident. We want correlations. We want RCAs. We want a lot of things. And what do we have? We just have events, alerts that are generated from the system. Now coming back to the L1 engineer, what would have helped him if there's an alert that has occurred? If there was a knowledge base existing where he could just search Apache and, say, the server name with the tags. And he would have got a recent history saying, OK, this was an issue that occurred. Or, say, a new guy has joined in today. And 10 days ago, a similar issue has occurred. But he has no idea of what occurred 10 days ago. So if there would have been a system where he could have knowledge bases. Or there would be a way where he could escalate alerts automatically. He doesn't have to call up folks manually saying, hey, dude, it's 10 minutes. I'm unable to fix it. Can you look at it? Then even after half an hour, it's not fixed. Again, he has to call up someone else. Hey, can you look at it? It's not yet fixed. So for an L1 engineer, that's what he wants. For a team lead, he wants dashboards. He wants product health reports. He wants to see if the uptime numbers are met. He wants to see which is the most problematic endpoint. What are the frequently occurring outages? What are the server that are going down frequently? So he has his dashboards, where he can see trends. What are my S1 events? How many S1 incidents are there? Which is the most affected colo? Which is the most affected product? What is the frequency? At what time am I getting these incidents? And he can see the list of all server and server that are affected currently. Or he can see uptime numbers. Say you have 10 different endpoints for a particular product. You can see which is the most trouble-making endpoint. For an end customer, as a customer, you want to know the status of your services proactively, not reactively. It's always good to be proactive for a customer. So if he can know what your services are like now, are they up? Are they facing any issue? If they are facing any issue, what are the actions being taken upon? And what is the ETA and everything? That's when a customer will be happy. So we built this system called Slant. It's an app suit which has a various number of apps. It manages automatic escalations. It manages SLAs. It manages incidents. It manages your contacts. It manages your on-call. It manages your calendar. It manages your scheduled maintainances. And it manages your product definitions. It knows when which server belongs to which product. So for the new culture, we are changing the culture at our company. So now the L1 engineer, when he joins, the first time he logs in in the system, his contact is automatically created in the app. He just has to make sure he's in the correct groups. His on-call calendaring is correct. And that's it. All the escalations are set from previously. And he has a knowledge base of RCAs. When we have outages, we make post-mortem tickets, we make RCAs, we add tags. And he has a knowledge base of it. So whenever something goes wrong, he just searches in that and he finds the answer. For a team lead, not only for a team lead, other members in the team, there are dashboards. There are apps which shows what is the troubleshooting endpoints, what are the incidents, what are the current actions, who is working on what, who has acted, who has commented. You can just type out a comment and make it public so that the customer can see it. For a customer, he has everything proactively.  Why is my mail not going out? So this is what we have built. Yeah, so any questions?  If I go to the architecture of the application, it's a very simple architecture. What we have done over here is we have built a system in Ruby. You can make API calls from any of your monitoring systems. So we are so diagnostic. We don't care what your monitoring system is. Whether you are in Pingdom, whether you are using Nagyos, or any of the monitoring systems, we don't care. We just deploy a monitoring system once and configure it to send events to our slant. You can make API calls, or you can send emails. You can write wrappers over it, and you can process those events. Once those events are received, we have a rules processing engine where you can write rules. This is simple Ruby code where you can write if condition, saying that if this is a server and this is a service, make it an S1 or make it S2. That's a rules engine, and once that's a processed event is formed, S1 events are taken into computation for outages and SLAs and notifications according to the escalation policies. So that's a fair, rough architecture about slant. How do we plug in my custom events into it? Let's say a simple cron is running, looking at some basic parameters, CPU, iOS, et cetera. And it decides that CPU is above 40% and there's a warning, CPU is above 60% and there's a critical. So it's not a Nagio or a Sensor, well, it stands for... Yes, so you can basically make API calls to the system, right? So you can just create a hash of... So it's a REST API? Yeah, REST API, yeah. So you just mentioned that your tool calls APIs from multiple monitoring tool, and then it kind of helps your bunch of group of people here. I'm just curious to know how many hours is your team spending to constantly build these REST APIs because let's say I build a service or an app, right? When I build my app, I'm gonna have a bunch of templates because I wanna know certain conditions about my app to be monitored, right? As a developer, like it's in my org, a developer builds their own template for monitoring. Then you have this tool which has to probably monitor the same thing, right? So you make an API call to us. So... Whenever there's a... If you are making... Your tool basically, I'm sorry, the tool basically is the front end for your knock person, right? I'm looking at your L1 person, right? So the API calls, there are all these different monitoring tools. Let's say there's an OS or whatever, right? There's gonna be templates lying in there. How are you connecting it? You're constantly spending cycles... No, so what we have done is... Not really... They don't need to be aware of your systems prematurely. As soon as it comes, it would process it into the common category, into the common data pool. And if you want, you can add your own service configurations where you would say, if I get an event with this particular condition that's met, process it in certain ways, have certain associations to it. So you are exposing... We are exposing the API, yeah. It's completely on a simple web server. I mean, it's all on the UI. You don't need to do anything. Okay. Will it send before the... Happen that event or after the event? Sorry? You're saying it is a handling on the events, so how we configure that event? Will it send that the event is happening before that or after that event? We suppose... So when and us, yeah? So suppose my API server or whatever server is hitting around a thousand requests or more than request, if I... Will it send before the... Before hitting that request? No, it's when the server sends an event to us. Yeah. So when the server sends an event, according to your escalation policy and your settings say email or mobile phone, whatever, you get an SMS or an email. So that will take care of the logic or whatever you're providing that will take care of. Yes, according to the policy I've set over there.