OK, and we're back. So you probably all know Rob. If nothing else, you ask him to clear the storage on Hydra. It's not the only thing he does. And actually, he'll tell us everything about what he does. Well, thank you. I hope you guys can hear me. I'll try to keep you entertained for about half an hour or 40 minutes until we go for lunch. And then, of course, that's the best part of the day. So I'm Rob from Mass. I work for a company called Logic Blocks. And you might have heard the name around on the IRC channel or start on the wiki. So this is my contact information. If you ever want to talk to me, I'm most of the time on IRC on most late times as well. So we're there to help.  So I would like to tell you a bit about who I am and what I did with Knicks. So I kind of looked back at my commits. And I found the first commit in 2004 when I added Octave. That was because I used to be a student at the University of Utrecht, which is also the place where Knicks was developed originally. So Ilke Dossai was one of my fellow students. I mean, he was a bit older, so he finished before me. But after my studies, I went to work there. And I was working on a project called Certigo XT, which made like a language to build compilers. And that consisted of like 20 auto comp packages. And of course, to build that, that's terrible. So what do we do is we actually started using the work that Ilke was working on. And Knicks actually built and actually sent to the users this project. So that was nice. Then we actually moved to the TU Delft, where I actually worked on HYDRA and a bit on KnicksOps. And I've also contributed to Knicks. Like Rock explained, I also do a lot of infrastructure support for, let's say, the infrastructure that we have. So HYDRA, the website. I do that together with Ilke Dostra. Sometimes Sander is helping. And there are KnicksOS community members who help with that as well, but basically more on the web drive, not on the back end of the servers. Also, this year, we started the KnicksOS foundation, which meant the support to give a kind of a vehicle for people to give us donations so that we can keep all this infrastructure running. So I would like to encourage you to give us all your money so we can do all kinds of awesome stuff. So as I explained, I work for LogicBlocks. We develop a state-of-the-art database system that is used to develop applications for many top retailers and banks. So that's the marketing statement. I'm not going to tell you too much about LogicBlocks because it's actually not too interesting for you guys because it's not necessarily like an open product that you can actually use. You have to work together with us. But what is important is that we have a few KnicksOS. So first of all, of course, me. I'm doing the presentation. But also Ilke Dostra, I don't know where he is. There, everybody knows Ilke. Our CTO is Martin Braveboer. You probably haven't seen many commits of him lately because I think his last commit was in 2008. But he was actually using Knicks very early on as well. At some point, Shay Levy is also a good community member. He works for us and we sometimes hire contractors to do work for us that we cannot do ourselves because of capacity reasons. So Yevgeny has worked for us and many will know him as freedom on IRC. So what I usually do when I have a talk about Knicks, I would like to remind myself why I use Knicks because I think it's very important to kind of reflect over why you do things. So my first point is always, Knicks protects me against me.  So I'm very good at making bugs. So I like the feature that I can just roll back because that kind of gives a safety net that we actually need. Well, that I actually need. You, of course, are like perfect programmers and never make bugs. But also, I'm very forgetful. So unless I make something that's completely automated, I forget how to do things. And what Knicks does, it actually gives me a vehicle to do things consistently multiple times and on different machines. And that takes away a lot of the pain that I have with software development. And I'm sure you guys have that as well. What it also does is it exposes the things I forget like implicit dependencies. I mean, we all know when you start a project that's existing, you make a Knicks expression for it. Oh, man, there are so many implicit dependencies and so many assumptions being made. So Knicks exposes what I forget to actually declare. And therefore, it helps, let's say, the previous things that I wrote. So another thing is that over the course of time, we actually have developed all kinds of tools like Knicks, KnicksOS, Hydra, and KnicksOps. And now we have just one language to rule them all, like it says on the Wi-Fi password. So that's what I like. And I hope you guys like it as well. So my presentation is basically a call out to make sure that you guys start Knicksifying companies. So this will be a crash course in how to actually do this. So there's a free-step plan. You can find a nice company, which I did with Logic Blocks. I'm sure you guys have nice companies as well. You apply Knicks wherever you can, because people and companies will have problems that need to be solved. And they can typically be solved very nicely with Knicks, so just do it. And so for you, we have profit or salary or whatever kind of your system is. So that's basically my free-step plan. So when you build software, you build software.   Sunder will probably say, no, you have to have five, because this Knicks is also part of that. But in my case, we don't use this Knicks yet. So it's the big four. So first of all, Knicks, which covers this whole pipeline. Of course, Hydro, we use for building and testing. And KnicksOps and Knicks West actually get it deployed on systems. So you get to a company that exists already, I think in the case of Logitech, it existed for 10 years already. So there's a lot of software there. And what do you get? Bash scripts, something called Jenkins. I don't know if you've worked with it. It's terrible if they maintain machines, all kinds of stuff. And of course, people want to use enterprise-y operating systems like CentOS or Red Hat Linux, which I get the shivers from. So of course, I want to change everything, but you can't change everything at the same time. So what you just as a first step do, because you can't immediately deploy it with Knicks or KnicksOps, you basically make sure that you do your builds first step in the whole pipeline. But man, closed-source software.  So they actually make sure they keep to certain build systems, standard build systems. So then relatively easy to build open-source software, even though, of course, there are shitty parts sometimes as well. But in closed-source software, people use build scripts, hard-coded locations, binary files. Who knows where they come from? Like library SO files. Someone once built them, put them in a repository, and you actually have to use them. Also like huge builds, a typical package in Knicks packages, probably like a few megabytes. Man, I think my first build of logic blocks was about 2.5 gigabytes. So the case also comes with troubles. But also, let's just download something from the network. It's a typical problem we have when we package in Knicks, right? Oh, man, or even worse, not just using one language. Use six languages. So it's all kind of nice. If you have one MPM package, then that's it. But oh, man, imagine using Java Scala, C++, whatever language they can think of. That was best to suit their purpose at the same time. So OK, you've Knicksified your build. And I must say it was very ugly at the beginning. It was a very big script hacking around all these kinds of hard-coded locations. But of course, you need to run it continuously. So that's why we have Hydra. So you'll know this screen, at least I hope you do, because it's kind of the main server in the whole Knicks sphere. So why I like Hydra? Again, most of the advantages here actually caused by Knicks. So it's basically a generalized Knicks run, in my opinion. Kind of like a Knicks scheduler, in a sense. And all bills are consolidated in one language, Knicks. And in my opinion, that's really nice, because I hate Bash. And not because you cannot do good things in Bash, but it seems to be that all programmers always forget everything they learned about programming in Bash. I don't know why it is. And probably I do the same. So it's not like I'm blaming others. I'm just as bad. And I need some guidance. And Knicks gives me this guidance, basically. So less maintenance, in a sense that you don't have to maintain computers. Like we had this Jenkins machine, that one version had to have GCC42 and one GCC, some other version. Also, I want to be able to reproduce the build, have caching of builds, which is very important when you want to compose builds. And I want to integrate it with our deployment tools. So to you who have never installed Hydra, we might call some things.  Or what is a job? Or what is a build? So in Hydra, you can actually group builds together in projects. So for example, at Logitech Blocks, we have for each of our clients, we have a project for our platform. And then we have a concept of job sets. It's basically you could see it maybe as you want to build a branch of your product or a pull request. And then you create a job set. And the result of a job set is actually a set of builds that can be multiple things. So the features that are important for us and that we use a lot is the fact that it's so easy in Hydra to create clones of job sets and build branches. But also, the ability to compose multiple builds is one logical build. So let's see. In our Hydra, we have 19 projects and about 205 active job sets. So that's actually a lot. So what do we build? We build our platform. So I think that we build a database platform. And this is our main builds for things. So I had to kind of rotate it to make sure that it is fitted on the screen so if everybody can move over. And then you can see that we have a lot of builds. And that's not because our platform might be just 50 of them, which is probably like a third of it. But we have a lot of things that we also trigger, things like benchmarks or tests or things related to deployment or you want to try to build client application. So we have a few different types of builds. And that's for a reason that our builds are getting so huge in the time of build time that they actually consume that we cannot commit, basically build on every commit. But we actually do that. But we just use a smaller subset of builds. So for example, if somebody commits something in our platform, that triggers about 90 builds, which is kind of like three hours worth of builds if you would do it in sequence. We also have nightly integration builds, where we also build our client applications with it to validate if, let's say, a change in the platform breaks something in those applications. And then it gets a bit harder, because each commit will actually cost 300 builds and 120 hours of builds worth. So let's say if we do that on every commit, we would need a lot of build machines. We have a lot of them, but not that many. We're not Google or anything. So we're building client applications. And how does that typically look like? So typically, they have two inputs. It's basically the platform release, which is an argument to specify which release we actually want to use, the source code of the implementation of the application. And it typically looks like a function call that calls like a generic app job set, the function that builds a set of jobs based on how to build the application. So there's a built LB config. We have a kind of like a standardized build system for LB applications that we can basically pass in logic blocks, which is the database, and blocks for which is our application server. So in Hydro, that will look like this. You should look familiar for you guys. There are like four builds that come out of this function. One is the actual build of the application, including the running of the test. But also, you can actually download a tarbled version of what comes out of it so that the developers can just download it instead of, let's say, building it for a few hours on their own machines. And we do some basic testing that is generic for each application. And applications can actually add their own tests and that will show up as extra job sets. So another thing that's important is the Garon closure. So probably not everybody knows, but Nixops was once called Garon when we wrote our first paper about it. But nobody liked that name, so we chose the boring Nixops name. So Ollie Charles, he explained how they are using channels to do deployments. We actually do something similar. I don't make it a channel, but we basically create a composite build of all the dependencies needed for deployments. So what you see here is a store path, which is the Garon closure, which contains everything we need for deployments. So that's the LB application, also software dependencies, like the platform or the application server, the operating system, Nix packages, some generic system configuration files, how to install LB monitoring stuff, and the machine definitions, which are part of the source code of the application. So basically, if we want to deploy, we can just give it this path, put the Nix path correctly, and it'll find everything you need for the deployment. And basically, we'll just run. So I thought it was very interesting that Ollie Charles actually did something similar at Finder. It's good to know that people have similar ideas, basically. So I explained that. Of course, I started building stuff with Nix, and we couldn't use it for the whole thing, like the deployments. And of course, they want to use CentOS, hate CentOS. But the nice thing about CentOS is that one thing that I hate more about CentOS than CentOS is the fact that people build CentOS images manually. Because building things manually, of course, that's the evil. So what they actually did is I actually automated the CentOS image generation so that at least we had a declarative specification what is in our CentOS image instead of somebody actually making the image, changing some stuff, then cleaning out the SSH demon public key. I know all the kinds of things that it's needed to actually fix all kinds of impure stuff when you create something like that. And one thing that's really important about this is that Nix packages has a lot of weird functions in them. So one of them is the run in Linux VM, where you can basically give it a Nix build and it will just run in a VM magically. So I think it starts up some key move stuff. Don't really know the specifics. But it just works. So if you ever need a build that actually needs root, for example, to create a file system or something like that, you can actually just use this function. And it's awesome, in my opinion. So it's also used for easy to do and the images for Google Cloud and Versalbox. So there's a reference there. So make sure to check it out, because I think this is one of these things that you don't really necessarily think of that exists, but it's undocumented, but still super awesome. So we actually use that also to test our platform. So we actually provide a binary release to our clients and to our developers. Of course, they're using Ubuntu and Fedora. But I don't want to have Fedora and Ubuntu machines in my build form, because I have to manage them and install them, and maybe I have to use Puppet or something like that. So we just do the Nix, because in Nix, you can actually build these kind of things declaratively. So here you see an example of, well, you see the basic dependencies that LBS. So we need to have Bash installs, we need to have Java, Python, and some proc.ps. That's probably for the PS tool or something like that. And we have a function called test download package, which basically does the equivalent of downloading the package, unpacking it, running some basic tests on it. And this is all done inside a Nix build. So it integrates into Hydra like automatically. So it's really nice. I mean, there's support for CentOS, Fedora, Ubuntu, Debian, whatever. Mm-hmm. Oh, really? I haven't used that yet. I always say we need to build everything. So also we build our documentation automatically, because doing things manually is super evil, and things will get not synchronized. So our documentation is typically built continuously, and every night there's actually a deployment going to our website to make sure that our clients have always up-to-date documentation. So one thing that we have been using Hydra a lot for, and mostly since this year, because we started a designated benchmark team, which is very important for database, because you kind of want to know when you introduce a performance regression. So we introduced a lot of benchmarking in our builds. The benchmarking is kind of tricky when you have a build farm, because you kind of want to run in a consistent environment. So what you cannot do is to build at the same time, because it will actually influence things, because there will be IO use by build processes, or CPU use. So you need to make sure that these kind of builds can run in the same consistent way. So how we have solved that is basically having a subset of our build farm in Hydra that only allows one build at a time. But the Hydra scheduler actually was kind of dumb. I mean, I also explained that yesterday. And that caused a lot of contention on the build farm, where the normal builds actually wouldn't go through, because all the benchmark builds were waiting for a build slot. So that's also one of the reasons why we improved our scheduler in the Hydra Q runner, so that we can actually run these mass scale benchmarks on there as well. So when you do benchmarking, there are a few things that you actually want to take into consideration. So the data set that goes in, so you can have data sets from 1 gigabyte, 10 gigabytes, 100 gigabytes, but also things like the software, which platform, which version, which type of benchmark you actually want to run, but also what type of CPU you want to run it on, how much memory, how much storage, what type of storage. And these kind of combinations are really nice to program in Nix, because we have these language and the tools to actually describe all of these things. So we have NixOS for the system, and Nix for the versions. Data set that's basically just defined as a list. But also NixOps to actually deploy new instances on EC2 for machines that we could normally not really buy ourselves. So we have benchmarks that go against a terabyte of data, but we don't have a built machine to actually fit that on. So what Ilco developed was the Hydra EC2 Provisioner, where given a certain type of feature that we need, like memory or storage, we actually deploy a new machine that actually will be built. And it's really nice, because now we can actually run benchmarks on really big job sets, which you couldn't do before. So I started at Logic Box in 2011, and we started out with three Linux machines. At some point, we got a Mac. But I was located in Utrecht, and our office is in Atlanta.   So we actually needed IT people to actually respond quickly. And of course, like any company, our IT department has a lot of other stuff to do, so the response time was pretty slow. So after about two years, we actually decided to run all our build farm outside of our on-premise network. And we actually moved to Hetschner. And I mean, Aslich, he made the Hetschner back end. And I'm so thankful for him that he did, because it's awesome. And it has saved us so much work. And nowadays, we have 21 machines at Hetschner, but also two Mac OSX machines. Not at Hetschner, because they don't do that. But there's an American company called MacStadium, where you can basically hire, or it's not hire, rent the Mac hardware. And basically, you pay some money for the hosting. So we have also kind of, it's not cloud, but they deliver usually within two hours if you need extra capacity. So it's actually pretty awesome. And it's much cheaper than ease too, because of course, we could actually run 24.7 on the EC2. But then our build would go skyrocket, and it's already pretty high. So that's probably not a good idea. Kind of to show you how our, so this is the beginning when I joined Logic Blocks. This is a graph that shows the number of builds. So our hydra is called Bob the Builder. And so if you have children, you know what it's about. And you see there's a big steep, like in the first year, we went from zero to, well, I don't know, 30,000 a month. Oh yeah, so it's per month. It's good to know. So currently, we do about 1,200 builds a day with hydra. But more importantly, I told you that we do benchmark builds a lot more since this year. You can see the spike there, because these benchmarks typically run really long, especially when you have these longer jobs. So I think it tripled the time that we spent in the number of build hours, so if we would have run it sequentially. So OK, now I've got all the builds. The builds farm, we're testing everything. So it's really nice. Now we need to start deploying. So for you guys who might not know exactly what Nixops is, it's basically a tool to deploy a set of NixOS machines. And that can be to anything. It can be to the cloud, or it can be to on-premise machines. It's based on Nix. It uses the Nix language to actually describe it, actually uses the NixOS module system as well. So I think it has an expressive configuration language, which is nice. It's also nice and composable. And it has kind of a separation, at least it allows you to have a separation of logical and physical aspects of the deployment. So what that means is that, in one case, we actually want to describe what is deployed to a system. I want to have a running LB instance, or like a database running, or I want to have a web server running. And on the other side, you actually want to describe where you want to deploy it to. So that's simply I want to deploy it to EC2 on this instance size, or I want to deploy it to virtual blocks. So that's useful for a lot of things like development, but also for, let's say, if a client needs to go to a different cloud provider, you actually don't want to do too much of changes. So that's how it looks like in a picture. So the typical example of NixOps looks like this. So you have this functional, so I want to have the NixOS home page. And I want to deploy it to EC2. Well, it turns out when you do a lot of deployments, what will happen is people will start copying this part, make a copy of it, change it. And of course, code cloning results in all kinds of inconsistent deployments. So what we have done a lot lately is because a lot of these client applications that we're deploying, we actually want all kinds of different versions running, some with a smaller data set, some with a bigger data set. But you also have the variability between production and development environments. So what our deployments typically look like nowadays is it has actually arguments. So it's actually a function which captures these variability. So things like which AWS account to deploy to, which region, which instance type, do you want to have the batch enabled? Because sometimes, let's say during development, you don't want the batches to trigger. But also, is it a production system? So we are a bit stricter with our production system. So I want to make sure that all the monitoring is set up correctly. So these kind of things you can cover with these arguments. And I think that's a very nice thing because it kind of declares what your variability in the different deployments are without needing to copy the files around. So do we use Nixops? Oh, yeah. We have networks and clusters of the 54 machines in one functional unit. So you can imagine, Database wants to do replication, all kinds of clustering. So our biggest network is 54 machines. So I went to dig into all the logs to see how many times people did what. So there were 100 Nixops create calls being done, 88 delete. So that basically means there were 100 new deployments and 88 thrown away. 1700 times people called Nixops deploy, which basically mean there can be anything from, let's say, configuration change to a full-scale new deployment. And 300 environments have been destroyed.  But it can be that, for example, you want to redeploy it from scratch again. And this is being done by 40 different people in the company. So to give you an idea about how many servers we run with NixOS and Nixops, we have a lot of internal infrastructure. You think like the BILDFARM runs on, like on Hasner. But we also have some local servers on-premise. But EC2 on Google Cloud, we have about 50 of them. For our clients, we have a whole lot more. Last I checked, like three days ago, it was 550 machines that we deployed for clients. So that includes big networks that have 50 machines, right? So that doesn't mean we have 500 deployments. But at least, yeah, there's quite a lot of machines. So most of our deployments are kind of like static deployments. So this has a fixed number of machines. But of course, when you use the cloud, you want to do all kinds of dynamic scaling. So we have a system to offload some work for applications, but also for developers, where they can send some work to be done. And that is actually running continuously. And it has, at any time of the week, it has either from zero to 1,500 machines running concurrently. So when Ilko was showing that graph of the unique IP addresses, that was probably me, or at least part of it. So maybe I'll talk a bit about how we actually arrange all these deployments, because there's 40 people actually doing it. We have different clients, and we don't want everybody to have access to all the systems. So you want to make sure you kind of separate people and groups from certain deployments. Also, like operations, they want to have full control of the production environment, so we can never touch them. So basically, we have two deployment machines. There are actually three, but two main ones, one for development and one for production, where we have for each client a different user account, which they can sudo into, at least if you have the rights to go to a client's system and deploy there. So that's how most people work. But there are some issues with that, because, I mean, go tell a salesperson to SSH into a machine and then run some kind of script. It's kind of scary for them. So they don't really like that so much. But there's also another problem. So in NixOps, we actually store the credentials for the cloud that we're using in the home directory of the user where you're running NixOps. So they call that, for example, for EC2, it's the EC2 keys file in the home directory. That basically means that anybody who has access to the user account will actually be able to just grab these credentials. And that's, of course, a bad situation. So we actually want to make sure that that gets a bit improved. But even worse is we have this nice language to describe our deployments. And you think that we have super consistent deployments. But what actually happens is that when people are inconsistent in, for example, how they check out files, which files they use, or suddenly they share a checkout of a repository between two deployments, they actually start to interact. Because people are not disciplined enough to, let's say, use the correct processes for it. So that basically means that you have a nice language to actually deploy your stuff, but you can still get incorrect and inconsistent deployments. So a second thing is I showed some statistics about how many people were deploying. It's actually kind of hard to figure it out. So NixOps actually logs every command that you're running to the Syslog and also logs who is actually doing it. So it is actually possible, but it's still not very easy to do. So we are, well, actually we, one of my colleagues, Osama, from Tunis, he's going to work on the NixOps dashboard, which will be a web UI for NixOps, which will make it easier for, let's say, our salespeople to spin up environments, but also mitigate a few of the problems that we had with NixOps. It will be open source. I'm sure that, let's say, in the next few months, it will just show up and we'll announce it once we have a usable version. I hope you guys will start using it or at least try it. So again, we want to have a bit improved security by actually not allowing people direct access to the AWS APIs. We want to have deeper operational visibility so that we can see what's going on, because currently we have 40 people going somewhere. And for example, when you deploy something to the same deployment as two users, the user just gets like, yeah, there's a lock on it, but doesn't know what the other user is actually doing. So it will be actually nice if you just, in the web interface, it says, like, no, these operations are actually running. And this user is doing that. And very important, we want to have a proper audit trail. So by redesigning this, we want to make sure that that's all covered. Well, and again, this consistency issue, things like checking out source codes for the NICs expressions to deploy. We want to make sure that that's also structured. And basically, by providing an API slash web user interface, that will actually improve as well, partly because we'll just basically allow only changes that were made in source control to be used. So to summarize, I think I was a bit quick, maybe, but that's OK. Kind of lunch. We want to have reproducible, composable builds with traceability. So that's very important for me. I really want to know what's going on in the system when there is a problem. We want to know what changed for which reason. And we want to be able to actually efficiently look that up, which I think NICs and Hydra give us that opportunity. So we want to have reproducible system configuration, no more changing files on the servers like we were doing before. And also making sure that we don't click in the AWS console, all kinds of infrastructure together in a manual way. We need to do automation of provisioning and creating these reproducible networks as well. So I want to say it's like NICs helps us a lot. The fact that you guys are here, so many people, it's awesome. And the NICsOS community has given us a lot of things, like, for example, the heads in our back end of NICsOps. So that saves us so much time. So LogicBlocks is also thankful for that. And that's why we try to give back at least changes to NICs, NICs packages, Hydra, and NICsOps back to the community. But they also support our infrastructure. So the fact that we have such a reliable binary cache nowadays that's fully being paid for by LogicBlocks. So it's hosted on the EC2, on the S3 with CloudFront. And then make sure that we have reliable caches and make sure that people who deploy over the world actually have a decent speed, which was kind of an issue before when we were just hosted in the data center of TU Delft. So I'm very happy that they're so supportive of all this. So this is my last slide. Thank you for listening. And I hope you guys have some questions. OK, we have a bit of time for questions. I'm curious if you ever experimented with having continuous integration or anything, so taking outputs from Hydra and then deploying them immediately with NICsOps? Yeah, so we actually do that. So let me go back to one of the slides, because I probably didn't mention it clearly enough. And I kind of skipped over it. So this is an expression that shows how to build a client application. And so sometimes, usually we work with fixed releases. So every month, we do a release that is, let's say, double checked also manually, except for the automated part. So you can actually pass it like a version identifier, so it's described there in the default value on top. But what we actually also do, what actually happens a lot, is that people build their applications against the integration build, so that's, for example, a nightly build or, let's say, the continuous build. And they can basically just configure that by in Hydra changing the inputs from, let's say, not a string, but to previous Hydra builds. And people point to the correct build. That will be automatically used. Application will be built, and that's immediately deployable in NICsOps. So yes. And then I just had a follow-up question. Is do you see any way that NICsOps can help with orchestration? Like, is that ever an issue? Like, right now you are defining all of your services and deploying them with NICsOps, but maybe there's some binding between which machines get which and all this. But I'm wondering if you see opportunity there to say, to not have to specify that and do some smart allocation of where the application's running. I think that's a bit more in the space of this NICs, who's actually doing that. NICsOps is really about machine configuration. And I don't think that's necessarily a good idea to implement, let's say, such an orchestration in NICsOps itself. I think it's a really good idea to look into the work that Sander had been doing on this NICs, because that's actually meant specifically designed for such a scenario. Thank you. So could you go forward to the test download slide? Yeah, so what is that actually downloading? Is that waiting for a commit to happen, triggering a Hydra build, and then immediately uploading something and then trying to download it again? Or is that OK? Well, so actually it doesn't really do the downloading in a sense that it will have it available, but it will actually do the unpacking. So the function name might be a misnomer, because it's not actually going to the network. It's basically testing what comes out of, let's say, our integration build or continuous build and extracting that and doing basic tests on, well, in this case, Fedora. OK, have you considered having a Hydra that does like an upload to a HTTP server, just to kind of know that you've got that full flow working? Yeah, it's kind of tricky, because it's such an impure thing, so we try to avoid doing that. And I mean, you have typical, basically, checks, maybe with Pingdum to see if you can actually download it, so there's not necessarily that much of a need for it. I have a question about Hydra. Have you had any thoughts of making it more declarative, like declared the projects in Nix or in some? Yeah, we've considered it, but we never got around to actually implementing it. But it would actually be nice, because then you can track changes over time. So currently, Hydra doesn't check, let's say, the job level specification of the inputs over time, but it does track, let's say, the individual revisions that have been used. So it would indeed be a nice feature to add to Hydra to be more declarative. But there's a downside to it, of course, in the sense of usability. So for example, it might be harder for, let's say, our consultants to actually make a quick change in the user interface. It will be harder, let's say, if they need to do it in code. So that's kind of a trade-off. There's one there. Yeah, you mentioned dealing with your built or your deploy users and all that and everything that goes on with there. We have similar issues. And I think we run Ansible and stuff like that. But the same problem exists, keys, SSHPs, et cetera, are just right there for the taking of users in there.  But I mean, I don't want to use a web UI. I want to use my command line. So I'm wondering if you had thought about any ideas on the deal with that. So the web UI, it will actually be an API. So it will actually, you'll be able to develop tools to actually call there as well. But then you actually still have to say, the issue is still there, right? So we'll have to look into solving the credentials, not just by the UI, because there's just one layer, but maybe by some other key management system, like, I don't know, these guys from Hashi Corp have some interesting stuff. Yeah. Hey, it's not a question, more of a remark, like, that maybe before being very ambitious with like, nixifying job sets, we can do hydro CLI properly. And from there, we can sort of like, be more declarative, I guess, at, you know, without doing a drastic step to refactor everything. Yeah, I'm not sure if that's a good idea. I think it would be better to actually focus on making it more declarative, because in CLI you'll still have the same issue as web interface, basically just a different vehicle to do this imperative changing of your, yeah. Yeah, maybe. We could consider it. But if you feel like you, if you ever want to contribute to Hydra, feel free to.


Well, thank you very much for that kind introduction. It's a pleasure to be here and I'll be talking, indeed, about searching new therapeutic targets in melanoma. So I have no financial relationships to the close and these are the goals of this talk. So it's well accepted that cancer is a genetic disease and one of the best examples for this is colorectal cancer since it develops morphologically clear distinct phases starting from a normal epithelial cell onto early adenomas, intermediate adenomas, later adenomas, ultimately leading to metastases. And these have been linked to molecular changes, some of which are seen in this slide. And this has become really a paradigm for genetic development in solid cancers. And we've been investigating this in depth, however, we still believe that the data is the tip of the iceberg and there's more to be found out. And we aim to do this for sporadic metastatic melanoma. So there's over 70,000 new cases of melanoma per year reported for 2011 in the U.S. And over 8,000 people died of this disease in 2011 in the U.S. The alarming part is that the median patient survival is only six months following diagnosis of late stage disease in most cases. So we believe that identifying novel genetic alterations may provide new opportunities for drug development. And so similar to what I showed you for colorectal cancer, this is the histological progression for melanoma development where a melanocytes resides within the epidermis and it develops into a benign nevers, a dysplastic nevers, radial growth phase, then it starts invading the dermis in vertical growth phase and disseminates this metastatic melanoma. Some of the mutations associated with these stages are seen in this slide. The best one, of course, is known BRAF mutations. And although more of these are known when I started my program here at NIGRI, there wasn't a comprehensive analysis of the genetics of melanoma. And so our goals were to perform mutation analysis of the melanoma genome. We then aimed to functionally analyze the most highly and most interesting mutated genes. And then we tried to translate this information back into the clinic. We established a tumor bank together with Dr. Steve Rosenberg at the NCI, as well as Dr. Gershon-Wilford N.B. Anderson and Dr. Robinson at the Colorectal Cancer Center. The majority of the samples, of course, are from here at the NCI. You see the numbers here. So we have metastatic tumor DNAs as well as matched normal DNAs. Most of the time, this is blood. There's OCT blocks, meaning the original tumor that was resected, frozen down. For all of the Rosenberg samples, they're matched low-passage cell lines, which are extremely useful, especially if you want to start evaluating functionally the effects of the mutations. You can make matched RNA from these cell lines as well as protein lysates. And for all of these samples, we have clinical information. This is a somatic mutation analysis where you have the tumor, sometimes the cell line, you extract the DNA, you sequence the genes of interest. You do the same for the normal tissue, as I said, mostly blood, and you compare those sequences. And we only follow up on somatic mutations, so mutations that occur specifically in the tumor. So I'll give you an example of this kind of analysis that we performed for the tyrosine kinase family. So this is the human kinome, and on the upper left hand corner, you see the tyrosine kinases. We decided to sequence these because it's known that tyrosine kinases are highly mutated in cancers, and they're known to be amenable to pharmacological inhibition. Now these have been sequenced in melanoma by the Sanger Center in 2007, but they only sequenced six melanoma samples, and so the mutation rate that they observed was too low to definitely implicate these genes to tumor genesis. So now that we had this unique tumor bank, we decided to revisit this family in that tumor bank. So this is the kinome screen, and every one of our genetic screens is made up of two phases. The discovery phase and the validation phase. And so in collaboration with NISC, here at the sequencing center, we sequenced 29 samples, so a low number of samples, for all the tyrosine kinases. We only sequenced the kinase domain because it's very expensive to do the sequencing. It's a lot of work, and one would expect that if any mutations take place, they would be found in the kinase domain. At this point, we searched for somatic mutations, and any gene that had one somatic mutation or more, and there were 19 such genes, moved on to the validation phase, where we sequenced in my lab 80 samples, and in this case, we sequenced all of the coding exons. And then we analyzed to see which was somatic, and then interesting genes moved on to functional analysis. The one that was most interesting was ERB4, which was mutated in 19 percent of the cases. And you see here the schematic of ERB4 and the various mutations that we identified. So ERB4 is part of the ERB family, which consists of EGFR, which is ERB1, EG2, HER2, which is ERB2, both of which are known good targets for inhibition, ERB3 and ERB4. These either homodimerize or heterodimerize on the membrane, and upon ligand binding, they transfer, correlate each other, forming docking sites for various signaling molecules. So we decided to clone seven of the mutations that we identified, and we chose these seven based on the crystal structure that was made by Dr. Leahy that year at Johns Hopkins University, and also based on conservation through the ERB members. What we did is a biochemical assay. We looked at the kinase activity of these mutants versus wild type ERB4, and what you see is that all the mutants have increased kinase activity compared to the wild type and compared to the kinase dead, despite the fact that they express similarly. So clearly these mutants have an increased basal activation. We then did a classic transformation assay in NIHVT3 cells, and what we saw was that these same exact seven mutants formed an increased number of foci compared to the wild type ERB4, and strikingly similar to the oncogenic rats. So these mutants are transforming. The next question was, are these mutants important for the growth of the cells in which they are harbored? And this is where the power of using these low-passage cell lines comes in. So we could go back to the low-passage cell lines and use SHRNA technology to knock down the endogenous ERB4 within the cells, and we could do this in cells that were either wild type for ERB4 or mutant for ERB4. We could then make stable clones and check various phenotypes. The first we checked, of course, was whether this affects growth, and what you see here is that when you knock down ERB4 in cells that are wild type, you see no effect on growth. Similarly to the control. However, if you do this in cells that have mutant ERB4, it reduces their growth compared to control, and these results were seen in several of these melanoma cell lines. So mutant ERB4 is providing a cell survival signal to these cells, otherwise known as oncogene addiction. So the next step was, is this going to be a good drug target? So we decided to test lapatinib, which is a small molecule inhibitor. It's FDA-approved for breast cancer patients, and when we exposed many melanoma cells that are either wild type for ERB4 or mutant for ERB4, to lapatinib, to various concentrations of lapatinib and plotted an EC50 graph, you can see here that the cells that were mutant for ERB4 were more sensitive to the presence of lapatinib, compared to cells that were wild type to ERB4. So clearly a mutant ERB4 is sensitizing the melanoma cells to lapatinib, and we extended this to additional melanoma cell lines. So we do see a range of sensitivity, which is 10 to 250 in more sense, a fold more sensitive than the wild type, there's a heterogeneity, but it clearly seems to be a good potential target in the clinic. The hypothesis being that about 20% of carmelanoma patients would harbor ERB4 mutations, an exposure of their cells would make them sensitive to lapatinib. So based on this, Dr. Udo Rudloff became the PI and wrote a protocol here at the surgery branch for a clinical trial where with the help of Dr. Meltzer at the NCI, there's clear certified ERB4 sequencing, CTEP is providing lapatinib, and this is a multi-site clinical trial, as I said, involving the NCI as well as Memorial Sloan Kettering, and this particular trial is ongoing and recruiting patients, and at this point we don't have any reports, but hopefully we'll have positive things to say about this. So if these are the mutations that we identified in our first study, these are the mutations that have been identified so far through the clinical trial, and you see that the frequency remains the same or even higher, so we're still finding these ERB4 mutations. These mutations have been identified since our publication through whole exome and genome studies. The ones in purple have been identified by Nick Hayward in Australia, and this orange one has been identified by Levi Garroway. The point here saying is that the frequency is still high, and we're also starting to see some mini hotspots, so recurring mutations in particular locations, which is a characteristic of oncogenes. This is some preliminary data through collaboration with Dr. Elenius, where he looked at ERB4 in the sera of some of these patients that are mutant for ERB4, and he actually found an increase in ERB4 ectodomain in these sera compared to healthy individuals on the T test seem to give a significant p-value. So this is actually very interesting because maybe we could have a biomarker to detect these patients rather than sequencing their ERB4. Okay, so I'll shift gears, and I'll start talking about the melanoma landscape because now we can start using different technologies. We can use whole genome and whole exome sequencing. And so this is in collaboration with the foundation of NIH, Eli Lilly, Elliott Mogollis and Stephen Parker. Our pilot was to first compare low-passage cell line to the original tumor from which it was formed. So melanoma cell lines are relatively easy to form compared to other solid cancers, but the main question is the somatic mutations similar in these low-passage cell lines to this original fresh tumor. And so we did whole genome sequencing, as well as whole genome sequencing of the normal of the blood. The main point to be made is that when you intersect the somatic mutations, 90 percent are concordant when you look at the copina at the CDS, at the coding regions. So this suggests that it is fine to use these low-passage cell line derived genomic DNA in order to look for somatic mutations because these are maintained. When you look at copy number variations, these are less concordant. It's close to about 80 percent, but we're focusing now at somatic mutations. So to continue looking at landscapes, we decided to do whole exomes since it's more practical than whole genomes. It's much less expensive. So we use low-passage DNA to perform whole exome sequencing of 14 tumors on their match normals with Agilenture Select and Illumina Sequencing. And any of the dep validation was done by Sanger Sequencing. We got 12 gigabytes of sequence per sample. The mean depth was extremely high, 180x or greater. We got over 90 percent coverage. We only had 2.4 percent false negative rates and 81 percent sensitivity. This is just the stages for this discovery screen. I told you we captured 14 melanoma samples. We got over 5,000 somatic mutations. However, once you assemble the data and you filter it through various different filters, you find about 4,000 somatic mutations that are divided into missense, nonsense, blycides, insertions, deletions, and synonymous mutations. So mutations that do not affect the amino acid sequence. And it's important to capture this because then you can look at the ratio of the non-synonymous to synonymous mutations. In this case, it was 2 to 1, which is what you would expect to occur by chance. So if this is what you expect by chance, suggesting most of these mutations are passenger mutations. Meaning they have a neutral effect. They should not affect the tumor genus process. Okay, so the challenge here is to find out which are the passengers and which are the drivers, meaning which are neutral and which actually have a role to play in tumor genesis. And this is an open question. Many people are working on this. There are no clear answers. But what we use are statistics, bioinformatics, and functional studies. So what I mean by statistics is to look for recurrently mutated genes, meaning hotspots, the exact same mutation occurring several times in different samples such as BRAF, highly mutated genes, non-synonymous to synonymous ratios. And the mutations should occur above the background mutation rate. So if you start looking for recurrent hotspot mutations, meaning the same exact mutation would occur in at least two samples or more. So of course, we found BRAF, that is a positive control. It's known to be a hotspot mutation. But to our surprise, we found nine novel genes with reoccurring mutations. So we validated these in a larger number of samples. And we found TRAP to have six mutations exactly in the same position. The likelihood of this happening is extremely low, you see the p-value. This is just that it's being selected for, and it probably has a functional effect. Now this particular mutation was also identified by Ruth Hallerban in Yale, as well as Paul Meltzer here at the NCI. And so this is a nuclear protein. It functions as part of a histone acetyltransferase. Its disruption causes defects in cell cycle progression. And our functional studies in melanoma cells suggest the same. Okay, so now if we look at the other side, searching for highly mutated genes, we found 16 such genes. Taking into account also the size of the gene and the background mutation rate. And so again, we looked at these 16 genes and validated them in a larger number of samples. And these are the 16 in this table. So fortunately, we see BRAF up here on the top, a good positive control. But all other genes were never linked to melanoma before. These are the p-values and these are the percent of the tumors that were affected in this discovery screen. When we validated this in a larger sample set of 52, you see that the percentages are concordant to what we found in our discovery screen, suggesting that our methodology is working. So we decided to focus on Grin2A. The reason being is it's the second most highly mutated gene and it's novel. So we focused on it and scaled it up in even more samples. And this is the schematic of all the mutations that we identified. You see there's a very large number. You see they're scattered throughout the protein. We have five nonsense mutations, meaning these would be truncating the protein. And we also have a few places where you have recurrent mutations. The same exact mutation. And actually, this particular mutation was previously seen in cosmic, which is a well-known reliable cancer database. And these ones in red were recently published by Garouetel in nature last week, again showing recurrent mutations along Grin2A. Now Grin2A is inotropic glutamate receptor subunit. And I will touch upon the glutamate pathway again later in my talk. So if this is a schematic of Grin2A and it's other subunit, Grin1 and the additional two that are not seen in the schematic. This is really a calcium channel. Upon glutamate and glycine binding, the channel opens and calcium enters the cell, activating and inactivating a few pathways, ultimately leading to apoptosis. Now, Todd Prickett in my lab decided to clone a few of the mutations in Grin2A seen here with red arrows. And first check the binding of wild type and mutant Grin2A with wild type Grin1 by overexpression. And what we saw here is that when you have the wild type, it binds Grin1. However, the various mutants have reduced a no binding to Grin1, even though they're expressed in the lysate. So since this is involved in calcium signaling in collaboration with Sylvia Gutkind, we decided to look at calcium. And so again, if you overexpress these in cells and activate with NMDA, this is the profile you get for calcium entry. However, if you overexpress a few of the Grin2A mutants, you see disruption of the calcium entry into the cells. So our model now is that if these different stars are mutations in Grin2A, we now have reduced calcium entry into the cell, leading to survival, which is what you would expect in a cancer scenario. So this is just preliminary data showing that when you knock down Grin2A in cells that are mutant for Grin2A, you do not see an effect on growth. However, if you do this in cells that are wild type for Grin2A, you see an increased growth. And this is actually consistent with Grin2A being a tumor suppressor. So this is very preliminary, but certainly an interest of our lab to follow up on. So I'd like to remind you that in addition to looking at specific genes, we actually looked at most of the human genes, most of the exons, so we can start looking at pathways. And the pathway that seemed to be most significant was the glutamate signaling pathway. So you see the p-value right here. And Grin2A, which I told you about highly mutated as well as some of its subunits, HerB4, which I told you about being mutated, and there has been a Nature Medicine paper showing that Grin2A and HerB4 interact, and that HerB4 phosphorylates Grin2A affecting its activity. Now, we've also seen Grim3 to be mutated, and this is another glutamate receptor. And preliminary data suggests that it also binds HerB4. So we're now focusing further in this pathway, since we really think it's relevant for melanoma. So in parallel to our exome screens, we have been looking at the G-protein coupled receptors. This is the largest human gene family, and so we had to use capture methods and second-generation sequencing again to look at these and again in two phases. And the gene that seemed to be most interesting, and I already mentioned it, it's Grim3, seemed to be mutated in 16% and 15% of cases in two different cohorts. So again, we have a glutamate receptor here that's highly mutated and has this particular mini-hot spot. So you see this exact same mutation that occurs seen here in the two different cohorts, and again, the p-value of the likelihood of this happening is low, so it probably has a functional role. And just to show you a quick functional analysis, you see here that when these are checked in melanoma cells and activated by an agonist, you see that MEK phosphorylation is increased, especially by this Grim3 hotspot right here. You see a 10-fold increase in phosphorylation in phosphomek. So you do have a functional effect. Now we know that the MEK pathway has effects on migration and invasion, and so one study we did was looking at pulmonary macromatastasis formation, and you see that this is increased after tail vein injection of cells that have a Grim3 mutations. So if this is the paper on Grim2A, you see another paper talking about the glutamate pathway, specifically in this case, Grim1 showing that it induces a melanoma in mice. This is another paper showing as well by people at the NIH showing that Grim5, another glutamate receptor, induces melanoma in transgenic mice, and this is our Grim3 paper, and the common denominator being, again, the glutamate pathway, emphasizing its importance. So what are we planning to do? So we're planning to further delve into the melanoma genome, mainly because if you look at the mutation frequency of melanoma versus other solid cancers, you see that it's extremely high. It's similar to lung cancer, probably due to the UV etiology that melanoma has. And so we really need more samples to be sequenced in order to find out which are the passengers and which are the drivers. And so that's exactly what we're doing. We have, at this point, 18 whole exomes. We have 10 whole genomes, and we can merge the data. And when we do that, we can search for recurrent mutations, and we're finding even more, in this case, that 38. And again, search for highly mutated genes, and we found 15. The good news is that seven of these 15 have previously been linked to melanoma, and these are the seven, BRAF, again, on the top. So this suggests that our methodology is working. It also suggests that we are reaching a plateau, at least with this particular melanoma cohort. So we will further analyze these genomes. The emphasis will be on tumors that are wild type for BRAF, since BRAF has recently had an FDA-approved drug, and we're looking for targetable drivers in tumors that do not have the BRAF mutation. Since we're doing whole genomes, we can look at structural changes, such as amplifications, deletions, rearrangements. We can look at non-genic targets, such as the regulatory regions, the introns. The gold standard, we believe, is doing functional analysis. Since we're doing this in low throughput at this point, we are focusing on Grin2A, ERB4, and the glutamate pathway. And we believe in genetic integration across melanoma data sets. There are many groups doing this at this point, so we are integrating data. TCGA has taken up melanoma as well, and so they will be releasing their data. And so we really hope that integration, such as the one seen here, where we're already integrated with Nick Hayward, and you can see that from the over 1,000 mutated genes that they found, 446 are found in our sample set. We hope this will help us look for common denominators and therefore drivers. So if now the disease is being categorized by clinical and pathological aspects, we hope that in the future, somatic mutation signatures will be added to this particular categorization in order to help to develop more targeted therapies. At this point, I'd like to acknowledge many people, especially in my lab. Todd Prickett, who did the ERB4 in the Grim3 story, Shomu Wei, she's the first author on the Exxon paper. Jared Gardner is doing all this integration on whole genomes, also done by Stephen Parker. Of course, a very, very big thank you to Dr. Rosenberg and his group. Without these samples, we wouldn't be able to do all this work, as well as other sample providers and many other people at the NIH, Elliot Margolis and Stephen Parker, who are helping us with the whole genomes. Thank you very much for listening and I'll be happy to take questions.


My name is Ada Koonle, I'm running a race in Brooklyn, New York. So it's a pleasure. Oh, yeah, cool. So it's a pleasure to talk and see you today. Before I start, I know my name is difficult. So I got an outline like that. If you guys want to see with me, just add a Koonle Oduye. So stop that hard. See, yeah, nice. See, everyone got it. Cool. During my day job, I'm a product designer at NASDAQ. We work on web-based platforms for invested relationship professionals. Sounds pretty complex, which it is, but that's that. I'm also one of the co-organizers for the Gotham SAS Meetup. Yeah, I see. And also for SASConf. So before I get into my talk, I want to talk about what I'm not going to talk about, which is if designers should learn how to code. This has been a debate for the past two years. I was hearing a lot about it. But for me, I think that's a place for both coding and non-coding designers, especially within our team. We have a 30-person team where some people are focused on the visuals, which is pretty good, because they push the boundaries while they got designers like me that lives in within the boundaries and knows what it's capable of doing or not. What I also would be when I speak about is SAS basics. Hopefully, I'm assuming that everyone knows what are variables, make sense, and so forth. So hopefully, I hope you guys will learn is that how SAS could introduce designers to programming concepts, and also how it can make a better developer and designer. I know for me it's definitely helping out just learning the ways of the web and how it works. And also, it seems kind of odd, but I think learning how to code actually made it better for me to communicate my ideas and be able to collaborate with both designers, developers, and also stakeholders. So let me get into my story real quick. I started out as a traditional print designer. I had no coding experience at all. I think the only class I had was like a dream river, and I hated it. Actually, I think I failed it, but that was it. So I came in to have an internship, my first, after I left school. And I was doing basically promotional items and typical print stuff. So what had happened a couple months down line was I did a couple of mock-ups for a website.  And I was like, not really, but I just, I got promoted from intern to actual full-time. And I was like, I guess I got to do it, right? So I did it, and I was like, OK, challenge accepted. So it was actually a OK experience. The good, it was actually functional. You click on links, it actually went to the right pages. And that's about it. Because if you actually look at it, I was very disorganized and redundant with my CSS. My CSS was very wet. Kind of like, what are you saying? Yesterday. Also, it took me a long time. It took me for the first project about six to eight weeks, which at the agency, it's a lot of time and money. It was kind of funny because for that project, the client refused to pay the whole amount.  So I was like, oh, man, I don't know if I'm up to this. But I was like, it was a great scene, my work coming to life, and it was like, it gave me a good feeling. And I was like, yeah. Yeah, I was doing exactly like that. So it was great for me to see my work coming to life, and I was like, I think I might want to do this for a long haul. So this is where my code addiction started. My code addiction, not code, but code addiction. So we went to the first stage of addiction was experimentation. I started after having at least four or five projects under my belt. It was, I knew the whole process of it. So it was coming to a point where I didn't really need to go to Google or CSS tricks or anything like that to actually figure out what I'm doing. I understand how to make layouts and all this other stuff. But the more I was doing it, the more I was like, I'm getting kind of bored. It was like the same thing over and over again. And I was like, is there a reason why it's boring right now? Like I said, there was a lot of redundancy. Since I was working by myself, I just did whatever I wanted to do.  And I was like, I don't know. I'm just Googling and stuff and it's making it look good. So my coding buzz started to wear off. And I was like, I need something else. I was like, what's next? So I was like kind of thinking, I was talking to some of the developers. He's like, yeah, you should try JavaScript. I was like, OK. Because HTML CSS was conquered. And I was like, I guess this is the next jug of twice, right? But something happened. We didn't mix at all. I didn't understand. I was like, I was trying to figure it out, but I couldn't figure it out at all. It was kind of like oil and water, or Batman and Joker, or New York or Boston. I know a couple of people from Boston here, so yeah. So there was four main reasons why it didn't actually work. First was understanding the syntax. It was definitely a big learning curve for me, since I was more of a traditional designer. I had no CS background, no coding background. So it was kind of harder for me to understand. Second was definitely a different thinking process. Because if you think about it, you have to account for many situations. Whereas in the CSS, what I was doing before was pretty much upfront. The docs are very heavy. I'll look at the docs, and I'll fall asleep. And I was like, oh, god. And also I tried many educational platforms like Linda, Code Academy, all that stuff. And it didn't stick. It didn't stick, because most of the time they'll teach you the syntax, but not how to think about it. So when I tried to do my own projects, I was stuck. I didn't know where to start. So this made me very frustrated and sad. And I was like, I don't know what else I'm going to do. And it was really interesting, because once I started going to Meetups, just trying to get better and stuff like that, people was like, oh, yeah, you're having trouble with this, is that you should try SAS. I was like, what is this thing called SAS? I was like, and they were talking about it's basically CSS with superpowers. And I was like, whoa, I want some superpowers. I was like, I want to be part of that. So I started, and the most interesting thing about it was that SAS was kind of like a presentation language with basically scripting language capabilities. So with the SAS script, you could do many things that you could do the same thing within JavaScript and any other abjuring language. So the problems to solve, now my code is kind of dry now. I was like, OK, I'm kind of feeling this. I was able to break down my CSS folders, because I used to work with a big, large style that CSS folder where it was trying to find things. It was like a hot mess. I was finding and replacing different stuff, and then it will mess up other stuff. And it made my code more scalable. So this leads you to the next stage, which is basically regular use. I started using it regularly. I was like, any side project I had, I started using SAS. And I was like, yeah, I'm starting to like it. And it came to a point where I just couldn't go back to CSS. Even if I had to produce CSS, I would just go somewhere. I do use a SAS Meister where I could just do my SAS, compile, and then everything's there for me to just copy and paste in. And also, it was so good that it just kind of made me lazy, actually, because it has all these things that make your life easier, such as pattern frameworks like bourbon compass, which is great for vendor prefixes and all that stuff, which you probably don't need anymore. But we have these grid systems, which I used to Susie, and I love it, because it removes. You don't have to actually change classes within the markup to actually change the layout, which was pretty awesome for me. And if you want to check out more, take out Sashay for more sassy stuff.  But with anything, with great power comes with great responsibilities. That's from Spider-Man, if you don't know. Uncle Ben, my favorite quote. But the thing I was doing there, I was kind of overusing it. Like, I was doing too much with it. I didn't know how to stop, because I was just so hooked. And I was like, I just got to keep on going. So this is the next stage, which is substance abuse, or a SaaS abuse, either way. So the first one was kind of funny, because for me, I thought you want your CSS to appear just like your markup, right? Is that a good idea, right? So you end up with the nesting inception. But this wasn't a good idea. And why it wasn't a good idea? Because your CSS is not componentized right now. And I didn't just go one, two, three, four, five, six level deeps. I went eight levels. Yeah. So basically what I was building, I was building pages instead of actual components. And it came to a point where I had, it was like for my actual personal website. So it was like 14 big pages that was there. So I was thinking about it, I was like, oh, damn. That's a learn, right? And I was like, oh, maybe I should have refactored it. But it was just so much work. And I was like, I don't have time for this. I was like, it was so much work. And I was like, you know what? This gives me another reason to actually build my website again. So that's why I did. The second overdose episode was, I call it the extend burgers. This is where your extends kind of look like this. I mean, you're selective is this extend after extend after extend after extend. Interesting enough, I thought extends is actually the faster way of doing it, but it's not. There's a great article from Shea from Belly talking about mixins versus extends and extends. I mean, mixins won by a mile because they were actually simple to use, faster to load. And then they were also creating the smallest file size. And also, it's also easier to debug with mixins than actual extends that I found. And also, I was actually using a lot of important, which is not a good thing, so don't fire me. The other thing was the overusing sass itself. I think sometimes when you're writing sass, do you forget that it's CSS and not JavaScript? So I don't know if you can see that. But when it starts looking like this, and especially for someone that doesn't know sass, you're going to look and it's like, wait, am I in the right file? Even to this day, I don't know what this does, but it does whatever I want to do, so whatever. So those are the three problems. So after I kind of figured out my overdoses, and I was like, all right, I know what to do now. Stage four, it was a full addiction now. It was a full addiction right now, because I was using so much that I couldn't go back. And every time I see a design, I was like, hmm, I wonder how to do that in sass, or how to make any way where it's just scalable and you don't have to write that much code. And I was like, I need to do it all the time. I was like, I was feeling for it. Yeah, I was like.  Sass, it brings me to the first benefit, which is basically how sass introduced program concepts to designers. Like I said before, I had trouble learning JavaScript, but once I started to get familiar with sass, it was just kind of easy, because it has the same concepts where it has variables. Mine is different syntax, but it's the same idea. We have arrays, which are lists in sass. We have objects, which is maps in sass. We have functions, and we have loops. So now I had a good understanding of how to make my work with JavaScript now. So I started experimenting with jQuery first, and then I started getting my feet wet with Azure JavaScript, which is more of like conditional and programming and all this other stuff. And with these two tools, it was very easy to actually design and implement your designs very quickly. At NASDAQ, that's where most of our team does our, most of our designers know how to code with HTML, CSS, and then JavaScript. And we do this thing called rapid prototyping. Rapid prototyping is basically an adjust when you design something in your application, you validate with team of designers, developers, and stakeholders. The process is iterative, you basically do three steps, you basically do three steps, which is prototype, refine, prototype, review, and refine. See with this is that sometimes your designs is not right at the first try. You have to keep on designing and designing and testing it out with your users to see if it works. It's like, if I borrow a pair of jeans, I probably want to try it on before I actually buy them, because if I commit to them and they don't fit, then that's a problem. So that was kind of like the story we were trying to solve. Benefit number three, like I said, it's definitely a great tool for today's designers. Most people believe that designers shouldn't be able to do USAS or JavaScript, but I think it's now more than ever that our job responsibility is it's so much that we need to actually design the actual product, close to the product as much as possible. And we should probably design and test our interactions and why it makes it easier, because now we have to test for different screen size, resolutions, browsers, now we have the eye watch and all those other stuff. So we have so many devices and situations we have to design for. We have to test for the user interaction, which is why most of our team uses JavaScript because it does click events and feedbacks, which you can't do just with CSS. It also comes with things called visual conditionals, which is basically a way to set up conditionals for when different designs activate. Like I said before, it makes the designers focus on interaction over visual design, especially when we work in sprints, you don't really have time to actually focus on how the type looks, how, what color to use, they should be something in place for that. The benefit improves communication and collaboration within your team. So in NASDAQ, we have 30 people in our product design team. We have about six, seven product owners and at least 40 developers. So it's, if we didn't know how to code or do any prototyping, it would be hard to actually communicate our ideas to us. So being that we have all these skills, the benefits to designers is that now we can set up design frameworks, like pattern library and style guide, which basically keeps the design consistent itself. So when you're actually designing, you don't have to actually spend your time thinking about it, you can just reference to the pattern library style guide and that will help you out. And also it benefits the developers too, because now you speak their language, you understand their constraints. It's less the developers. I know for me, I used to do mockups and then do annotation, which I hated. I was like, I don't want to type words, I just want to code and stuff like that. But this was a good thing just to deliver some actual prototypes to them. And it also gives them freedom to design too, because most developers won't admit, but they kind of like designing too. So you give them the tools, style guide, you give them a pattern library, sometimes they come back to you and they're like, yo, I got this, what are you thinking about this? I was like, oh, I thought you didn't say you liked the code. Now you know. And they will love you.  Benefits for the stakeholders. I think it's very important to have actual working prototype. And a prototype is different from a static mockup because a prototype actually includes both interaction and it tries to include real data too, which I think is important just for testing on the market, especially for usability testing, which what we do in NASDAQ. And also when you come to a meeting, it's a great presentation tool. So usually you send this out before the actual meeting and what they will do is they will play around with it and then they will have a set of questions when they come to the meeting, which makes the meeting more efficient rather than having a two, three hour meeting. They ask the right questions too. So it's not about how it looks or is that font too big or it needs more splash or whatever like that. It's more about is it answering the right question? It also gives the ability to see any pit holes in the ideas because I know when you're designing it, you're focused on one thing, but you gotta think about it. Whatever you design is a reflection of the whole product itself. So teamwork makes the dream work, it makes you happy. I think I started in NASDAQ and it's kind of funny because even it took me a while just to get used to actually showing unfinished work and that's one of the benefits of actual showing the prototype. It gives them the ability to actually test it out and it's interesting that most, if you include them in the process itself, people will get excited about what they're doing and basically you're all in the same team because I know in some companies where there's definitely beef between the designers, developers and the product owners and but you have to understand that if the product does well, everyone wins. So I'm actually gonna stop kind of short but the takeaways, hopefully you get, is that how SAS introduced programming language to concepts. I think now for me, I don't really consider myself a designer developer, I consider myself a prototyper, which is I think an interesting role. So now I'm more focusing on if what I'm designing makes sense. So I'm able to validate my designs of decisions. Also, it improves collaboration and communication within the team itself, which I think is very important. Working and as they actually told me how like, you have to be patient with everyone because sometimes you can't just do whatever you want, which is probably best. Some resources, definitely check out sasling.com, SAS way, those is great tutorials for basically events, beginners and anyone in between. SAS Mites is a great application where you can actually write your SAS and see how it compiles in TSS, which is a good thing to know about. We have SAS Bites, which is a great podcast where that's on YouTube. And also there's a lot of people in the community like Hampton, Chris Epstein, Claudina, Sam Richard, Scott Kellum, follow them on Twitter. Yeah, thanks for listening. Questions?


Hi, I'm Tomasz Weinger. I'm from Kulabara and I will present to you improved documents searching the LibreOffice. So first, I would like to clarify what is meant by searching, improved searching. We can search for documents in different ways. First is like searching internally and searching externally. So searching internally like inside LibreOffice is when we are just traversing the internal document model and search for some string. But there's also another possibility here. This is searching externally. With searching externally, what I mean is generally that we input the documents into a search database and using the search database then we can search multiple documents for phrases. And this is generally what is this improved search with LibreOffice meant in the title. So generally, when we search externally like with the database, we have to transform the documents into text and feed it into a search engine which then searches inside this transferred text for a search result. The problem here is that we don't get a really good context of these searches when there is one result found. So we get, okay, we have in this document we found this search result, but we don't really know where exactly this search result was found and what is the context around the search result. So this is what we want to improve here. So you can search for phrases in multiple documents as I said, and for this there exists already multiple search platforms, search databases. The one we used and it's very popular is Apache Solar, which also uses Apache Tikka to transform the document into text. Like Apache Tikka is a library like Java library that can open the document and transform it into HTML or just plain text. Another one, which is also very popular is Elasticsearch, but I really don't know much about it. Mainly I export Apache Solar. What is the general idea? The idea here is to use LibreOffice and collaborate online to add this context of the search results. And we had the idea for a search result we would render an image where this search result was found inside the document. So this is now the solution description, what needs to be done to get this idea realized. We need to somehow create the search data and put it into a search and indexing platform, like into the search database. And we have to import it inside the search data platform itself. And then we have to search on the search platform and get some result. After that we have to render the image of the location of the search result. So generally first three steps are already implemented elsewhere. But the last one, I don't know if there is some solution to also render the location inside the document and show the search result to the user. Okay, so now the first step. With LibreOffice we can create the search result data and how we implemented this is to implement it as a new export format. So this means that you can also export S or save S and it will save the current document into a search indexing data format XML. The good about this approach is that we implement this as export format is that it provides a lot of things out of the box. So without just implementing it as export format we can just already use it on the command liteco with this S office convert to switch as it says here. Additionally it also we can use just LibreOfficeKit API save as function to just create this search data document. This LibreOfficeKit API is already used by Collab or Online which provides a REST service convert to so we can already use that we don't need to implement it. Okay, so next I would like to talk about the data format. The data format of the search indexing data is just a flat XML file. The idea here is why is that this flat like means it doesn't have a lot of nested elements. It's just to be very very simple so we can easily transform it to a vendor specific format like the one used for in solar for example. So on the top now we can see how one example of this search data format has like root element indexing always and then it has child elements either paragraph or object where paragraph is just one paragraph inside word inside writer. And object also not just writer it's any shape that has some text is also exported as a paragraph. Another element is also object which can be shape or image or phone work. This one is mainly so that we can provide additional metadata for the object so we can also search inside the metadata not just paragraphs. And the paragraph has an attribute. The most important attributes are index and node type and addition to this also object name. With this we can then identify for which paragraph we are searching for inside the object document model inside LibreOffice. For objects the important attributes are object type and name. Name is always uniquely identified so we can always identify inside the document each object just with the name. Then we also export other attributes. These are like additional metadata as already said old text and description of the object. So how this is implemented is that we have an indexing export and indexing node handler classes. Indexing export class just a root class for search data indexing export which then delegates everything to the indexing node handler. And indexing node handler is just a subclass of model traverser. What a traverser is just a class visitor that visits all the elements inside the document model and then delegates this to what to do with these elements to the handler. An index node handler then just writes this inside into a XML file with the structure that I already explained before. Model traverser is derived from accessibility check functionality which also needs to traverse the document model. But currently because it's not it's like it's just a copy in accessibility check is not yet using the model traverser. But the idea is that both reuse this class and maybe some other other uses can be found for this. For example there is one one uses to to search our colors document colors that the document uses could also use this model traverser. But this is something that we will implement later. So the next step is then to render image for a search result. So we now from the we now perform the search and we get a search result. The search result now has to have all these metadata additional metadata index node type. That is important for the identify which paragraph or which object it goes inside the document model. And with this information then we can we can render the result. So this process is divided into two parts. First part is that we need to get the rectangle the location where the document the search result data in the document is located. So for this we have a search result locator class that is used for this. And the search result locator can then use either XML or JSON search result data as the format. It also also uses the special structure which can which is used inside tests. When we then get this rectangle from the search result locator we can then just render the render the image with paint tile API that is already implemented inside LibreOfficeKit. So next is then to implement render image service inside Collabora online. So we can use this service for the on the web. And for this created an array service render search result which is very similar service that already exists converts to which I was mentioned previously. That is used to create search data for indexing. So what is needed for this search rest service is that we need to provide document and we need to provide the search result. Then we send this execute the service as send this both to the Collabora online server and we get back the rendered image of the search results location. So this I now explained mostly what was done on LibreOffice and Collabora online but how everything is now fitting together. This is including the database including including the Collabora online server including all the pieces so that the user can search. And for this I created a proof of concept web application which this is looks like this. I will demo it later so maybe first I will explain what this proof of concept web application does. So it's just a simple web application that demonstrates how everything should work together. This is then using Apache Solar as the search platform. The HTTP server is just Python simple HTTP server which is then using Python for also Python for server side processing also for sending for sending rest executing rest services on Collabora online and Solar. And it uses HTML and JavaScript for the client side. And the framework that uses is AngularJS. This is just something of us previously familiar with and is very strong with data binding and rest services. And bootstrap for the UI so it's easier to build how the application looks. So the last thing is of course what we need is Collabora online server so that we can render the image for the search result and also to open the document itself. So that application has like three major processes that it performs. So first one is the re-indexing process. This is just needs to fill the Solar database with search data from the documents. Maybe what I forgot to mention is web application is taking care of one folder in arbitrary folder where all the documents are stored and all these documents are then taken in account and printed as are available for opening and indexing and searching. So the trick is here that we need to re-index every time that document changes. So how it is implemented currently we always delete all indexes and re-index everything but ideally this should happen only when a document changes. Then we need to re-index and we need to only update the indexes not just remove all indexes and add all indexes. We just need to remove all indexes for our document and add new indexes for the change document. Of course if the document is deleted we need to remove the indexes from the database. So for each document how we do re-index is for each document in this document folder we request the XML search data from the Collabora online server using this convert to service. Once we get this XML file back we can then transform it to the solar format. Solar has a little bit different format for entering all the search data into the database. So it has a notion of mainly documents and fields. Document is not like a LibreOffice document but document generally corresponds to a paragraph or object and fields are then additional metadata. We also add special field file name to identify which document it handles and special field content which is then the paragraph text. And then we can submit this search data to solar using HTTP PostService. So search process. Solar has a very extended querying API and we just don't need everything here but we can use of course all these querying API if needed. So how we search with solar is we just send a simple GET HTTP request to the solar server and as a response we get a JSON document with the results or with no results depending if the database found something or not. Other formats are supported like XML but if we are dealing with a VAD application, JSON is the simplest to do. We don't need to parse it like for example XML. Yes, web app only searches the paragraph text currently so only content field is important and this is the only field that we search in. But we could also search in other fields so for example to limit only for certain type of objects, certain type of paragraphs or something like this or just additional, just for example if we want to search just one document we can search only in the file name field. When we get the results back from we need to transform the search result again for something that the LibreOffice can render the image and Libre, currently LibreOffice supports either JSON, as I said, a JSON or XML and the search result needs to be compatible with that. And also we can reuse this JSON inside the VAD application itself to show the search results. Generally it's just an array of objects which has key and value pairs for the meta button. Then of course then we have to show the results in the VAD application. Now last is rendering the image now. After we show these results on the VAD application we can then request rendering of the image for each search results. This is done asynchronously so we can show the results first and then render the image for and update the search results when the images get rendered. So we use a render search result series, we send search results and the document to the collaborate office, a collaborate online server and then we get back the image in just pure binary PNG image. And with that we can transform the image to base 64 string. This is generally done because it's easier to deal with this on the VAD application. So the demo. So this is the web app, here we have a list of documents and as first thing we need to re-index all the documents. So this here shows the status of what is going on and the current says re-indexing and now it's finished re-indexing. Now we can search, we can simply search for let's say LibreOffice, search. And now found in about LibreOffice document we found a couple of search results and this is now rendering where the search results are found in which paragraphs. These are now images inside the document. And now we can search for something more general. For example we can say web and now there are multiple results of a web that are in multiple documents. Here we see that this result is font work and it found a result this web inside font work text. If you go down you can see like this is image and it found the result inside the caption of an image. The similar like image is also a shape in font result in the caption of a shape. The following two ones are shapes and rectangles and in font the result inside the shape text. And so on we have next a couple of paragraphs in other documents. So if we are interested how this document looks like we can just click here and it will be open the document inside the collab around line. And this is the document. We see that we have here the image, we have here a shape. There is also a table which is also shown as a result and another shape. And this is the font work object that we show previously. And then I'll go back maybe search for Lorem. And we have like this Lorem documented. Lorem is found in a lot of places, a lot of generally just paragraphs. And here we search finished and it found nine results. And generally if we go now inside here and change the document we always have to re-index the document or the search result won't be found. All the changes in the documents won't be found in the search result. So this is all for me. That was my demo and thanks for watching and bye-bye.