It looks like it's about one o'clock if my computer's right, so I suppose we'll dive in since there's kind of a lot to cover here. So the title of this talk is, What's the fuss with all this JavaScript? So real quick, a little bit about me. My name is Blake Hall. I'm a developer and trainer with DrupalizeMe and Lullabot. If you haven't checked us out yet, we've got a booth downstairs, 407 and 411, they're right next to each other. I bought by and get some pony stickers. I forgot to bring some with me now, but I'm sure we've got some down at the booth. Lullabot, for those of you that might not know, is an interactive development design agency.  Today what we're going to cover, we're going to talk a little bit about the problem that JavaScript's trying to solve and where JavaScript frameworks kind of came from. We'll do a really quick kind of one-on-one level intro into Backbone, Angular and React and kind of compare and contrast all of them and then hopefully we'll have some time for questions at the end. So a little back story, I've been doing Drupal for give or take 10 years and I got kind of burnt out with it a couple years ago and I just sort of didn't feel like I was learning a bunch of stuff and wanted to go play around with other things. So I spent last year going to JavaScript events instead of Drupal events. This was taken at NodeConf in Petaluma. It's a tech conference that involves camping. You're out in the woods, it's really hands-on. I highly recommend it, it's the closest I've ever slept to another Lullabot. That was a tent for Jared Bittner and I, but I highly recommend it if you have any interest in Node whatsoever, it was a really, really fun event. Another great one was JSConf that was at Amelia Island in Florida. The parties there were ridiculous. The last night we had sort of an adult carnival. So that's a bouncy castle for grown-ups. I got to put on the giant arm-length boxing gloves and punch Eric Durand, which was really, really satisfying. And it was just a really good time, there were a lot of really good talks. I highly recommend getting outside the Drupal ecosystem if you haven't done that for a while. Like I said, it was a lot of fun. Grilled turkey legs, amazing, funnel cakes, all kinds of stuff. So what does this have to do with any of this? This talk was actually kind of either shamelessly stolen from or inspired by a couple of talks at JSConf last year. Marco Rogers did one on finding common patterns across different front-end frameworks he's worked on. At the time, he was one of the team leads at Yammer. I think he's moved on since, but super interesting talk. It's on YouTube. You can go kind of check it out. And then Ryan Florence did one. I'm not even going to try to pronounce that session title, but it was sort of a mash-up of Ember, Angular, Backbone, React, Polymer, yeah, okay. Another really good talk. He goes into a bit more detail in that one about sort of the different areas of concerns between the different frameworks that I think we'll have time to tackle. So those are two really, really great resources to kind of check out if you're interested in this stuff and want to kind of go in more depth. There will be links to those in the show notes for the slides. I'll get those uploaded as soon as the session is done. So meanwhile, back in the Drupal world, I double-checked this morning and Backbone and Underscore were committed to core as part of the Spark project about three years ago already. So they've actually been in Drupal 8 for quite a while now. So just like when we added jQuery back in, I don't know, what's that, Drupal 5, there's a new tool in the toolbox to kind of figure out how to leverage and take advantage of. So I think it makes sense to kind of step back and look at what other JavaScript frameworks are doing as well and see sort of where Backbone fits into that landscape and when it's appropriate to use Backbone versus maybe some of the other tools. Another thing to kind of set some context is I pulled these numbers yesterday, actually, and I saw this morning on the carpet that Drupal.org was actually first launched in 2001. So the website's been around for, give or take, 14 years. Back when I started 10 years ago, there were about 350 contrib modules. So there was actually a point in time where I'd installed and played with every module that was on Drupal.org. I don't know if you can see those in the back, but right now we're at 30,000 modules. So I've lost touch quite a bit. I haven't installed probably 26,000, some of those at least. And I also, I went down and filtered and it's something like 17,000 actual modules and then the rest are sandbox projects, which kind of blew my mind. I hadn't looked at those numbers for a while. I was sort of back at the envelope guessing we were at 12,000 modules. So 30,000 is a bit crazy. But just for comparison, in 2009 NPM came out and that's sort of JavaScript's package manager of choice. So it's only six years old. It's sort of the home for all of these JavaScript libraries and frameworks or at least it's an easy place to sort of download and deploy them. And if you can see that number, I'm not sure if you can in the back, but there's 147,000 packages. And this was yesterday, this morning when I checked it was 149,000. So in six years they've managed to share five times as many packages. I know it's not a total apples to apples comparison, but it'll give you a sense of sort of how quickly that community is growing and how much JavaScript code is out there, which again was just sort of ridiculous. 148,000 packages in six years is a lot of code. So where did all this kind of come from and how did JavaScript sort of rise to popularity? So back in the old days, and by old days I mean 1996, MapQuest kind of launched for the first time. So I think back in 96 this was probably a good example of a pretty complicated UI. So you've got some buttons that are basically doing form submissions and full page reloads anytime you want to do any sort of navigation at all. So this is sort of the complex internet circa 1996. Then this Ajax thing kind of happened somewhere between 2004-2005 with Gmail and GMaps and whatnot. And a lot of times we're still doing the similar kind of Ajax stuff that's been around for now 10 years. And a lot of times it's not actually Ajax, right? We're just doing kind of naive HTML replacement via third party API request. And more or less if you think about it, it's just a transformed version of what we were doing back with MapQuest. It's just happening in the background and we're doing some client side processing. So this has been around like I said for about 10 years already. So 2010 or so Backbone came out. It was written by a guy I believe was at the New York Times at the time or maybe he transferred to the New York Times after the fact. But it was sort of a thing that was built to help manage the complexity around some of these client side interactions with data and being able to keep your client side browser and the server in sync. So it was just sort of improved code organization to handle all of that Ajax management stuff that was happening in more complex applications. Browser functionality was sort of increased quite a bit and people were building apps that were a bit more native like at least back then about five years ago. So now we kind of find ourselves in this situation where there are pretty much JavaScript frameworks coming out left and right every day, right? And how do we go about making sense of it? How do we figure out where to invest time and energy? Hacker News certainly isn't a great place to sort of figure out where to throw your business and kind of which particular framework to put your time behind in investment and learning. So stepping back a bit, let's talk a little bit about what these frameworks have in common and what kinds of problems they're trying to solve a little bit. So this is a really, really basic definition both Ryan and Marco used it last year and I think I agree with it pretty much. If you're going to have a client side framework you want it to do some sort of data modeling. So an object oriented style like method of encapsulating and representing your data. You want to have some sort of presentation layer that you can handle updates to that data model and keeping things kind of in sync dynamically. You want to do some kind of routing so you can use the back button in your browser and so things are URL addressable and whatnot. And then obviously you probably want some sort of method for talking to the server that's feeding you data so you don't have to do page refreshes all the time. So really, really basic definition but for the purposes of the next hour this is what I'm kind of considering a framework. The other thing I want to point to, I'm sure some of the code samples that are going to be up on the screen are going to be hard to see in the back. They're all lifted from this to do MVC.com site. If you haven't seen this before I highly recommend checking it out. It's basically a really, really simple one page to do list management app written in every single library you can imagine basically. All the codes on GitHub, you're comparing apples to apples in that case and you can kind of see in most cases the actual author of the framework is the one that's contributed back to this project. So it's sort of like a best practice version of an implementation in language X, Y, or Z. So it's really kind of a handy resource. Here's an example of what the to-do list app actually looks like. All of them are pretty much identical. The CSS is the same. The markup's obviously different depending on what the framework recommends. The one that's screen capped here is actually just vanilla jQuery stuff. So you can, if you're already familiar with jQuery and you haven't touched one of these frameworks before, you can sort of look at that and make sense of that and then compare and contrast it with some of the other frameworks that we'll get to. It uses local storage, so as long as you're not quitting your browser, you can kind of maintain your to-do list. It's just some pretty basic functionality. You add new items, you can delete items. You can filter them based on their status if they've been completed or not. And then you can sort of flush the logs so the completed ones disappear. So it's a really, really simple single page app. Most of them are in the neighborhood of 200 lines of code or so in the more complicated portions depending on which framework we're talking about. So let's start with the backbone example. I mentioned it a few times already. It's in Drupal Core and we're at Drupal Con. So since we have access to it, it seems like a good place to begin. It's also the oldest. So one of the sessions that I watched kind of prepping for this came from JSConf in Uruguay last year. And the original author of Backbone basically said the problem he was trying to solve was building the ideal web API for the client. So if you're in a browser, what would the ideal way to interact with remote API be or what would it look like? And one of the things he mentions that I think is really, really true, especially if you've done a lot of jQuery development, is it's really, really easy to just grab data from the DOM and treat that like the actual source of truth rather than something that's coming directly from your server. So if you wanna say highlight posts on a page that have more than ten comments and you've got all that data in a table, you can kind of pull that and sort it differently. But that's not necessarily the greatest object-oriented way to organize things. If you have an actual model that has that number associated with the actual data itself, it's a little easier to reference, especially if the DOM's changing because of some other crazy ad on the page or some other plugin that somebody kind of stitched together. So the moral to that story is basically the DOM is a user interface. It shouldn't be your data source, at least according to the backbone folks. So most back-end devs especially are probably familiar with the MVC pattern. Backbone kind of calls it MVP, Model View Presenter.  I'm not gonna pretend to understand the differences between all of them. But if you wanna go read Wikipedia, Backbone considers itself Model View Presenter. Here's a pretty good slide I think that kind of wraps what Backbone is trying to do at least at a high level into a nice little picture. Basically, you've got user input on the actual page is gonna trigger some event in the view that then triggers a data change in the model. The model will sync with the server. It emits a change event, and then the view re-renders itself in response to that. So the presenter layer is sort of not clear in this diagram. And that's one of the reasons I kind of left it out in the last slide is it sort of doesn't matter where it is. You've got a view that is showing the data. You've got a model that is the data. And then there's sort of some magic that's happening in between to get those two in sync. So let's take a look at the actual Backbone model from that to do app real quick. I don't know if you can see it there, but on line 11, basically there's a base Backbone model class. And anytime you're creating one, you just extend it with your own new custom class. In this case, it sets up a couple of defaults. So there's an empty title and completed is set to false. So it's a really simple data model. And then there's the idea is that you add sort of extra rich methods that sort of add functionality to your data model. In this case, the only method is toggle. And that sort of sets the completed state to either true or false, depending on if the box has been checked. So it's pretty simple kind of 00101 stuff. Just come up with your data model and extend Backbones built in class and build it out. So the view layer. Again, there's another base class that you extend. In this case, it's backbone.view. You'll notice that on line 16 there you're passing in an actual template. Backbone's default render method is a no op. So there's no template rendering at all that kind of comes with backbone. So there are quite a few folks that are using Backbone plus react now, which we'll mention again a little bit later. But you basically need to come with your own method of rendering a template. And it comes to backbone. Since it depends on underscore, lots of folks use underscore for the templating system. But it's pretty decoupled and you can kind of swap in whatever you like. So something like twig.js might make sense for Drupal folks if you're already doing twig development or handlebars or kind of whatever else you're familiar with. The other thing that we're doing here in the view class is that we're kind of adding the event binding that's going to happen. So that's how that communication happens between the model and the view. You've got the events that happen on the view side and then the methods that they should call to effect change in the model. I mentioned this before, but here's the actual render method. So you're basically responsible for, in this case, they're updating this dot the element that's from the HTML file. And then when the data comes in from the model, that gets updated via the render method here. And then as an aside that sort of best practices, you return this from the render method so it can be chained later on if you need to do kind of something more fancy. So I sort of mentioned that that presenter bit is a little bit amorphous. And at least for me, that's kind of okay to think about. This is sort of the data picture and the data model that you can use for what's going on. And when it comes down to it, it's sort of what I would call like an observer pattern. So you basically are setting up kind of a key value observation pair. So you've got a particular event that's happening on the view layer after user input that's talking back to the model to change the model data. And then the model that's emits an event that then the view responds to and re-renders itself. So we'll see this again, especially with React. One side effect, again, that we'll talk about in a little bit, is Backbone has another concept called collections. And they're basically just a way of nesting models together. And how you set those up can really affect how this key value event binding react with each other. And you can have some pretty crazy complex situations that develop if you don't set things up cleanly, which is a problem that React actually helps solve. So one thing that's sort of interesting, I think at least, is to take a look at who's actually using this out in the wild. It's probably not a surprise that Backbone kind of has the most sites since it's been around for the longest. But some of my favorites that I found from their Wiki are Ardio. So the entire navigation interface for their web app, and I think the desktop app too, is Backbone. Airbnb is also using Backbone, Foursquare, Bitbucket, WordPress.com. So just like Drupal, WordPress is apparently fans of Backbone. And then the tonight show is starring Jimmy Fallon. The NBC team here in LA actually rolled out a decoupled site that's using Backbone to render all the front end stuff. So it's kind of built by a local company here, which is kind of cool. And it's a project that Lullabot's done a bit of work on as well, which I think is kind of cool. All right, so next up we've got Angular. So Angular is a project that's kind of sponsored by Google. The original idea is they were trying to build a JavaScript framework targeted at designers and sort of non-back-end devs, which to me is fascinating. Because once you start diving into Angular, concepts of dependency injection are everywhere. And that's not something that most designers I've met are super familiar with. I'm not even, I'm learning still. So Google must be filled with really, really smart people. The other thing that they're kind of shooting for is the way HTML and the browser eventually should work if you're on the Angular team and you work at Google. So we'll talk a little bit about what that means in a second. So Angular kind of came around after Backbone, obviously, to sort of help solve some of the pain points that were exposed with this single-page application architecture. One of the things Angular does quite a bit differently than Backbone is that it uses a more declarative programming method, which winds up being the easiest way to think about it, for me at least, is it's template-driven rather than model-driven. So when you look at Angular code, you'll see the templates look quite a bit more complicated. And there's sort of more magic happening at that layer than there is the data modeling layer. The two big kind of key features or pieces of functionality in Angular are data binding and directives. So those are two things. You'll see all of the blog posts and pretty much everybody talking about in one way, shape, or form. So taking a quick look at two-way data binding, this is sort of like the whiz bang feature that gets all the attention on hacker news when it first came out. Basically, you've got this declarative template setup that has Angular directives in it. And then Angular will compile that. And then it's basically constantly watching the data model and live updating the template in real time. So you'll often see in demos like an input text box and then outputting the input of that in text right below it. So it's really handy for sort of complex UI interaction where you want real-time updating. But one of the side effects of that is the way it has to do that dirty checking can lead to some performance issues. They're actually changing that in Angular 2 to a model that's more similar to React. So looking at the sample code from the to-do app, you can see that we're calling add to-do on ngsubmit in the form input. So in your template here, you're basically defining the method that's going to get called when the form input runs. And then down here, we've got an example of a directive highlighted, the ng-repeat. There's this to-dos data object that we're looping over and filtering through. So you've got kind of logic in your templates, if you will. So looking at the actual JavaScript code, here's the method that actually handles adding the to-do. So we're setting up a new to-do. We're grabbing the text and trimming it that came in from the input. We're setting completed to false because it's a new to-do item. And then we're using this store.insert to save it to our to-do store. So if we go up, scroll up in this file, it's the controller. You can see that store.todos is mapped to this global scope object. And that's basically how Angular handles all of its variable management. So there's this scope thing that you'll see dependency injected wherever it's needed. And you can basically do scope.variable name and stick stuff in there that's then available inside of your template. So if I go back to the template, this to-dos that's on line 25 that we're looping over, that's the same as scope.todos. So I mentioned that ng-repeat thing is a directive. In an hour-long session where we're trying to cover three frameworks, talking about directives in great detail would take way too long because it's one of the more complicated parts of Angular. But you can basically think about it as a way of creating a domain-specific language for your HTML document. So I've got a simple example coming up that walks through what exactly that looks like in practice. But basically, you're teaching HTML new tricks. You're inventing your own new tags that have complex UI interaction and behavior. Things like the video element that was in HTML5. If that didn't exist, you could reinvent something like that in Angular, which is pretty cool. Here's the example again of the ng-repeat. So that's one of the core directives that comes with Angular. It's just looping over each data item that gets passed in to the directive. So moving on to the, this is a totally trivial but simple example that kind of illustrates the point and the power of directives. So let's say we want to add share buttons all over our site and the people that are doing the content input aren't necessarily really great at HTML or they don't want to copy and paste like the 75 share buttons that are gonna go on every blog post. So we can use this new custom markup and have a share button tag and pass in the link attribute that we want those share buttons to have. So this markup is pretty simple and pretty easy to understand. You can probably teach people that know HTML how to do something like this. The actual Angular directive looks something like this. So we create a module called share buttons. We register a directive called share buttons. You'll notice there's some name normalization stuff going on where you've got the hyphenated version is the module name and the HTML tag and then the camel case version is the directive name. It's just an Angular convention. Kind of get used to it quick. And then with the directive API in the callback there, you return an object that you can restrict the type of directive you're dealing with. So in this case, restrict E means Angular will only treat this as my custom directive if it's an actual HTML element. So that share button tag is represented by E. There's another option where you can do, you can use attributes as well. So that would be an A. And then you would pass in like the list item. We had an ng repeat here. That's an attribute, not an element. So you can kind of tweak how the directives work, which is sort of handy. Scope is definitely the most complicated part, especially to get in too quickly. But basically that sets up how the data binding is gonna work between your directive and the template. So if you want two-way data binding or one-way data binding or you wanna inherit scope from a parent directive if they're nested, you can do all kinds of fancy and crazy stuff there. In this case, we're just setting up a really simple one-way link variable that gets passed into the template. And then we've got a template URL. So that's the file that's gonna get loaded and actually rendered when Angular compiles our custom directive. So taking a look at what that template file looks like, this is just the full markup that would get output for a Facebook, Twitter, and Google Plus button. So you can see in each case, there's this double curly bracket link thing. And that's what's getting passed in from that original share button markup we had. So when Angular runs through and renders this output, it'll drop whatever link was passed in as an attribute to that custom tag into the output. And then this is sort of what the final product would look like. You could style it however you want or you can have it kind of custom. So you can imagine situations where you can create these directives with really, really simple tags and pass in some data values and then render like a canvas element that has a drawing or some sort of calculator widget or sort of whatever you can come up with, which is pretty powerful. Being able to make your own HTML tags and sort of define your own markup depending on the actual domain you're working on is a really cool thing. And I think it's one of the reasons Angular kind of got super popular. So 2.0, I think it's coming out sometime. I don't know when. Google's sort of not committed to anything necessarily, although I heard somewhere they were planning on releasing an actual Angular 2 app around now. And that was at the conference back in February. So I don't know if it's actually come out or not yet, but it's all the developments happening on GitHub, it's all kind of out in the open. You can kind of follow it if you want.  So all of the Angular 1 apps that had been written would either need to be rewritten or they'd need to sort of figure out how to solve that problem. So for me, it was kind of fun to watch another community wrestle with that kind of stuff because we've done that in the Drupal world for years. But the bottom line is I have no idea when 2.0 is gonna come out and I'm not in touch enough with the Angular community to sort of provide any sort of best guess for that. But looking at some blog posts about how it works, it's pretty radically different. Both Angular React and Ember have all kind of taken the best ideas from each other and are now working towards something that I think in the end is gonna wind up to be somewhat similar, but with wildly different syntax and sort of functionality. One example with Angular 2 is they're using something called AT script. So it's like a typed JavaScript that compiles to regular JavaScript, which is really different. And instead of doing the two-way data binding with dirty scope checking, they're gonna use an architecture similar to Flux, which is what React uses, which we'll talk about in a couple of minutes. Oh, they're not? Cool, I didn't know that, thanks. So who uses Angular? I pulled these from builtwithangular.org, which is linked on sort of their official site. So there are probably tons of other people, but I just grabbed a couple. It sounds like Google's using it internally for double click, but most of their big products aren't using Angular, which to me is a little bit curious. A Drupal site, the weather channel is using Angular, which is kind of cool. It seems like Angular is really caught on, especially in the Drupal community in the last couple years, which I think is kind of neat. Another example is MSNBC. So this was another Lullabot project that's got some Angular code going on. So I'm trying not to be too biased, but Last is my favorite framework, React. So this was originally released in 2013, so it's definitely the newest kit on the block. And it's quite a bit different than the others. And I think the differences are sort of the big strength and the reason why it's the most interesting, at least to me. One key thing that's different is it's really just a template engine. It's really just a view layer. There's code that kind of can come along for the ride that does the rest of what Angular or Backbone can do as well, but really React is just a way to handle templates. So like I mentioned before, there are a lot of people that have kind of wired together Backbone and React to take advantage of the way React handles display and performance, but still using Backbone for data modeling, which I think is kind of neat. So React is built around this concept of components. So you build out components in usually JSX files, and it's got sort of a XML-like syntax that then compiles down to JavaScript. But you can also write just native JavaScript and it works the same. It's just a little bit harder to read, at least in my opinion. So basically you set up the different components of your page and then you can nest those. So in the case of R2Doo app, we've got one component that sort of contains the whole app, and then each to-do item is another component inside of that. And then the second line here, you're basically just grabbing an element from the index file to put the component into. So if we take a look at the render method for this particular React component, it basically has the markup that's gonna get displayed, and then it also is tying together the event binding along with the state from the React component that gets displayed. So calling out kind of how edit functionality works with the React component. On the particular label that's next to the to-do item, we're defining this on double-click thing, and then we're saying call the handle edit method when the item's been double-clicked on. That handle edit method, it basically says set the on edit property and then set the state to be whatever they entered into the edit box. So this state management thing is sort of how React components deal with data. They all sort of manage their own internal state. And then the render method again is using that state value with the edit text that got set in the previous method. So one thing I think that gets conflated a bit when people start reading about React, and it was definitely hard for me to wrap my head around, is how React and Flux interact. React by itself isn't really a framework in the definition I threw up before, but once you tie in a Flux-like library, then you sort of have something that resembles more a framework. Flux itself isn't actually a library. It's more an architectural style. So it's sort of a competitor to MVC rather than a competitor to Angular, I guess. So one way we can kind of visualize it is I mentioned before, you can wind up with a lot of complexity when you're doing this observer pattern if you don't set your models up right. So one of the classic examples was at Facebook. If you get a new chat notification, there's the chat icon in the upper right-hand corner. There's the actual chat message that pops up. There's somebody's online status if they just open their phone to send the message. All of that is representing the same data in terms of how you would do the data modeling on the client side, but you're not gonna set up one view that controls all of those different components. So basically you can have these weird circular dependency cascade effects going on between your models and views and things can get pretty hairy pretty quickly. So what Flux basically does is simplify that and say all of the event and data flow is gonna be unidirectional. So it's only ever gonna flow one way and we'll just re-render everything every time. So this is sort of in contrast to that backbone image we had before where you've got user input that happens with the view layer. That triggers an action. The action then goes and talks to this thing called a dispatcher. The dispatcher then handles updating data in the store. So that was the set state in the to-do example. And then the store triggers the render method and basically everything's re-rendered constantly. And the way React gets away with this is with something called the virtual DOM. So it's not actually re-rendering the full DOM by itself every time, but it's keeping a lighter weight version of the DOM that it's making changes to when it does the re-render. And then in batches, it's writing that out to the actual DOM basically with a diff algorithm. So it's kind of similar to the way game engines work, actually, which sounds like it'd be a performance nightmare, but it turns out that it's actually kind of an okay thing. So who's using React? It actually kind of came out of Instagram, so their whole web interface was actually where React sort of came from. And then I mentioned that Facebook's using it. So if we're worried about performance and sort of how this re-rendering would affect our app, I think I would guess Facebook's probably the biggest site on the web, and they're probably dealing with the most complex data model, at least of any site I've ever worked on. So if it's working well enough for them, it's probably gonna work well enough for anything that I'm dealing with. Another recent project that launched with React, I only know about this one because I worked on it earlier this year, is the SNL 40th anniversary site. So this is another example of decoupled Drupal using React components on the front end. So there's a handful of other frameworks that we could kind of take a look at and compare. They're all on the TodoMVC site, I think. These are probably three of the more popular ones you'll hear about. Ember is really big, especially in the Rails community. It's sort of a more traditional, more full-featured MVC framework. So that's sort of a good one to look at. Marionette is sort of backbone plus some extra convenient stuff. I'm probably gonna misremember this, but I think it came out of or is heavily used by Etsy. And then one of the newer kids on the block is Ampersand.js. That comes from a company called And Yet, and it takes a much more modular approach and uses smaller pieces glued together similar to a backbone, but a little bit different. And then another resource I would really recommend is another Lullabot, John Hannah, wrote this really great blog post on choosing the right JavaScript framework. So he sort of goes through these in more detail and kind of walks through how they all work and why you might wanna pick one over the other. But that's all I've got for slides. We can dig into the actual code examples and sort of see how they work and run in a little more detail. But otherwise, I'd love to sort of open it up for questions and sort of hear what other people are doing with JavaScript frameworks and decoupled Drupal specifically. Cool, thanks. Well, I'll be down by the booth in a little bit if anybody has any questions or wants to talk anymore. Thanks.


Muhammad Ali is a very inspiring figure for me. His last words inspires me. Why do we get sad in our lives? Why do we worry a lot? Believing God puts your trust in him and you're going to be saved.


Hello everyone, welcome to the Jenkins Weekly Infrastructure team meeting. We are the, I don't even know the date, 17th of January 2023. Today we have self-time at the portal, Hervé Le Meur, Stéphane Merle, Mark Wait, Kevin Martins, and Bruno Verrarten. Hello everyone, I hope you are doing well. So, announcement, the weekly process is going well, tags are okay, packaging and Docker image are almost finished, if not already, and checklist in progress. Am I correct, Mark? That is correct. Cool. Do you have other announcements, folks? None for me. None for me. So, we can proceed with the upcoming calendar. Next release is the 23, is that correct? We are 17, no, 24. Next LTS, I have no idea, I haven't looked at the calendar and I'm asking for help. So, I will look it up. I believe it is February, hang on, it's on the calendar. So, February 8th, yes, February 8th, 2.375.3. The release lead is Alex Brandus. Okay, thanks, Mark. Jen Kinn's advisory. Do we have publicly advertise advisory? We don't. So, NA, and next major event will be first them, the first week of February. As far as I can tell, some of us will be there, some of us alas won't be. Sorry, folks. There any comment about calendar? No upcoming event that we forgot. Okay, so let's get started with the work we were able to completely finish. So, add Jen Kinn's Infraio component to CI Jen Kinn Sayo and Infraio. So that experimental work from Gavin is going less experimental and it needs a professional continuous integration. That's why that has been moved from GitHub Action to Jen Kinn's. Is that polite way to say it? Thanks, Serge, for taking care of helping Gavin. One of the main takeaways that Gavin is using a nice tool name Playwright, which is a NPM dependency, which allows you to run tests on a web browser. So that kind of tool exists in one decade, but that one seems to work quite easily, but required some system dependencies that we had to install. So, Gavin is now working with a Docker image that he was able to test and check for that specific project. And we are going to install that tool or ensure that at least the dependencies require dependencies out there. So, any developer could install it if needed without requiring an update to get installed or whatever command. Which is not allowed in production, of course. Thanks, Gavin, for the huge work on that area. GitHub app for plug-in health-coring. So, thanks, Serge, you helped Adrien. As far as I can tell, the GitHub app sounds installed and ready. Now, it's a matter of providing the correct key to the application or in the correct format. Yes, we went through, if the key had to be converted, it seems so. So, I've redeployed the application and we are waiting for the next probe run job, to check if it's working or not. Okay, so I'm quite optimistic that you will help and both of you will finish that. So, nice job, folks. Components has been archived. The name in English is fun, but in French it's way funnier. Trust me. Foufou is like, you're really crazy, crazy. No, I searched for a bar, bar. Or, that's a pure Anglisism. If a French person would have done this one, it would have been Toto, T-O-T-O. However, an improvement on C.I. Genkinsayou. Thanks, Stefan, for adding AWS Virtual Machine based on Windows 2022. So, that's an add-on to the actual Azure Virtual Machine Windows 2022 from last month, which means now, if Azure goes down, first, if Azure goes down, we still have machines in IWS and C.I. Genkinsayou spread a lot between the two clouds and the coast. So, thanks, Stefan. That's also the opportunity to clean up some labels and all configurations. So, nice, thanks. Netify site for Genkinsayou components. So, Hervé, do you confirm that everything is set up for Gevin? Yes, this was an issue he created when it was built by GitHub Action. So now, it's from Genkins. We don't need this issue anymore. Okay. But still, it's off. So, yeah, superseded by Genkins instead of GH. Because we still have the challenge of providing a Netlify token on the Genkins pipeline instead of the GitHub Action. Am I correct? Yes, sorry. We still have the challenge of providing a token to a low Netlify. So, while moving the build to CI and Infra, I've created a GitHub App dedicated to this repository. So, you can extract, you can use a GitHub token in the pipeline. To deploy to Netlify? In Netlify, it's set for Genkinsayou components while it's not the correct term. The core of the issue was he needed a GitHub token. Okay. He made itself, himself, he took care of the Netlify token and for the migration to Genkins, I had to create a NPM token and a GitHub app. So, you could use a GitHub token. A token? Oh, isn't it... There is free token, there are free tokens for this pipeline, a Netlify token, a NPM token and a GitHub token. Okay. And so, which Netlify... So, you already generated a Netlify token in Genkins, man? He made it itself, himself. I didn't touch Netlify. Kevin took care of it. Okay. The issue was... Okay. My question is then... Is it... Is personal Netlify or is it the Netlify of the Genkins infra? I don't know. I don't... You open the issue and check the comment in there. I didn't have to add to... to do anything about Netlify. Okay. I think that somewhere else. I'm not blaming you, you looked on the defensive. I'm not blaming you. I'm asking question because I don't know. And since you migrated the pipeline, you are the person with the most fresh memory. I don't know what to respond to you about Netlify. I think I didn't touch Netlify. Okay. And because Kevin took care of everything around Netlify. And his previous issue was requesting a GitHub token. Okay. So we have to check as a team what is the status of the Netlify deployment website? Is it only temporary? Is it only for a preview website? Because if we have a production website, we should use the Genkins infra Netlify that is used for other websites. Okay. But it's closed and it's okay. We had an account. Next issue is an account related issue that has been solved from my point of view. Can you please read the title? So I remember it was translated on Google and it was something I've been cooked by the anti-spam system but in Russian. Yeah. So and I've replaced the title on GitHub now because I would never remember that translation. I don't receive an email with a password. I'm trying to register with another email. So I put it, pasted in what Google Translate says. Okay. So I've checked, but that was word because the person opened a twin issue. So I triggered a password reset and I checked that the SMTP relay was okay based on the logs in the printed. Let's say the Gmail SMTP received the email. So if that person doesn't see an email on their mailbox, I can do anything for them. So I told them to look their spam and we'll see. That one was word because using 200 related emails with 200 related account name. Well, anyway, HSTS bloc use of trustees and third CI, the GenSec team, security team was blocked when using Chrome to reach both controllers because for both of them we use private channels. These are SSH tunnel or VPN private and both private VPN. Sorry. And with Chrome since publicly the Jenkins.io domain says that all the sub domain require HSTS which tell the browser to always switch to HTTPS with a valid certificate and don't accept manually signed certificates untrusted certificate or HTTP. The thing is that in the case of these two machines they are private and they are machines so we use let's encrypt with HTTP challenge for the virtual machine usually and in that case since they are private machine they cannot validate the let's encrypt challenge for a new certificate because they are private and they are not exposed. So they cannot expose the challenge to let's encrypt which mean we need to use a DNS method where instead of exposing a challenge for an HTTP page or an HTTPS certificates instead the let's encrypt system create a DNS record with the challenge so you don't have to expose your web server. We never had time to spend on this so I generated manually the DNS or using the DNS technique a first set of certificate to unblock them instead of hold backing the HSTS because HSTS takes time it has a long TTL it will have taken 3 months so instead we generated a valid certificate and it unblocked them immediately and so that issue links to another issue where now we need to apply an automatic renew system one for trusted CI and one for third CI so these are two subsequent issues now that we have valid certificate for both of them so that's why the issue is now closed because they have a valid certificate and worst case we will have to run the third bot renew command manually in two months on the help of these machines if we don't succeed in automatic renew Maven 387 is now generally available for developer and CI Jenkins.io so thanks everyone involved on that especially Stefan but not only Airvielse did this work we haven't heard about any issue with that Maven version and it's all container machines and setups and thanks Mark for adding it to the CPUZ machine we had an LTS release last week and all of our instances were updated at least the one using LTS version so thanks everyone on that part as well I wasn't available that day so thanks Stefan and Airvielse for backing me up on that area that prove that the team is not dependent on me on these areas which is a good thing which is healthy any question about the job being closed or can we switch to the to the work in progress I don't want to cover the close as not planned because these are free account related issues just a personal thanks for the HSTS thing it's a lot easier to deal with trusted CI insert CI now that they've got SSL certificates thank you I know that's more work still to be done but thank you very much my dealings with them are simpler now thanks to that Happy that everyone was able to simplify the life of other teams then the work in progress issue we spend a lot of time as a team in mob programming with the the automatic update of that certificate only for trusted CI Junkinsayo so that machine doesn't run on Azure it runs on AWS and the DNS records are stored in Azure which mean we had to put a structure a permission structure to have a technical account allow to create and remove DNS record only for something trusted CI Junkinsayo for the DNS challenge and nowhere else we don't want that technical user being able to change record for let's say pkg.junkinsayo so manually if you do everything manually that works very well we validated that as a first step but then automating that permission model with Terraform and the Azure API was quite an issue thanks a lot team for pointing us to the element that we missed or misunderstood and thanks Stefa Nervé for revert the king with me and asking a lot of good questions which allowed us to succeed on that area that we spend one day and a half on that topic we learned a lot on the process so now we have to apply to Poupet to change the let's encrypt system to use DNS so we need to play around with Poupet but the good news is that since we manifested ourselves on the Poupet let's encrypt module saying we are interested in the support for DNS Azure last week they already merged the pull request that was selling for months and they released the whole thing so we should be able to not deal with certes but binary at all and do it all in pure Poupet so the chain should be minimalistic for us so almost there and thanks everyone involved on that part because that wasn't easy thanks for the moral support folks because I was on the verge of throwing my laptop through the window yesterday so a new issue taken by Stefan about playwright tool that we mentioned earlier so Stefan is working on that we know it works on the context of Docker image now the goal is to move it to packer images the reason is because we want to put every builds in the near future on only the packer image container image so we don't we need to install it and transplant the changes we did on the webbuilders we had a new issue from gave in about sourcing index html for getjunkinsayo we didn't have time to work on that we should be able to do it next week reminder getjunkinsayo is a mirror system hosted on Kubernetes the Kubernetes system has two web services the first one is a mirror hosting and the other one is a mirror redirector the mirror hosting system is a web server serving files from an Azure centralized bucket and the redirector redirects when you have a request it goes through the redirector and if the redirectors finds the file it get a hash and from the hash on its database it decide to redirect one of the closed mirror or to serve the file as a fallback in the case of the index html file we need a need that's the root file on the buckets that one was moved manually on the buckets the request from gave in is can we source it somewhere so it can be changed the goal in this case is to add the web components so anyone going to getjunkinsayo or one of the mirrors at the roots if the mirror the index html to have a proper web page so i propose so sorry i am i'm a bit tired so i forgot to reschedule to reschedule the task so that task need to be finished during the upcoming iteration is there any voice against that choice plus one for me okay play rights it's not top priority but it sounds like it's almost done so is it okay to finish it during next iteration stéphane oh yes about the getjunkinsayo i'm not able to evaluate the complexity of that task at first glance it should be easy getting the index html and putting it somewhere but we might need a process to to a deploy it from the source so that might be a task that could spawn across different milestones i propose that we keep it for next one and anyone volunteering can assign themselves if they want to start working on that and i might help or do it by default is that okay for everyone yes yeah although i think it also be okay to say no we're gonna wait one more milestone i don't think that this one will harm gavin if it has to wait a little bit okay i have a personal matter that there is that giving ask us an irc three weeks ago oh almost every day i ask him either publicly or privately can you open an issue for that so i would feel bad if we don't take it now even if it's not prior but that's only personal so if it bother anyone else i don't mind taking it because of that well and that's a good that back story is a good reason to do it that makes sense okay we had a new issue and that one i don't mind waiting for this one i will add a message because last time i had an exchange with that user the exchange was quite complicated so i will add a message and a risk deal for later that's related to the way sentos redacted mirrors are managed and basically i don't want to provide any more support for that person because that's how they handle mirror management on their own infrastructure so either they use a standard mirror or not my problem besides they are using really old version of Jenkins so i would say maybe you should upgrade one of the three older LTS at least so no objection if i move it to infratim next and i get a message yeah no objection at all we don't we don't know anyone's support for their private infrastructure yeah i still plan to just throw an eye and decide if it's an issue there might be an issue on our own so i just want a sanity check from someone of us but when time will come next issue bump terraform module for AWS EKS so that one was moved on that milestone back because we weren't able to so sorry stephan discovered that there was an older cluster consequence of that issue that had to be cleaned up to avoid spending too much credits on AWS and we have a solution for the let's encrypt setup on that one now that we're able to do it for trusted CI there is no we can use the same method on the permission level on Azure to let that cluster to create record on the the correct DNS so that one we should be able to solve it quickly besides based on what mark extracted from the G frog data the artifact caching proxy might be a bit more important than what we say during the past two months is so we need that cluster fixed that's why i propose that we add that one to the next milestone but first we finish trusted CI and then we update that one exclude non numeric plugin version from update center thanks mark for forwarding the discussion to the correct people i saw there has been a pull request directly on the date center i propose that we remove any milestone from this one because we don't have any more expected action and the course of the course of action will be that pull request being accepted or merge an update center so it's outside our scope is that okay for everyone agreed and the pattern i specifically asked daniel beck a while ago i have merged permission to the update center repository but i asked him specifically do you prefer that i exercise that permission or not and his preference is that i not exercise that permission so he would rather be the one who merges all changes to update center and i like that that is perfect he maintains it very well so even though i have permission and others should take the same advice i think rather than any of the rest of us doing any merge to update center it's much better that we let daniel decide when he's ready to merge it perfect and given the title of the pull request once if that pair is merged then that will close the issue on a desk back because it's the same so we don't have to worry about cleaning up the issue thanks mark renew the sign or certificate for Jenkins so for that one we need to work on that mark but with neither of us at time Olivier is okay to help us to share information so i propose we keep it that one up because when that will be required realize repo Jenkins CI or mission so we have a record here mark it is seventh biggest consumer as far as we can tell for December on that repository that is another proof that mark is our testing person for Jenkins and i'm sure Jenkins will collapse if mark homelab wasn't able to run so there are lots of things that we get from that that data analysis and we'll take action on those right we've got several places that we need to we need to improve including more caching on my homelab and more caching on ci.jenkins.io and and we've got some consumers that we're reasonably confident are not contributing enough to the Jenkins project to justify the bandwidth they're using so we will take active measures to ask them to stop and we may even take active measures to prevent their access rather than just being nice and asking them to stop so one of the takeaways we will we will have to confirm i will have a message on the issue after the g frog meeting tomorrow we have two meetings one stefa nervian i and mark about mark will show us the data and how to treat it and then mark and i will have a meeting with g frog to ask them to send as if possible weekly reports to discuss about what mark discovers but one of the takeaways is that we we need to confirm by with the real IPs but we have let's say a gut feeling that our infrastructure is still consuming a lot so the work that everybody did on the caching proxy might have way more impact than what than what we expect which mean the work on private gates and public gates the two new cluster with the network solving is top priority in order to put the ACP back the rest will be exchanges with g frog to ask if we can ban some IPs that marks that mark discover where high consumers i don't give name right now and eventually if g frog doesn't have any objection the world authentication of hippo one or cash or mirrors and held up in high availability modes that are closely related that's the peak might be deprioritized with the other steps first because we cannot treat all the task at the same time we have to say cancelize and the ACP might make more sense for now right yeah the data the data the data bandwidth or the bandwidth data certainly supports the artifact caching proxy being a crucial part of this of this effort because a big chunk of the actual data transfer that I saw is really right from our own releases repository so it's delivering our own bits to ourselves they prioritize and less so thanks a lot mark for that bunch of information that help us prioritizing and choosing carefully our tasks so that issue keeps being on the milestone Hervé your turn private gates can you give us a status not a lot of progress since last week I've got the certificate ready get some certificate ready I'm currently trying to run the Kubernetes management job on this cluster so yes experimentation about this job if and when this job is running correctly we will be able to back up the current phrase I think in Salyo data and move it to this new cluster then we will be able to to move the DNS record to point to this new cluster nice that's quite clear that's that's the same battle plan thanks migrate dns upgrade and benefit and profit after cleaning up thanks a lot that's cool don't forget the pull request to add the the outbound ip to Kubernetes management yeah I'd like to know how I can identify it also but yeah can you start by persisting it and then you will think about automating otherwise what happened is that you will spend three days automating a fire no never mmh we all do you're not alone on that war mirror start report wrong results didn't make any progress on it no I didn't have time I propose to move it back to infratima suggested last week I was a trigger because I told I was going to help you and later you and I had time so I propose to remove Assini uh we might spend some time on it but I propose let's see later I would even yeah move it in yeah definitely not for next week blocked from creating a new account by the anti spam system might be an issue to close oh no they tried yeah okay did it again came back to it and we didn't okay and it's on me I forgot it was on my side and they didn't checked so definitely to be done next milestone all inbound agent publish as latest hey that one entrusted cian kin sayo it's been numerous months where and a set of old tags on the agent images inbound agent and ssh agent these old tags are rebuilding uh sometimes daily sometimes weekly sometimes monthly we weren't able to find why and we thought it was a bug or a word behavior but today we were able to find one of the causes maybe there might be other causes but one which is plausible um let me open that commit that we push on ssh agent can anyone see what does commit does it's important so I prefer sharing the information that pipeline instruction says for that pipeline so for that specific tags all the tags that add a Jenkins file with that instruction in the past once they have been parsed so one build trigger read the Jenkins file and say oh I need an additional trigger at the pipeline level not at multi branch level that's the trick part and the pipeline say oh I will try to pull the SCM and if I see any change I will rebuild myself that mechanism used to be there before it's from an era before webbooks before multi branch pipelines and in that era we were using a policy and because we were using branches but first of all policy does not make any sense for a tag second we don't need a policy we have a multi branch pipeline which takes care of how at least once a day let me scan the whole repository and decide if a build should be triggered or not and all the analysis that mark daniel team hi and other people did was to check the multi branch set up with build strategy that say if a tag is older than free day then consider it should not be built only discovered which work well we demonstrated each time you manually scan it work as expected and as configured so that set up is okay but now the challenge is these old tags we don't want to change all the old tags from the past to remove that from these old tags but these tags already have the polling enables so I've asked the question on different location and I'm waiting for answers if there are a way to disable polling SCM at the multi branch level that will be easy otherwise I suspect I should be able to run a magical grep command to remove from the config XML the directive polling SCM from the triggers that will be really ugly but I expect it to work another solution will be to recreate the multi branch from scratch that they would check all the tags and say oh these tags are old I won't try to build them meaning I won't parse the Jenkins file and I won't enable polling SCM so that one can be tested immediately on docker agent and docker inbound agents for SSAT agents we will have to wait a few days because yeah it was done five hours ago so of course the previous tag release yesterday of that image will have the polling and then we will go back on that problem again so issue has been commented and we definitely need to work on that we worked the three of us on that one for knowledge sharing yeah James ask you you will see later if we need to keep the old job in CEVI I know it has been back up backed up already and I don't mind removing it because most of these tags have a build build rotation policy so Jenkins already deleted the builds the last succeeded builds and that's why the policy and keep rebuilding because it lost the previous one so it's a before nothing so of course it's from no known state to code it's a oh new code I have to build but it fails of course so that's one of that's a reason explaining that behavior we might have other surprise like with it so creating new multi branch job from scratch will do so and show that we have a clean state to start with any question so we keep working on that one finally account Jenkins a admin access for Stefan I did not have time to check on that one Stefan are you mad if I move that one to infras team next to be sure that we walk because clearly we had too much issues on our plates for that milestone so I'm not sure we'll have time on that one that will cost you a coffee when you see me but that's okay okay so let me click on that one it's it's unacceptable cost thanks for your understanding okay on the backlog I don't think we had so I'm checking for the new issues first do we have new issues that we don't have either on the backlog or that one we just removed no new issues that are not on the current milestone on the backlog right so now I'm looking at the backlog just one time if you see an issue that is catching your height don't hesitate I see a new message on that one yeah okay so I propose okay I don't see other thing that we absolutely want to proceed time private okay I test back up maybe but okay that one can be ignored what cluster backup yep yep of course that's part of what you're doing it's not yeah it's not really about migrating this cluster it's more about having a backup of our cluster okay do you want to start working on that one no at least thinking working against issue no maybe not okay no problem I don't think we should plan for more and see if we can terminate our other issues if we are able then we will take on the backlog just a few things for next week we will have to start thinking about the Ubuntu 22.04 campaign because we have a lot of machine on 18.04 and 20.04 20.04 is okay because it's five years support for LTS but we have a lot of machine on 18 we can upgrade this machine in place we can migrate these machines to new machines with updates or we can change architectures I don't mind but we have to think about that and as every fund in December we are now able in Azure to get Ubuntu 22 which mean we should be able to update the agent virtual machine and container templates to Ubuntu 22 so it's not priority for this milestone but we have the go everything is green for going that direction and we have to do it before April as well obviously because that mean all machine in 18.04 won't be supported anymore starting May a bit more pressing though Kubernetes update we will have to schedule the next 1.24 infirm. not mistaken because we risk to have one of our providers stopping to support that version if it's okay for everyone I mention it now so I plant the seed and next week we can start scheduling based on where we are with the rest of the priorities is that okay for everyone yes cool those were the two main topics I don't have any more topics to either discuss writes 2k 8s yep prepare to create a new public cluster in bear with Stefan totally I propose that we second salise so that your mind is not overloaded is that okay for you if you are waiting for builds to take one hour to finish and your mind is free I propose that you start thinking about the battle plan so then you prepare on one issue that you can exchange on a written form with Stefan before you start discussing the two of you of what need to be done is that okay for you so first a written battle plan on the issue that is okay and as soon as you start synchronizing with Stefan then you can't move to the current milestone the related issue is that okay for you that gives you enough space to think and start thinking and start preparing the battle plan without forcing you to do it at all costs so you can focus on private case does that make sense for you in the way you used to work yes cool okay I don't have any other topics do you have some or can we close so again thanks everyone for the huge work show our whole doing and see you next week so I'm now stopping the recording


Today I'm here to talk about validating particles at the ETH one, and hopefully at the ETH two level using AI agents. So we focused on the AI technique known as reinforcement learning, and we may be wondering why are we looking at this, how does it fit, why are we looking at machine learning, and I'm going to explain all of this and talk you through why we decided to go that uprope. So in the past we've looked at something called selfish mining, which is a strategy and attack on the Ethereum 1.x vertical, where essentially you're trying to attack the network, manipulate the difficulty calculation, and you do this by essentially keeping a private chain that it's ahead. So essentially you don't send your blocks as soon as you receive them, but you will hold them and send them strategically as you keep getting heads from other miners. And so the idea is that you're going to make them waste your time by looking at the blocks that you just sent, and you always keep ahead so that you ensure that you're winning. And so this is a problem that's been studied in different papers, and there's different algorithms that you can follow. As I described before, you can hold a number of blocks, and whenever you have a lead, someone sends you a block that's increasing the general height, you publish a block that you've already mined for that height, and essentially always keeping ahead so that you ensure that you can maximize your reward that you're getting, because when you send them, you'll either get a main chain reward or an on-call reward, which are almost essentially the same. And so I work with the Pegasus team at Consensus, and we focus on research, so we've developed a framework to build simulations for different types of protocols. So we've looked at proof-of-work, proof-of-stake, and so we decided it would be pretty interesting to implement a selfish attack and look at the data that would return. So this is a bit of a code that it looks like. We actually took the calculation of the difficulty. We're looking at how it's done, and if you're ever late, just to get a sense of what the rewards are. And so depending on what you're doing, as I mentioned before, you can get a reward from mining a block, which is right now to ETH, with the transactions included. And if you include any of those that you receive. So your reward may vary to 175, 125, or two. And since the transactional fees that you receive is quite small sometimes, we decided to not include this in our own simulation. And so we decided to build a test case where you only have a network of nine miners who are always liners, and then you have an honest miner that is selfishly mining. And so what happens is that there's some variation, and we decided not to include the rewards of transactions, and some of the results are this. So essentially what we found is that once you have 25% of the hashing power of the network, you can start attacking and doing selfish mining, and this is more beneficial for you. Because this manipulates the difficulty calculation, and it increases your reward. So the more hashing power you have, the better for you. And so we wanted to test how would this prove if we're giving it to a reinforcement learning agent. And since we have actual data to back it up, we could compare the results and see what we would get with it. And so that's why you might be wondering why are we doing reinforcement learning when we're focused on watching it. But just to give you a sense of what reinforcement learning is, in case you haven't heard it, is the concept where you have a piece of software, it's an agent that's interacting with an environment. One of the most famous ones is DeepMind, where they implemented a deep queue network. And essentially it's interacting with different itinerary games and it learns how to solve it. So we thought, okay, we can take this concept and implement it into different protocols and just see what happens. And just to make a sense of why we chose that too is because it's an automated system that essentially has proven to beat humans and many other algorithms and strategies. So it just made sense to go that route. So how does it work? You have this loop where you have an agent that is collecting observations in the environment. So it's partial information on the current state. And then you have actions that you can take which will return a reward based on those actions. And so the process continues in some loop where you just keep iterating to different actions in different states. And so what are the different observations that you can get in terms of our protocol? We're looking at the number of blocks mined, the number of blocks we see, the percentage of on-calls, the different things that you're actually doing in the network and feeding that information into the agent so they can learn the impact of their actions. And so to reduce complexity we didn't want to be too broad in terms of what a miner can do. So we decided that you could only mine up to 10 blocks hidden in your own private chain. And you should be able to publish up to three blocks in a row so you could publish one, two, or three based on whatever strategy seems the best. And then you would get some reward based on that. And so I think the most complicated part of setting this up is trying to understand what is the best reward you can give to an agent because you want to give a reward such that it understands what's the optimal strategy to pick and also how to discover selfish mining without necessarily implementing the entire set of actions that it needs to do. And so what we decided to do in our case is to incentivize staying ahead, just mining as many blocks as possible faster than the other miners. But there's other strategies that you can take. We tried to test it a few to see what was the most optimal result but eventually just decided that using the metric of staying ahead was probably the best. And so going back to what this loop looks like, we have this agent that has been implemented in Python and it's calling code that is implemented in Java that works in the later. And it's essentially going through this loop over and over again, selecting actions and then collecting the information that it gets from the simulation. And so there's a number of different algorithms that you can use. As I mentioned before, like one of the most famous is the two networks. But you can also model free Q learning algorithms, policy grading based algorithms. They all play a different set of statistics and modeling and use market chains as the back of how they calculate different states and awards. And so we decided that probably the simplest one to go for was Q learning. And so I'm just going to explain a little bit what Q learning is essentially. So we use this Python framework. It's developed by OpenAI and it allows you to create your own environment and interface that you can use. And so we developed that calling the Java code and then we tested for many different scenarios. So we wanted to see what was the best strategy when you have 10% of the hashing power, 40% and 60%. And then we set some hyperparameters that are related to these Q tables and how you calculate the reward. So you look at the learning rate, which essentially is just telling you how much you value and new information that you receive versus information that you've recorded in the past. And we also have a discount factor, which also pertains to the reward. You want to look at the current reward that you get for an action plus the reward for making an X action because you're trying to discover what is the most optimal path. And so it's something that you have to tune in terms of how do you value a future reward because you could think, okay, I can mine a block right now, just send it as soon as possible and I get a reward of two. But if I decide to wait a couple of blocks ahead to publish it, there's time that goes by and so money might cost less because of things like inflation. Or you could just think, well, I mined four blocks in one minute, that's negligible, it's worth the same. So based on what you value most, you will tune this parameter. And then finally you have epsilon, which is just related to randomness. How many choices do you do that are random versus how many choices do you do that you're maximizing your strategy by making the biggest reward. And so there's two states, as I mentioned before. The first one is driven by the epsilon, where you're just trying to make random choices of the different actions you can take and recording the results that you get in this table. And then this will allow you to generate different values so that you can reach an optimal and deciding the best action at each state that you reach. And so there's some theory behind this that you use something called Bellman equations and it just says that basically by maximizing at each state that you find yourself, what is the optimal action, you will maximize for the entirety of your decision process. And so you're just looking as I mentioned before at the reward and then you're also adding the value of the future reward of the next state in action but with a disemptive factor. So that's what I was talking about before. That's zero point in value. And so just to make this a little bit more concrete because it can be very extract, we can think about this Q matrix being something like this where you have four different actions so you can hold your block and not publish it, just keep it in the private chain. You can publish one block, two or three, as I mentioned before, this is a different set of actions you can take. And then you can hold one, two, three, four and more but for simplicity in this example it puts it three. And so when you start to initialize it and all the values are at zero and as you keep going randomly through the actions you will change the values. So let's just walk through one of these examples. So let's say that you start in action you decide to hold and so you're going to calculate what happens when you hold and then maybe the next step you want to publish it. So that will generate a reward, let's say two even. And so you continue doing this through random paths until you've generated enough values, this is like random values, this is not the actual ones but just to give you an idea of what happens. And then after a certain point what you start doing is that okay I find myself at a state holding two so I have two blocks, what is my next best action? Should I publish them or should I hold them? And so you take the maximum value for each action and that's how you can find an optimal path. And so as I mentioned briefly there's the Markov chain process that backs all of this information and ensures that this equation holds true and it's just an idea so that you can also visualize this in a different way. You can have a chain where you're mining and then you can start your own private chain and keep switching between one and the other and then essentially because of the properties of a Markov chain you reach a steady state and that's why this table is useful because you can assume that after a certain number of steps you've reached the same if you were auto-affinity and so that's how you can maximize the values. And so there's a lot of potential challenges when you get this approach because you need to understand what's the best data that you need to feed, you need to test for all these hyperparameters and keep changing them until you can find something optimal. And it's also a consideration of how do you build this what are the behaviors that you want to incentivize how do you define something that's Byzantine and so on. There's a lot of things that you can find challenges with. And so just to go back to our example of proof of work we took observations, we were just tracking what is the number of blocks that you've published in the public chain, private chain, keeping track of on-calls and forks and then letting it know so that it can make the best actions. And so we read this similar to what we ran in the self-contained simulation, nine honest minors and then one Byzantine minor that is acting through the reinforcement learning agent. So we measured it at 10, 40 and 60 percent hashing power and we collected the results. And so as I mentioned before we used the simulator and drama, we used this framework, OpenAI and Python some of the typical machine learning libraries and then we ran all of these tests in AWS so that we could test and collect all the data. And so this is just again an overview of what we were doing at the different hashing rates. We collected the reward over a total of one hour of mining, we also collected it over the total of about 10,000 episodes or a simulation, so essentially I'll let my agent run for an hour or until it has mined a thousand blocks and record that behavior. And I run it for 10,000 episodes while you're changing the actions that you select. And we also measured for three different strategies. We just wanted to make sure that we were performing better than if you were just mining honestly or if that you were mining just randomly and we wanted to prepare the results. So what are the results look like? We found that when you ran it for even just like a thousand episodes the agent that was incentivized to hold his chain at 10% is actually performing worse because the reality is that at 10% you're not you're never going to be ahead of all the other miners you don't have enough power. So you're just end up losing more. However at 40% you already start getting an increasing your reward after maybe like 500 episodes you've already seen a little bit of an improvement in terms of what you're doing and then at 60% it's just like taking over the network. So the results, we summarized that here and we're also looking at this metric which is the ratio of blocks to mine. So this is how many blocks did you mine versus the number of total blocks mine. So what you see here is that for 10% you're doing worse than if you were just being honest at 40% though you start improving. We set two different reward systems just to test things and for one of them to be better at 40% while acting selfishly so he's getting like almost 50% of the blocks which is a slight increase and at 60% it's just mining all of the blocks because you get such a huge advantage from selfish mining that essentially you're publishing most of the blocks because nobody can catch up to you and what happens is that you're also decreasing the difficulty of blocks so you can mine faster with the same hashing rate. And so why are some of the key takeaways of all of this is that at 60% you're doing a lot of damage but we all know that even at that percent that you should be able to move a lot of damage. At 40% you start achieving some interesting results but one of our hopes was that by implementing this automated way we would find perhaps other flaws something different than selfish mining a different strategy and then at 10% it's just doing very poorly. And so like I mentioned before at 60% you're producing over 90% of the blocks but what's interesting is that you're not actually making any more money if anything you'll be making around the same amount of Ethereum. And so you're just attacking the difficulty but it's not necessarily beneficial. And we couldn't find a better strategy than selfish mining but it's interesting because then what you can do is use these types of systems to test different protocols and understand if there's something going on so perhaps you can see it as a signal detection system that there's something wrong with the protocol because if you don't follow the normal strategy you're doing better. And so this can help you understand and evaluate in different ways what's happening and you shouldn't just think that this is a wholesome solution but rather a good indicator of what can happen. And so this is an open source project it's all on GitHub so if anyone feels like they want to test their own protocols you're welcome to do it and yeah, that's it.