Oh. Hello. Hello. Just seeing how close I need to stand to the microphone. So my name is, I guess I'll just get started. My name is Robert Caracus. I work in Australia at a web agency called Previous Next. And yeah, I'm here to talk about Drupal generated markup, not being your friend. How you should use a style guide. We've been using a style guide for the past, I don't know, two years now at Previous Next and it's been, you know, it's really up to my theming skills, I guess. I guess I would say it's just, it's been interesting using it. I want to talk a little bit. I heard Dries talk, my last session I did in Drupal Con, Austin I talked about Angular and all the stuff that Dries is talking about in his keynote about how you know, like how these front end frameworks are kind of happening and stuff. So I talked about creating a Drupal theme in Angular and I thought it was really cool. And I still think a lot of these client side frameworks are really cool, but Drupal 8's coming out and there's going to be some tools that, you know, we can use in Drupal 8 on the front end and I thought it would be good to put forward a presentation that talked, went into a little bit how we're doing theming now how, you know, what, how we've done theming in the past, how we're doing theming now and how in the future, you know, front end might change a little bit, especially relating to the release of Drupal 8. The last speakers slides were really nice. Mine are kind of figuring, me figuring out, reveal JS and some code examples and stuff. So I'm a front end developer, not a designer. But yeah. So let me get started.  That's good. I don't have to explain things too much then. The last one, everybody was back end developer. So SAS, templating engines, may handle bars, things people are familiar with or semi familiar with. So that's good. I'm going to have some of that in my presentation. How many people here work in like an agency? Say an agency versus internal, a lot of people. So I'll have a lot in common with people. This is like a common design that you might get. I don't know if people are doing design in different ways. A lot of times we get a Photoshop file with something that looks like this. Really common, something common that you might put into Drupal. You know, you have an image, you have a title, might have a link. And a back end developer might think, okay, or anybody, if you're familiar with Drupal site building, you might think, this is a perfect case for something like views. A list of nodes that link off to a detail page, classic case for something like views as your site building choice. And the quickest way to style something from views, say a back end developer is in charge of configuring some advanced configuration for that view and hands it off to you, being the front end developer of the theme or maybe you have to do it yourself. And you look at the markup and the markup looks something like this. And I made it small because it really doesn't matter what is there, just you can't read it and it's kind of terrible. But somewhere in there, you're going to find these classes that you can grab on to and theme and make what is given to you kind of look pretty. Yeah, they're in there somewhere. I've kind of mocked it up a little bit using position absolute. This is the class you're going to find. And I just, I don't even know if that's the right one I looked at it. I think so. It doesn't really matter. But basically Drupal classes, they give you this prefix that you have to, that's usually the name of the particular site building module of choice and then followed by some sort of ID that you might identify by. And that would be the container, right? So views, view, home page grid, say that they wanted the view on the home page. And then you might have an item. These are the things that are important to you.  So this might be the, if depending on the views plugin that's being used, depending on a couple of different things, this might be the class that you choose to attach your grid base styling to. Yeah. So this is, I guess, the code that you might use. I mean this is SAS, so it's kind of nested. Yeah. You might start off by just putting some code into styles.scss. You might just have one file for all of your different components around the site. So you have a max width and you have a width 33%. Kind of makes sense with what we've shown. Obviously there's going to be a whole other set of slew of properties, CSS properties that you're going to have to attach there. But yeah, I'm not going to put those in for example sake. This is just as an example. Yeah. So you're able to pretty quickly get that view up and theme the way that design looks just by attaching some styling to some Drupal classes spit out by views. But then say further down the line, you have to add other implementations of this grid in different sections of the site. You might have one on the home page that we discussed, but there also might be another instance on a landing page where you need to do some promotional items. And there might be some related content on the bottom of a node page like an article page that shows a list of articles in a very similar fashion, in a grid fashion as the design I showed before. And the site builders and product owner and client might have special requirements. So the views is fine for the home page, but there's going to be all sorts of other site build. I mean, on every Drupal site to me it seems like people are using a million different site building tools. We don't just settle on one based on what you need, based on the content editing experience. For instance, the promo might be done in paragraphs. That way you can give the client or the site builder a way to add images and things curate content on a per node basis. The related content might need to use entity reference because you might want to have database relationships between the articles and other articles and show them on the bottom of the page. And then the promo might need to be purple or something. Anyway, the specifics around this is not important. Just kind of say that we're using a lot of different site building and there might be different sections of the same design on other parts of the website. So using paragraphs just to go through it quickly, you might have a container, whatever paragraphs gives you, paragraphs container and a paragraph item. And you're going to do a very similar thing that you did in the view. You're going to style the view and the paragraphs in a very similar way to the view. Obviously there's a little bit of variation here. But the point here I'm trying to make is that even though these are using very simple properties and two different implementations of this grid using different site building tools. But this CSS, even though it's listed together on the same page here for demonstrations purpose, could quite probably be in very different parts of your style sheet.  But oftentimes, you don't even know. You'll style these things again, depending maybe a project manager a week later says, style this bit. You'll go and style it all over, even though you've already done the styling, something very similar in the past. So, yeah, I mean this stuff might happen, might be in a lot of different places, let's see if I can zoom down here. And here's your entity reference. You might have another thing. And this happens, I've looked at some old projects and you've had this very same thing. This is what was happening two years ago, three years ago. And here's an example. Excuse, I'm not pointing out that he just made the last commit, but well, for will. This is just an example of a project. This is when we, but maybe two or three years ago when we were putting all the styles in the same style sheet, basically everything in styles.scss. Because that was the easiest way to kind of do it. With this sort of classes and this sort of markup, how are you going to organize it in any other way? How are you going to make partials? What are you going to make partials based on paragraphs or based on views? They really have nothing to do with what the actual design looks like and the different components that the design might look like. The different patterns that make up the design. This is hard to see, but it really doesn't matter. I just noticed this, but this is on another project a couple years later where we decided that we wanted, because the one style sheet's very hard to maintain, so we decided we wanted to break things up in a more organized way. But in my opinion, it kind of just made things harder. I can't really see this, but it says node article. This is the article featured and the classes inside of bottom lines, the classes inside of the particular partial have nothing to do with the actual name of the partial itself. So by looking at the partial tells you nothing about what's inside of it, which to me makes things hard to organize. So with these problems, and there's more problems to do with default triple markup, but I have to move through this. Industry as a whole, front end as a whole, I have come up with a kind of a better solution. And there's all solutions, but I think the settle on a solution in the last couple years has been what's called design components, where you use, and it's very similar to Drupal modules, if you think, where you come up with a name, any name for a component, and pretty great. It really doesn't matter. There are some rules that have to do with the naming, but in the end of the day, as long as it's a unique name that describes the pattern, it doesn't matter. You describe, it's kind of like a Drupal module, you know? You have a name space, and then you have subcomponents that go along things. You're grouping things together, like in a Drupal module, you might have the name of the module in the beginning, and then as a hook or something, and then you might have some sub, some functions or methods inside of there. So, and that's what, you can follow the same pattern in, when you create what industries call design components. It's important that the naming of these components don't relate to Drupal configuration. So you don't want anything, any naming in your, I mean, this is all opinion, by the way, so don't take anything the harder, you can come talk to me after, but this is how I kind of do things. The naming of your design components, or your name space, to kind of conflict with Drupal and confuse you. You also probably don't want it to have anything to do with the actual content, because you might have different content flowing in and out of these design components. For example, if you call this article grid and you wanted to have pages or some other content type flow through here, then that could totally cramp your style when it comes to, you know, styling it in the future. Yeah, and you know, sub, you can have the item, but you all, you can have some components, tiny sub components too, like maybe the title is slightly different than something else. So what does the CSS for this looks like? In my opinion, a lot cleaner. You have some namespace classes. I'm using the actual name of the component itself, pretty grid, just made it up, by the way. And you're actually putting it into a partial case, so you can see this partial is named the same as the class. So all of a sudden it becomes a lot easier when you're looking around your markup and you're looking around your website. If you see a component named pretty grid, then you know that you can go and find a pretty grid in a nice partial somewhere, and everything to do with pretty grid is inside of that partial. And you can see it, I'm using the name of the component as a container, but you can also just as easily say underscore, underscore container. And these underscores and dashes and stuff, this is using a methodology called BEM, and I'm not going to get into the rules. It's a little bit complex to understand. John Albin, my co-worker, has gone in through over the last couple of his sessions and gone into kind of the nitty gritty of BEM and talking about a lot of the syntax, but I wanted to bring this kind of methodology up again, because I think has to do with some more stuff I want to get into. Anyway, so here's pretty grid item. You have your width, your grid item widths, and you might add a larger font size for the title, just example. But with your CSS, you really, in my opinion, you really want to have example markup. First off, for two reasons. First off, prototyping. I mean, in Drupal, maybe five, six years ago, when I was seven, eight years ago, when I was first beginning, I didn't even know how to write markup. Drupal did everything for me. I made views and it sped up the markup. And I was theming. I was doing a lot of CSS and I was working for companies and stuff, but I didn't know how to create markup. I just knew that Drupal sped up divs. So when you're creating the CSS, you have to actually write the markup by hand, which is good because you have more control over it. So for prototyping, you need to write that markup. But also for documentations, I mean, this is a simple component right here. But imagine if it was more complex and you had grid systems in there and you had responsive media queries in there, it can start to get quite complex. So you need to understand the source order and actually what's going on inside of each component, especially in a responsive age, especially when you're dealing with accessibility attributes, ARIA attributes, all sorts of things that you need to be able to have a tab on. So documentation and for prototyping, you should have example markup. So basically in my opinion, the way that we do it anyway, every class that you write in Drupal, like when you're theming a site, every class should have markup documentation that shows what is happening with your CSS so people can understand coming in maintenance in the future. So Fuglies, this is a concept brought up by my co-worker John Albin, he's the author of the Zen theme Drupal's, you know, I guess largest or most used base theme and most downloaded. And yeah, this is how you might apply. So you apply these nice design component namespace classes to your Drupal selectors. This is his solution for that. And what you do basically, I mean Fugly can kind of a funny story is that I was working with him on a project and he named these things Fugly selectors, we all know what that means I guess. But he's I guess for a kind of more high profile client so he's changed the name to just Drupal selectors. I like the politically incorrect name.  And this lets you manage your different instances around the site in a good way. And what it also does is it allows you to cut these out. If you ever want to use your design components or if you ever want to use your design which is now translated into HTML and CSS you might want to use it in a non-Drupal setting you might want to use it for some splash page or some promo for another brochure site or something you want to use some of those components. You can just cut out these Fugly selectors and then you can go and use that. And one use case you might want to do is going from Drupal 7 to Drupal 8. I remember going from Drupal 6 to Drupal 7 a lot of times what we would have to do a full rebrand because our design was so embedded into whatever site building opinions we had which were the actual Fugly selectors, whatever the site building tools we were using at the time, those were embedded into our design because we were attaching our styles directly to Drupal selectors. So this is in my opinion a must have for, you know, be able to maintain your site and be able to re-theme your site or transport your site in the future or moving to Drupal 8. Anyway, I thought I'd put an example. I've talked about some of the different parts but I want to talk a little bit about a full example of what a complete component might look like, one of these partials. And this is not just something we're using at previous Next. I've had a look at some other frameworks and Ubuntu's vanilla framework has a very similar example to this particular example where you actually put the markup inside of your SCSS file as documentation, as some sort of to show people, okay, well this is what the CSS is actually doing. So you can click there and see their example there.  And you can see that these Fugly selectors start to get a little bit intense. You have to start mapping them. But they're mapped in a pattern. Oh, I didn't escape. Those are supposed to be underscore underscore. They're mapped in a pattern that is consistent. And it's abstracted. Your design is now abstracted from Drupal, which is really important moving into the future. So here's just another example of a little component. So in my opinion, no matter how small your CSS, no matter what the little tweak is, it should always live in a component. It's confusing at first when you first start getting into this kind of idea, but it ends up making a lot more sense as you work with them. So even something as small as like a read more link or something, obviously you would probably have more properties than that, just for example's purpose. Yeah, so that is another example of a component. And it has a little bit of markup. So extending design patterns. So when you start building your designs like this or building your themes like this or whatever you're doing, you can start amassing these components pretty quickly. You start having quite a few of them. And some of them might be, I don't know if anyone's heard of like pattern lab or atomic design. You might have smaller components that fit into larger components. And for example here, the way you kind of do that from the CSS perspective is, you know, you can, well, before I say that these components do something really important as well, the namespacing of them. They basically break CSS. They take the C out of CSS. They take the cascade out of CSS to where whatever happens inside of one of these components shouldn't affect what happens within the other components. Now you can write things and be totally sure that because they're namespaced and we do that all, you know, in computer programming, whatever, all over the place. But yeah, that's an important thing. So the cascade is basically broken. We don't care about it anymore. Unless you kind of do care about it though if you are wanting to use CSS from one component and a smaller component in a larger component. And here's a perfect example. In our pretty grid, title, remember the title with, let me just go back real quick. Here it is I guess with the title with a little arrow. You might want to extend. That arrow could be a complex. You might have an SVG. You might need to position the SVG over to the right. You might need some padding. You might need some CSS3 properties. You might need auto prefix or stuff. You might need some responsive things that happen. So you might have quite a few properties there. I mean, you might not want to write that over and over and over again. So you might, in the title you might want to say, well, this is pretty similar to Read More, but actually the font size is a little bit bigger and then maybe you have to move the arrow around or maybe the font, there's a different font family or something like that. So this is a way by importing, this is a kind of a SAS thing, but other preprocessors have a similar sort of setup, by importing the subcomponent in the head or not in the head, in the top of the page, you can ensure that that CSS based on, I mean, using Compass in this particular instance, you can ensure that the CSS will actually come, the Read More CSS will come before the pretty grid title CSS, which actually becomes important. In that case, the cascade is important. And by using dependencies, by listing out a hierarchy, you can start doing some pretty cool stuff and reuse code where you might not have been able to reuse it. And when you start doing this, you start seeing a lot more patterns in your design when you actually have the tools to do something like this. So this is just something we've been doing recently and it's, so I showed you an example of having the HTML documentation inside of the actual component itself. And putting all the different languages in the component is, I guess, if the component is complex, it might make the partial pretty big. I guess they do that in React components and some other components, they put everything into one file. But I like the idea of kind of splitting out your different language dependencies into different files. So pretty grid might have some CSS that it requires. It might have some HTML. It also might have some JavaScript that does something fancy, maybe equal height grids or some fallback or something that you need, particularly for that component. I'm not going to get into JavaScript dependencies and stuff. I think there's some other sessions that talk a little bit about that. But yeah, I like putting HTML long story short in a separate file because then with your IDE and if you ever want to do anything like linting, you can use your IDE won't bug out, freak out. That's the main reason for me. Yeah, and they do this. This is not just something I made up. They do this in Google Material Design Light, which is a really cool framework by Google. Google came out with their Material Design spec a year ago or something. They're hoping to implement it with web components, but I guess they're a little bit too complicated or not ready yet with Polymer and the like. So they've come out with well, Material Design Light is just a vanilla HTML, CSS, JavaScript implementation of all of their Material Paper Design components. Anyway, so yeah, like I said, you might have a lot of these components and you have a lot of this example to mark up and you have no word of prototype and you're creating static sites and boilerplate HTML files just to see your components because you want to create components in this way. Really, you need some tooling that's going to allow you to take this HTML and actually visualize it because just looking at the HTML at the top of a file, I mean it will give you some hint of how it's supposed to be used if you're familiar with HTML, but you really need to see it with the CSS actually applied to it in a setting that is referenceable in like a catalog sort of setting. And that is where a style guide might come in. So a style guide, when I first heard about style guides, I think all I thought immediately of the AP style guide, like of writing content and how to structure content, because I've done it in college, but MailChimp does something similar to where they talk about the voice and the tone that they might use. If you ever use MailChimp, they have the funny little jokes all around when you're trying to create email campaigns. Yeah, so this is a style guide for content, but this is not really what I'm getting at when I talk about like a tool that can help with our markup. What a lot of designers think when you tell them, you know, we need a style guide, we need a component-based design, they think, oh, like element collages or style tiles where you have some colors, they give you like a photo shot file with a couple of colors in it and some typography and it's like, here's your style guide. Well, it's a little bit more than that as well. And going back to material design light, they actually have a good component library, and I think that's more the word we're looking for. A component library, and I guess I call this a static component library because all the components are just HTML, CSS, and JavaScript. And this is really cool. You can actually click on, you know, these, you can't do it in here obviously, but you can click and do Chrome Inspector and actually see how these things are made and you can actually open them up in CodePen. I like this style guide a lot. Yeah, and you can see the markup here as well, which is really important and has a catalog of all the different components. So, the tool that I'm talking about is more of a style guide generator, and that's what we use. We use what's called a style guide generator, and we use a project called a KSS node. Now KSS stands for Nile Style Sheet, just something that some guy made probably a couple of years ago, and he made it in Ruby, and GitHub uses Nile Style Sheets, the Ruby version for their style guide, component library. Yeah, so why do we use KSS node instead of KSS Ruby? Well, front-end developers tend to know more JavaScript than they know Ruby. And so it's easy. If something goes wrong in the style guide, you can make a commit. The other reason, I guess we use KSS node at previous Next, is because John Albin, who's my co-worker, he's the maintainer. He's not only the maintainer of Zen, he's a maintainer of KSS node. Another reason that people in Drupal might want to consider KSS node is that now in the latest alpha version of Zen, a KSS node is included by default, and all of the components, CSS components inside of Zen's starter kit is basically built around the KSS node style guide, because John's the maintainer of Zen and of KSS node. Now just to note with KSS node, and this is a pretty big when people are coming and learning it, KSS node does not care about CSS or JavaScript. It only cares about markup. So it parses your CSS files and looks for markup basically, and a feature of KSS node that you can specify that we've added recently, what I mentioned in the past, and before in the previous slide is that you can add a separate markup file, and KSS node will know about it. KSS node is just an NPM package. You download it, and you can use it in the command line and feed it a few arguments. Yeah, so we need a way to get all of the CSS, all of the HTML out of all of our components and all of the CSS out of our components, all the JavaScript out of our components and get it into our style guide, or KSS node style guide and our Drupal site. And the way we do that is with some tooling. So we use Gulp to basically to watch our saves basically, and KSS node will parse out the HTML from each one of our components, and it allows us to follow this component-based workflow and parse it into these performant one HTTP request, CSS file, JavaScript file and HTML file. Now KSS node is just a blue. It just cares about HTML. We have other sessions that might talk about the CSS. I guess I talked a little bit with the imports and stuff, but then JavaScript is something else altogether. This component-based methodologies is going to be pretty prevalent when web components come out, or when they're supported in most browsers. And yeah, it's important that we start looking at building front end in this way. So this is what it looks like after your HTML and your CSS has been parsed. Your CSS has been parsed of all the markup documentation.  And look, it's just a very small component here. It's nothing big and it's not very pretty. It doesn't look like the Material Design Light Style Guide. But it serves an important purpose because now we can take a URL, a reference to this particular component, and we can send it around the company, send it around our client or our development team, and we can iterate on this particular component and talk about it and reuse it, and maybe even do testing on it. Maybe we can do some sort of screenshot rate testing or something. I mean, that's a little bit too much for this presentation. But having a URL for something is really important. And your markup's there as well. You can actually see the markup that's used. Obviously you can look at it in Chrome Inspector, but it's nice being able to look at it right next to the actual component. And you can find all of your components pretty easily. And this is a huge benefit when you're developing.  So a little bit more about a design component, but then KSS Node has some interesting things in it as well. First off it's, I mean, we have a CSS file in our normal static HTML, CSS, JavaScript components, but KSS Node doesn't necessarily need to use HTML. It can use HTML, but it can also use the templating language of your choice, like a template file. And using a template file means that you can have variables, just like, you know, think of any of our Drupal templates, or if anyone's played, everyone said their friend developer's played with, you know, templating engines or anything. You can actually, in our components, we can have data that moves in and out of our components, so it's really important if you want to start using your components in different places. You need the titles to say different things. You need to have different context. You might need it to be different colors. You might need to feed it different classes. You need to have variation in the heap of mark, you know, in the component that you're trying to edit. And it also lets you, so KSS also lets you specify a JSON file which has some default data, basically placeholder data for that component, because obviously in the style guide, it's not something that's dynamic. You just want to have something there that you can, you know, just see when it gets generated. So I'll show you an example. Some of this might have been confusing waiting for the video. I have a little video which I made on the plane. Let's see if this works. One second here. Can I do this? Ooh, okay. Huh. Okay, I have to do it over here, one second. So this is just an example just so you can get an idea of how this might work. You know, you have your component, maybe your read more component. That's what this one is. And it has, you know, little specs of data in it. And in a JSON file in the same component, you're going to have, you know, you're just going to specify in JSON the URL and the title. And then you can do gulp watch it will compile the style guide. You can start to see how the tooling comes into effect here. And if anyone has any questions about the logistics of that, I can walk you through how that works.  Here it is. And then you can see where the data, the placeholder data has been shown in the style guide. And this becomes really useful. It might look, you know, a little bit hodgepodge here, but it's very useful to be able to do this. I'll show you why, because like the CSS being able to reuse components from one CSS file, one partial, and reuse it into another and be able to import a whole list of properties into maybe a sub part of your component, you can also, with CSS, you can also do something like that with markup. And that's what these template engines are all about. It's taking pieces of markup and being able to mix and match and reuse them and place them within other pieces of markup. And with that you can start to build bigger and bigger components. And this is pattern lab. This is their idea of a page. You know, where you see you have all these components that are compiled together, not compiled but included. They're just including a bunch of different components on the page and those components have micro components, I guess they call them molecules and atoms, etc. And you can do a similar sort of thing with KSS node or whatever template language you're using. Now a question is like, how do you get some of this markup that and these classes into Drupal? So we've created these nice CSS component-based selectors and we have markup too that goes along with that. And we need to either get the classes in the Drupal or you need to get the markup into Drupal. Let's talk a little bit about these fugly selectors. So I have a little fragmented slide thing that I made here. On the left, on the left you have the pretty great cart partial that we talked about before and inside of it you have some extended fuglies. On the right you have your views view grid or whatever your views template file is or your view implementation that has Drupal markup in it. Now the problem with fugly selectors is that if you get rid of that Drupal implementation, you then need to go into your partial and you need to fish out all of your fugly selectors if you want to maintain your code. You could just leave it there. Nobody would know the wiser probably. I don't know. But you have to go and fish through your components and pull out the fugly selectors. If you actually use the markup though in your template file, you don't have that problem because you're able to get rid of your template file which has the nice markup in it and you don't need to worry about maintaining the component because the component doesn't have, the design component doesn't have any Drupalisms in it. It's just fresh and it's by itself and it's abstracted. So in my opinion using classes is a preferable solution. But we all know Drupal. Getting classes in Drupal is not easy. You have these theming functions which go on forever. Luckily they're gone in Drupal 8. If you have to use one of these, just use a fugly because you don't want to have to maintain all that PHP code, forked PHP code. Then you have markup UI. I can go on a long time about using panels. I know Drees said we need to make it easier for people to point and click. But maybe five years ago when you can add a few bootstrap classes and you can kind of make a desktop style site with a few classes. But in my opinion having to when you have more complex components that have responsive properties, you have to deal with performance, you have to test them, you have to do a lot of things. Getting all of those classes and all of that markup into Drupal using display suite extra or using panels classes is quite difficult. Now I worked on a development team where it was great. Because we had the style guide, we had all the classes listed, the devs were able to go into the style guide, rip all the classes out and get them into Drupal which is great. It really sped up development. But the problem is maintenance because on that site currently if somebody needs to go in and change anything, they do a command F for that particular class because you have the panels are all full of markup and you find C tools configuration because that's where all the classes are stored and C tools exports. So it's a little bit difficult and it's just very verbose filling these things out over and over again. I don't have a solution for that. I'm just complaining a little bit. Layout UI is a similar sort of thing. It seems to me whenever you drag a pane into the panel, you know, you have to re-style it again. It doesn't, just adding a class or something, it's not themed. You still need to get the theme or to iterate over it again. And maybe if you create it in a really good, in a perfect way, it works. I'm running out of time here, but yeah, so the, actually I think I'm doing all right. So benefits of Drupal templates. Let's just rehash some of this again. Preserve the components. Preserve the component source orders. If you're actually using Drupal templates versus, you know, fuglies or just attaching your classes directly, your styles directly to Drupal classes, you're able to see the markup and, you know, preserve that component source, sorry, markup source order, not component source, markup source order. That's so important. There's no configuration management. So you're not having to, if you're using template files, you're not having to actually go to features and refresh that configuration because it's just stored and get the configuration or the HTML is stored in version control. And then yeah, you can search in the ID. I find it a lot easier to search things in the ID. So disadvantage, managing template files in Drupal is hard. Like the naming convention, the naming conventions around Drupal templates like node, dash, dash, article, dash, dash, my special suggestion in template.php is not easy to deal with. And people come up with all sorts of nifty ideas on how to manage that. It can be difficult sometimes. So I'm not denying that. Duplicate templates. So I want to talk a little bit about this. This is a problem that bugs me in particular. It's probably more of a pet hate than a worldwide Drupal problem. But duplicate templates. So because we have the standard like previous, next, that whenever you write CSS, rather you write a component, you have to have example markup inside of that. You have to evidence your CSS with a working example, HTML example. We're writing markup no matter what. We have to write markup because it's part of the development process. Because we're writing markup, we're also writing markup twice. Or we're copy pasting markup. So what we end up doing, because KSS node is written in handlebars and Drupal is written in PHP, the only thing that's different here is the variables. So one's PHP template and one's like little brackets. We should be able to somehow reuse and duplicate, especially when you have a more complex component that has a lot more HTML inside of it. This is just a simple example. But remember we had views. We had paragraphs and we had entity reference. Imagine taking that example on the right and having a copy pasted in each one of those template files. You start to find yourself duplicating templates a fair bit there. So what if Drupal could actually use style guide templates? So what if the templates in Drupal could actually leverage what we have in the style guide? Because the style guide becomes such an important part of our development process. Drupal would have to be able to use handlebars. That's not happening. It's Drupal 7. That's not going to happen. I'm not dealing with that. KSS node could use PHP template but then you'd need a JavaScript implementation of PHP template. That's not going to work. Luckily the Drupal 8 got twig and that's where things start to get interesting. Because twig is written in PHP so Drupal could potentially use it. But it has a syntax that's more similar to handlebars. It uses the same brackets. But that's not really going to help us. Drupal 8 and twig is awesome for other reasons like it's more secure. We're not writing SQL queries in our template files but just because it's similar does not mean it's handlebars. And that's where it gets interesting again because there's something called twig.js. Twig.js is a JavaScript implementation of twig which means that node or KSS node which is written node, node is a service side implementation of JavaScript can leverage twig.js and use it in its templates. So instead of using handlebars KSS is currently written as a generator that uses handlebars it could potentially use twig. And Drupal is already using twig. Drupal 8 is already using twig. Yeah. So I wrote a pull request to John Alban's KSS node and it's passing now. It's like it wasn't passing then. But yeah. I basically wrote a generator that lets us use twig inside of KSS node which is interesting because now you're using the same templates as Drupal might use which gets you closer to making your style guide look like what is going to be in Drupal 8 and core. Now I have to write some tests and stuff. That's always the hard part when you have to learn JavaScript a lot better than I already know it. But yeah. I'm trying to get that in and I'm sure when Drupal 8 is released there's going to be a lot more momentum around getting something like that released. So let's talk a little bit about twig. Some of the syntax for twig when I'm particularly interested in it are the includes and in handlebars the includes I showed an example before the includes looks something like this where you have this little arrow and it just kind of includes things for you. And twig I've spaced it out weird but just so it can be used in the example. Yeah. You can just include in a very similar fashion and then you can start includes the first step into building these larger components. And we can do this in Drupal 8 core. This is not necessarily a style guide thing. So we can start mixing and matching our components hopefully whether you're using a style guide driven development or not. Extending is another interesting concept to where you can have this markup and this is actually extending is used in core as well I think in the classy theme or somebody will tell me after the after the session but it basically allows you to add markup to particular templates and yeah so you can have you know here's block.html.twig which is in Drupal core and this potential child template is extending it and then just adding a little bit of extra markup around the content to make it you know maybe make it match our design or what we need to do to have it better. So we're using KSS node Drupal template so you might want what you could potentially do and I've had a play with this and it's looking pretty good but it's not 100% is you might want to take something that is inside of your Drupal template and inside of your style guide sorry that's written in twig and extend it inside of a Drupal template and this would let you avoid having to have having to write you know having that duplicate markup and all these different template files you can just extend what's already in your style guide and then potentially if you change something in your style guide it's also going to be changed all throughout your Drupal site which some people might see as scary because who knows what's changing but we shouldn't as developers we shouldn't think that that's scary if you're you know you should have to be able to test for things like that and hopefully you know that sort of stuff evolves. These are just some ideas at the end of the presentation that I'm throwing out there. I just want to show you one quick thing that you know you might want to do here just one more video about reusing templates. So this is an example I put together you know I played with this is always the fun part of these presentations and this is the KSS style guide and this is a title of the grid item that we had seen before and this is a Drupal site so this is Drupal 8 site that I spun up and it's using that same template so I got them using the same template I was able to extend the template in here into the Drupal 8 site so it is using the same template here to do that now to kind of prove that I guess I'm going to put a inline SVG into my template which is being used by both the style guide and Drupal 8 and I'm going to compile my template or I guess generate that static HTML style guide and now you can see that obviously that inline SVG is there but because Drupal is also using that template and I have twig debug mode turned on obviously so it's not caching the twig file so any changes that I make it will automatically be seen and I go to my Drupal site and I refresh then obviously the inline SVG is there so there's an example of you can imagine having the views, home page, the promo landing page and then the entity reference and having all those template files just being automatically updated and there's no mistakes there, everything got made if it's using that component and extending it it got made but one interesting thing that can also happen and this is our style guide by the way is because we're using twig.js templates can not only be leverage like twig works in KSS node on the server side because it's in node but obviously javascript works in the browser as well so there's no reason why front end like the browser can't leverage these templates as well so what you can do is write a little bit of javascript that maybe lives in its namespace nice with your component lives with your component and without even having to do anything and Drupal or anything just write some javascript that includes twig.js in your browser and then you're just able to kind of pull in, leverage a template, feed data through it maybe using that graphql that Dries is talking about and leverage the same templates on your style guide and on your Drupal in the browser and yeah this is I'm not going to get into the javascript it's relatively simple and yeah and then your Drupal site remember we just did that in our HTML in our style guide but it should theoretically the same just work in your Drupal site as well no need for Ajax, views, plugins or anything like that and just yeah so just knowing that oops yeah so just knowing that that stuff exists that stuff's out there that using twig is going to be huge having a template engine in core is going to be a big tool for us as front end developers we're going to be able to mix and match and do a lot of things with HTML and with client-side development and you know we're going to be able to do a lot of stuff so just knowing that those tools exist understanding that with the benefits of a style guide being able to catalog all of your components being able to iterate on a development team being able to show the client and show the designers things so we're not rehashing all this work over and over again and writing all this stuff time in and time again time in and time out it's going to be really advantageous to us moving forward going in to Drupal 8 so I guess that's it from me you can find me on Twitter that's my name and yep thank you very much appreciate it does anybody have any questions yes ma'am yeah I use KSS for prototyping so whenever I get a new design in I go into the style guide and I make it all in the style guide first so I don't make a whole site in the style guide I would never do that because you know you don't want to waste time the client's time and stuff you want to do in like an agile sort of fashion so when the story comes in you just make the HTML and CSS that has to do with that particular component you build that component and and then you prototype it and then you worry I'd estimate them separately so I'll estimate the HTML and the CSS and then separate is the Drupal integration and then everybody can see the cost of Drupal what is Drupal cost what is the integration actually costing us because you shouldn't have to worry about Drupal when you're actually doing all the responsive stuff and performance and trying to figure out how the component should work and the browser is the best pattern labs written in PHP but I think yeah I'm sure I haven't used pattern labs so the question is whether KSS nodes similar to pattern lab I'm sure they have a lot of similarities and yeah I haven't used pattern lab extensively not yet but I don't see why not like once it's all set up it's really easy if you know just HTML and CSS you should be able to iterate on those doing presentation this presentation is as much for people in my company as it is for the outside world because I like people to understand that the style guide is not just some you know some things some like colors and stuff on a page it's actually a really important development tool that can allow other employees maybe people who don't have experience with Drupal to do half the work on the project yeah interesting yeah yes sir good question I would put the image if oh I'm sorry my bad so the question is whether or not I put the images that have to do with the component in the component folder and if it's like a background image that has something specifically to do with that component then I would do that or if it's a placeholder image that has to do with the demonstration and the testing and the documentation of that component I would put the images in the folder but then you're gonna have to reconcile that you're gonna have to reconcile the paths you're gonna have to deal with the tooling and how it's pulling the image out and whether the path is absolute or relative and there's still problems moving forward but hopefully the tooling will get better and standards will get better around things like this anyone else alright thank you very much appreciate it yeah sure where you have some plugins like right now you put the right abuse plugin and put your grid or something or a great HTML into the abuse plugin but it's still a bit annoying to write abuse plugin and then you cannot use it if you have for instance field items and just want to have field format with field items and you cannot use the same grid or whatever you have and so what I'm doing is writing a bunch of different plugin types even if my own plugin is the product but right now on this demo site you just have this list format and there's another one entity display plugin and for instance this is my article view and now I'm integrating this in just a list format and right now you just have this too and each one of these have different options but in the client side what I work on


Hey there lads and ladies, it is Petrifying Pumpkins here and welcome to another firewall ultra updates. This one I'm very happy to give. This one is about an update that's going to come out in the next 24 hours and that's going to fix some major issues with firewall ultra. So this was posted to the firewall discord and I'm just going to read out what they said and we can all be on the same page. So new upcoming patch updates version 1.05 in the next 24 hours will be deploying a patch focused on improving the stability of squad matchmaking as well as accelerating player progress through reputation levels. You should expect some momentary downtime while the patch is applied. We are committed to improving the player experience and will continue to refine squad matchmaking as we monitor the live environment. We've heard your feedback in regards to the pace of earning reputation and you should now experience faster progress through ranks. We appreciate all the time you've put into firewall ultra already in its first week and we are dedicated to bringing you the best experience possible. We will continue to bring improvements across to the game so stay tuned for more updates as we find the sweet spot of challenge and fun. So excellent news, well done to the team at first contact entertainment, not only forgetting the matchmaking patch out, assuming it works now of course, plus my fingers crossed I'm feeling optimistic that matchmaking is going to be fine now, but also from how quickly they got the feedback for you know how people were complaining about the grind of the progress and now they've already got a patch out that's going to boost us across the board so that was rapid you know that's a pass on the back for first contact entertainment on that I think well done to them and hopefully before I stream firewall ultra tonight that patch will be live so that I can finally play public with my squad and start earning crypto and xp because I've been in limbo for the since launch basically. Anyway that is it for this video it was just a quick update video this time let me thank Decepticon for let me use his music in all of my videos you can check him out in the link below let me also thank my channel members who are helping by supporting this channel they are the following muzz dead eyed dan I've never seen such behavior in the war room before chop PPE no one knows move master Mick Alva worldly commentator cast digs the pumpkin patch kid Pete Hawkins crumb and mr. 777 thank you very much for that support it's greatly appreciated if you'd like to join and become a channel member too you'll see the join button beneath the video somewhere as well anyway thank you for watching if you want to stay tuned for all ultra updates you can consider like and subscribe and and all that usual youtube and shite and I will see you in the next one until then please stay ultra moist


Hello, everyone, and welcome to the WordPress Briefing, the podcast where you can catch quick explanations of the ideas behind the WordPress open source project, some insight into the community that supports it, and get a small list of big things coming up in the next two weeks. I'm your host, Josepha Hayden-Champosie. Here we go. A couple of episodes ago, I mentioned the community summit in the small list of big things. That's coming up August 22nd and 23rd right before WordCampUS, and for some of you that made complete sense, and the only thought in your mind was, wow, our last one was in 2017, how could so many years have passed since then? And since so many years have passed, today we're going to talk a bit about the community summit, what it is, where it came from, and why it's so important for the WordPress project. First things first, let's talk about what exactly the community summit is. The community summit is a small event where folks from around the WordPress project and community come together to work through some of the most difficult topics the project currently faces, many of which are easier, or at least less fraught, when we can be face to face. The community summit is usually done in an unconference style, and when we were smaller, we left topic gathering and voting to the day of. That's evolved a bit as our group of fearless contributors has grown over the years, and this year we have been asking for topics ahead of time so that we can make sure we have the right folks in the room and are making the best use of everyone's limited time. It's easy to take a look at this event and think that it's like some fun, exclusive thing with a who's who of WordPress, but I assure you it's a working event. Decisions are not finalized during the event, but since we try very hard to account for many, many viewpoints, it ends up being two days of hard discussions, contentious viewpoints, and problem definition at a level of complexity you don't really see every day. Hearing how hard this event is, you may be wondering why we put in that effort. There are a lot of reasons, but there are three that come to my mind immediately. So for starters, working across cultures is hard, apart from the cultural differences we tend to be aware of, things like where we're located or our lived experiences, things like that. Working remotely or distributedly is a whole different set of skills than working in person. This helps remind everyone that we're humans, that there are humans behind those comments and behind those messages in Slack. The second thing is that I'm a big supporter of Tuckman's theory of group development. If you don't know what that is, you can look for it, we'll put a link in the show notes, but it's that forming, storming, norming, performing kind of concept of how groups come together. Because there are so many of us and our community has such a large footprint, there are little storms, a ruin all the time. Some get really big, some say small, but at some point most of them have to be addressed and this is a space that is specifically designed to help us do that, which brings us to the third reason that we do it. This event uses something called Chatham House Rules, which creates kind of a temporal psychological safety, right? For safety, if you're a leader, you know that that's something that is built over time and requires a lot of trust and a lot of conversations with people that you're working with, and we can't quite do that. So Chatham House Rules builds an environment that helps just create that suddenly in the moment and requires some faith in one another. But basically no one can be quoted about what they said in those conversations. No one's examples can be attributed to them, but the conversations can be summarized and published, which we do on the community summit website. And then we publish those for our collective knowledge over time. This lets folks who are attending advocate for themselves and others fully without worrying over whether they're going to be taken out of context later. And finally, one of the biggest questions we get ahead of any community summit, why is it by invitation only?  You want it to be large enough to have good representation, but small enough to have high quality interactions. It's just a really narrow Goldilocks moment, if you will. But that reason doesn't necessarily address the need for invitations, rather than letting it be first come, first served. The reason for that is more of a philosophical one and requires you to go on a mini historical journey with me. This also has changed a bit over the years. The first ever community summit way back in 2012 was before my time. But if I recall my history correctly, it was truly by invitation only. The summit after that included a closed nomination process. The next included a team nomination process. And then the last two, 2017 and 2023, have included open nominations. Now, even in the nomination era of community summit organizing, there is still a selection process. The organizers review the list of suggested attendees and check for the same types of things we expect major word camp organizers to look for in their speaker selection. Things like which teams they contribute to, what communities they advocate for, how long they've been a member of the community. And then they adjust for balance. In addition to those things, there are also four types of voices that we always want represented at our community summit. So first is leading voices. People who are already in the community and kind of are helping us to make decisions. I am considered one of those leading voices. I have put in my application to be included in the community summit. Really hope we select me. The second one is future leading voices. Specifically, those are people who are active in the community already and are showing a lot of promise either because they really understand the values that the WordPress open source project are putting forward or understand the basic processes of communicating and guiding people in such a complex ecosystem as the WordPress project represents or because they have said quite plainly like they are interested in helping us to make sure that the WordPress project is able to move and continue to create and continue to support democratizing publishing. So it's a little bit potentially folks who are self selecting for that people who already are showing that they are doing that either in WordPress or in their local communities. That's one of the types of voices we want to include. The third one that we want to include all the time is voices we need. So voices that we need to hear from people that specifically we are building WordPress for. People that have indicated to us that the CMS is not necessarily perfect for some of the use cases that they run into regularly. So the people and users and community organizers that can and are able to for the types of user interactions, the types of community interactions that we absolutely want to be able to see. And so that's a third group of voices that we want to make sure we have represented. And then the fourth and final group that we always want to have represented is a group that I call voices we miss. And so those are the people that we want to be able to hear more from in our project that we don't necessarily either have a good group of representative voices for. So it's hard to hear them or that we know are probably users of the CMS or they are attending events. They are somehow involved in the WordPress project. But we don't necessarily have any way to have accounted for them while we were building solutions way back in 2012 or 2006 when things were being built for us. And so those are the four groups of people, the four types of voices that I absolutely want to have represented at our community summit. And I ask organizers to go through this incredibly complicated selection process because we want not simply a microcosm of the WordPress community as we see it today and hope to see it in the future, but also an equitably voiced forum during that critical problem definition phase. So TLDL for listen, TLD, re-listen. Although if you didn't make it through that, you definitely are not getting to this point. So a TLDR for folks who skimmed to the transcript and got here, I guess. We keep this invitation structure because we want to account for voices. We don't hear every day in the WordPress project. Not because we don't value them, but because we already hear them. And now that brings us to our small list of big things. This week it's actually kind of a big list of big things, but there it is. First things first, the applications for the community summit are now open. Those are the applications to attend. It's pretty short. I filled mine out this morning and it's like three questions about who you are and your username on WordPress.org. And then three questions about the topics you are most interested in and the experience that you have in those conversations so far. Yeah, it took me, like, I think 90 seconds, like full minute and a half. So head on over there. We have a link in the show notes, but also you'll be able to find it in newsletters across the entire WordPress media ecosystem. I am pretty sure about that. The second thing is that there is a proposal out for a project-wide mentorship program. This is a huge potential win for us. It is aiming to fix some of our broken ladders. If you're not familiar with my broken ladder theory of the WordPress project, I'll try to remember to find a link to that post and put that in the show notes. Number three is that Openverse moved. I shared this last week. That happened last week. They didn't move very far, though. They have a new URL. You can find them at openverse.org. It's the same team. It's the same product. It's the same group of excellent openly licensed images and media that you have come to expect. It just has its own standalone URL now. Huge kudos to the contributors who got that done. Another thing that happened last week is that WordPress 6.2 has moved into its beta phase.  There also was an excellent, excellent write up about how to test any given release. And I think it also includes how to file a good bug. And so we'll send all of those things into the show notes. They'll be easy to find.  And number five, longest, smallest of big things in recent history. I got some interest on women and non-binary led release for 2023. And so since there was some interest shown for that, it is hereby verbally confirmed. Keep an eye out on make.wordpress.org for more information about what that process is going to look like and how to volunteer your time for that if that is something that calls to you. And that, my friends, is your small list of big things, your big list of big things. Thanks for tuning in today for the WordPress Briefing. I'm your host, Josepha Hayden-Champosi. And I'll see you again in a couple of weeks.


Hi everyone. Everything has started. But before we do, I need to read the fire exit announcement. So please note the locations of the surrounding emergency exits. Locate the nearest exit, nearest lit exit sign to you. In the event of a fire alarm or other emergency, please calmly exit to the public concourse. Emergency exit stairwells leading to the outside of this facility are located along the public concourse. For your safety in an emergency, please follow the direction of the public safety staff. Great. So hopefully you're all here for this talk. We're going to be talking about transforming a bank with a highly opinionated, automated release pipeline. I was trying to win the prize for the longest title at the conference, so we'll see. It's very slow. So who am I? I'm Reid. Reid Lebeck. I work at RBC, which is a bank in Canada. I've been there for eight and a half years, and I've been on the cloud team for the last three, two and a half, three years thereabouts. And in that time, I've written several microservice-based applications, and we've started to try to roll that out to the rest of the organization and try to write some tools to help them do it. So what are we going to talk about today? So I want to kind of take you through, for those of you who haven't worked at a big organization, government or bank, that sort of stuff, I want to talk to you through kind of how we've done it in the past, some ways we tried to fix that, and then kind of the problems that led to how we solved those. So hopefully everyone knows what monolith is, a big, giant thing that's unruly. So that's not really the interesting bit, but more what comes out of having monoliths. So we end up having these long development cycles, so typically waterfall three months. If you're lucky, you'll get a deployment out. And each application deployment is a special snowflake. So we need to work with every other team. You know, you need to work with the network team, the database team, the five app teams that also use your database, et cetera, et cetera. So that's a mess. So we don't really want to be doing that anymore. And the other thing to note is that we do a lot of ETL. So a lot of the banks' applications revolve around these three steps. Get data from somewhere, transform it into the right format, and then put it somewhere else. And so traditionally that's been done with some database and then some process and then another database. So our idea at the bank was to stop doing that, which I thought was a good one.  So vent driven is new hotness. The idea is that you take your data and turn it into events, and then you can write event processors instead of having batch jobs. So streaming is ultimately better than batch because you can always express a batch process as a stream. Reading a file in is just streaming the file, right? So we chose Kafka as our bus event stream, and we wanted to put the data into Elasticsearch. So we have a lot of use cases where there's search use cases. So someone calls out the help desk, hey, can you help me here? And we need to look up the user's information. So Elasticsearch is perfect for that. And then we didn't want to do microservices. Sorry, we didn't want to do monoliths. So we decided to do microservices, put a REST API in the front, and then the data is exposed to the rest of the bank so we don't need to go through this mess all over again. So just briefly through the application architecture. And so the reason I want to focus on this is what we've done is we've written an opinionated framework for this type of architecture. So we didn't write a framework for every application at the bank. We wrote a framework for event-driven applications using Kafka in some sort of data store. In this case, it's Elasticsearch. So the data comes from somewhere, goes into Kafka. We have some consumer which will do some sort of transformation, put it in the bespoke format that we need for answering the question. And that question will be answered by the REST API, which will ultimately just ask Elasticsearch. And then the client will connect to the REST API. So fairly simple architecture. So we should be able to do something to make that pretty easy for everyone. So we went from monoliths to microservices, and now we have a whole new set of problems. So all the processes at the bank are set up to deploy very infrequently. So when you do a deployment every three months, it's really hard. So you want to be really careful and you want to be super diligent and you got to coordinate everyone. So there's all kinds of processes about change records getting everyone coordinated. But if you want to release once an hour, once a day, once a week, that becomes quite onerous and you spend more time doing the process than you do doing the actual deployment. And so the things that they want to make sure that are done in the process aren't necessarily bad things, like making sure testing's done, making sure that you're not going to break stuff, regression testing, et cetera, et cetera. But the idea is we want to automate those checks. We don't want them to be some manual process. The other thing is as we get more microservices, things get hairy in a hurry. So if all our operations is set up around monitoring one application, then we're going to be in for a world of hurt when messages are in Kafka and we have some eventual consistency in our API. So things get a lot harder. And the other bit is that every team has to do this themselves. So every team has set up monitoring. Every team has to set up a pipeline to go to production. Every team has to figure out how to solve all these problems.  So we need a pipeline. We need something that everyone can just use and benefit from. So there are too many pieces to do at ad hoc.   We want something repeatable, and we want some way to automate all those processes away. So what do we want to automate? We want to automate testing, deployments, rollbacks. Ideally, we do some zero downtime deploys. And those are not necessarily trivial. So it'd be nice if everyone didn't have to do it all themselves. All right, so the first time we tackled this was for an application. So we were on the cloud team, I'm part of the cloud team, and we were kind of doing the first of these event-driven architectures. Our very tight deadline is about two months, two or three months, which is fairly tight when you have nothing going on, and it's incredibly tight when you work at a bank and everything has a three-day SLA to get access timing. So what do we have? We had Cloud Foundry and we had our code in GitHub, and somehow we needed to get it from one to the other. So there was nothing. We had nothing there before. There's no pipeline we could leverage. We had a few tools, though. We have Jenkins, we have a central Jenkins. So we opted for Maven because we knew Maven. No other reason than that. And the Jenkins was already up and running, so we didn't really need to set up our own. So that was a small win. We had this other tool called Urban Code Deploy. So a quick show of hands. Who's ever heard of that before? Wow, that's way more than I thought. Okay, so for those of you who haven't, Urban Code Deploy is a IBM tool that lets you draw pretty pictures, and then those pretty pictures will somehow deploy your application. So it had a few nice features to it. So one, it was bank approved. So we didn't have to go through the process of figuring out, you know, how do we get whatever tool we're going to use? Approved, we can just go ahead and use this one. And for those who've worked at big organizations, they'll know that that's a non-trivial task. The other thing is it let us store secrets. So it was a place that was approved to store user names, passwords, et cetera. So if we need to connect to databases or Kafka, then that's good. The bad part is that we couldn't store our config in social control. So in our case, git. Which is bad because anytime you make a change, you make a change to the code, and then if you need to update the pipeline because we're still fairly new, then you have to go into this UI and make the change. And that's bad because as your code changes, this doesn't really change with it. So it's a situation where one branch needs to change the pipeline and then master doesn't, and you can only deploy one of the two. So that was no good. And then we had a UI, and I don't really like using UIs for development. It's just not my thing. So we decided to use Ansible. So who here has ever used Ansible before? Okay, who here has used Ansible to deploy the cloud? Yeah, so, okay, we'll get to that. Ansible was good. We knew Ansible, so that was kind of, we wanted to leverage some of the things that we knew. We had a tight deadline, we wanted to go ahead and not reinvent everything. We can store the config in git. It was good, we can move along with the branches, and we can have different config per environment. So Ansible has a concept of groupvars, and you can have a dev config and a prod config, which is good because you're going to have different servers for Kafka, Elasticsearch, et cetera. Bad Ansible is really good at deploying. I say you want to install Tomcat on 10 servers, and you just kind of list the servers and the way it goes. That's great. Deploying to Cloud Foundry, it's not so great. So there's no plugins for it.  So that was a bit awkward. We also wanted to achieve unimpotency, so the idea behind Ansible is if, say, Tomcat's already installed on server A, then it doesn't do anything. So we wanted to get the same idea from that for Cloud Foundry. And so part of that was driven by the reason that we used mono repos, or A mono repo, I should say. So all our microservices were in one repo, and we can talk about whether that's good or bad, but it was both good and bad in this case. So because every time something changed, it was going to rebuild and redeploy everything, we wanted to make sure that it only redeployed what actually changed. And so we did some fun, and by fun, I mean terrible ideas that you should never do at home. We decided to use MD5 sums to compare whether things had changed. So we took the MD5 sum of the jar we're deploying, so it's great, so we got that, and we'll store it in an environment variable in Cloud Foundry, and we can just compare for the next time, right? So the problem is, how do I even notice this, but jars in their headers have a time stamp or something that changes every time you build it. So if you zip up the same files twice, you'll get a different MD5 sum. So it's like, okay, well, we can work around that. So we expanded the jar into files, and then we did MD5 sum of each of the files and put that in a text file, and then we did MD5 sum of that text file. So it got complicated real quickly. But it did serve the purpose of we only redeployed what actually changed. The other tricky bit was services. So we have a few user-provided services to point to Kafka Elastic Search, any other services we needed. And it's very easy to update a user-provided service. I mean, it doesn't really need to be unimportant. The only problem is that if you're not deploying the application that the service is bound to, then you need to restage that application. So we had to do some finangling to figure out which applications were bound to the service that changed. So it got way too complicated, way too quickly. So we then had a second application, and it was very similar to the first. Very similar event driven architecture. We also had a tight deadline because we did such a good job on the first one that they figured we could do an even better job on the second one seeing as we solved all the hard problems. So there were still a few problems that we wanted to solve. We were going to use the existing pipeline because we didn't really have time to rewrite it, but we wanted things to go a bit faster. So I think for the first application, we had five or 10 somewhere in that range, microservices. So deploying that every time gets real slow. So what we wanted to do is deploy all of those at the same time because there's nothing really dependent between the applications. We could do five CF pushes at the same time. And all of this, again, is driven by the fact that we had a mono repo. So we added some parallelism. So our first attempt was to have urban code deploy called Ansible, say, five times with different parameters. Say deploy application A, application B, application C.  So what we did then is we added bash in front. So you know you're in trouble when you're doing parallelism in bash.  So it was great. The other thing we really wanted to do was to do that automated testing bit. The first one was a UI, the first application was a UI. And it just didn't have time to automate. Automating UI testing is difficult. So this one was all kind of just back-end stuff. So we could easily send some data in the front and check that it made it at the end in the right way. So the problem we had though with automated testing was that we ended up running on a live instance. So we would deploy the application and because everything was so entangled, we had to have all the services up. And then we ran testing on that. And so that's okay, but if you find a problem, it's kind of too late, so you have to roll everything back and it takes time. So this was definitely a lot better, but we needed to work on a few things. So what were our shortcomings? Well, it took too long. The whole thing took about 20 minutes, even if you didn't change anything, but more often than not, you'd change some central library and you'd have to redeploy the whole thing. We had five or seven developers. And if we all wanted to do something, we all wanted to do work, right?  On top of that, our automated testing was a bit flaky, so we weren't sure if it had just timed out or if it was an actual failure. So we ended up rewriting it. So we were lucky if we got four or five pull requests done in a day. So it was really slowing us down. The other bit is we... If you haven't figured it yet, we didn't do the wrong tools. So Ansible wasn't necessarily the right tool for deploying to the cloud. Bash is fine, but not necessarily that robust. And Urban Code Deploy wasn't really getting us too much. And on top of that, there were still a ton of manual steps. So if we wanted a new Kafka topic, or we wanted a new Elastic Search Index, or we wanted a new Cloud Foundry space, it was all manual stuff. We'd have to do that manually. So if I wanted to deploy to my own particular space, I would have to set up the whole pipeline. Set up all the pieces myself. So welcome to the third and ultimate pipeline. So this time, we had a bit more time. So we decided to rewrite from scratch. So we threw away the code, but we kept all the lessons we learned. And we still had a few things. So we had GitHub. Still, that worked well. And we had Cloud Foundry. We wanted to keep that. And we wanted to keep Maven. So we like Maven. We know Maven. So we kept that one. Now, one of the things we wanted to do, and you'll hear a lot of it, other people talk about pipelines, and they'll focus a lot just on the deployment side. And that's great. That's great that you can, you know, it certainly solves one part of the problem. But we wanted to go a bit further. We wanted to provide, not necessarily lock people in, but give them a head start on building their code as well. So when we're doing these applications, they're fairly standard. So everyone's going to have more or less to see, you know, you'll depend on Elasticsearch, Web, Kafka, et cetera. So we can at least fix the versions.  So we have a default parent palm. So you can just inherit from that, which is great. And we have some Maven jobs, or sorry, some Jenkins jobs, which will scan your repo and just grab your Jenkins file. And say, okay, well, that's kind of pretty standard, right? But then what we did is we wrote some functions, which we added into our Jenkins. So you could just call those from your Jenkins files. You just say, you know, do the CI or do the CD. And then you wouldn't have to worry about all the steps. So like stuff, and we'll get to what they are, but stuff that are not necessarily straightforward. So we have Jenkins. We decided to do our scripting in Python. We have some experience in Python. It has a fairly good REST API, fairly good to call REST API, which we're going to use for Cloud Foundry. We can run stuff on the shell as well. So it's good. And the other thing we used is Docker. And this is so key. So we ended up running Jenkins on Docker, and then the jobs would run inside Docker containers on Jenkins. And so that's super important, because now we don't need to worry about, like, is CFC-aligned installed? Do we have the right version of Python? Oh, we need to scale out to another node. What's our procedure to set everything up? And I know it sounds like we could give this talk five years ago, but it's pretty revolutionary for RBC to run this stuff on Docker, and it really helped us tremendously, because once we had those base Docker images with those prereqs in, we could just reuse those over and over again. It's just great. And the other thing we decided to do was go multi-repo. So we didn't like the way that Mono repo was working out for us. And I know some companies have made that work tremendously well, like Google and Facebook, but if you look at how they've done it, they end up using a lot of custom tools. And I know that Concourse, which we're at a Cloud Foundry summit, so we might as well mention Concourse. Concourse does very well where you can grab the subfolder inside your Git repo and say, well, just build if this subfolder's changed. But we didn't really want to go learn Concourse. We had enough to do. So we stuck with Jenkins. We wanted to make your changes at the top, so the top level changed, and there you go. So we went multi-repo. So every microservice has its own repo in GitHub. So the other thing we did is we wanted to automatically provision Elasticsearch in Kafka and Cloud Foundry stuff. And so we created a DSL. So it's in YAML. So that's how we got to talk here, you know. If it was JSON, they'd kick me out. So we have it in YAML. And it's just simple stuff. So what's your app called? How much memory do you need? What's your topic called? It's not all the options. So it wouldn't be everything you put in the manifest, for instance. It would just be a subset, because there's a lot of stuff that you don't really need to worry about, like where the Kafka server is or where Elasticsearch is. We'll take care of that as the pipeline. You just need to know you're going to use Elasticsearch, and you want your index to be called this, and here's the JSON and such and such. So you're checked to make sure that's okay. So you want to know sooner. You don't want to go all the way to deploy.  So we want to know that as soon as possible. So we checked that here. And then what we do is we put the artifacts in S3. Now we can argue all day as to the best artifact repository. Some people like Nexus, other people like Artifactory. We chose S3, partly for the reason that we had more control over that, and we didn't have to rely on other teams. S3 is to do stuff you have control over. The other bit we did was, so part of our goal was that each deploy, each branch would get its own pipeline all the way through. So if I'm working on my branch, Read, and my colleague's working on his branch, John, then, you know, we'll have a Read space, and we'll have a Read Kafka topic, and we'll have a Read Elasticsearch index, and so on and so forth. So I'll be able to deploy everything all the way out, and we won't conflict with each other. So the nice thing of using S3 is we can put the buckets named after the branches, or named after whatever we're deploying. So we have a prod bucket or whatnot, which makes it really easy for cleanup. So whenever we delete the branch, then we just delete the S3 bucket, and it makes life a lot simpler. And then grand plans, which haven't quite been realized yet, I'll be honest, but the idea is to have a chat bot in between the CI and the CD. So when you're deploying to dev or maybe staging, it's great. You can just kick it off automatically. But especially at a bank, you'll want to hold off your prod deployment to have some sort of manual step. And that manual step will be someone saying slash pipeline bot, deploy. So that's kind of how that will work. All right, so that was the CI. So let's have a look at the CD. So what we did now is we created the environment parameters. So these are things like server locations, API URLs, et cetera. And then we combined those with the application parameters to get our full picture. So we know it's going to be called Reed's Awesome App. And we know it needs a large amount of memory. So we give it four gigs and so on and so forth. So we get something that we can actually go off and deploy. And then we'll go and provision our Kafka topics if they're not there. Again, item potency. We'll create any elastic search indexes if needed. And yeah. And then we'll do the actual deployment. So the idea is we're going to do a blue green deployment. So we want to have zero downtime as much as possible. So I can never remember which is which, but so we'll do the green first, deploy that.  So we have our temp route pointing to our new app. So we'll do the testing there, which is great because now if the testing fails, we can just not switch the route over. And then if the testing does pass, then we will switch the route over. So the idea is that this automated testing is something that the user, the developers will provide. So their development team will plug something in there and then it'll call whatever testing it needs to happen. And then we'll just check the return code. So either success or failure. And then we'll finalize the deployment. So we'll make the running app live. So what, so this is a picture of what the developer experience looks like now that they have this pipeline. So all they have to do really is inherit from the parent palm, add a few parameters to say, here's my app name, here's the memory, here's my Kafka topic. And that's it. And they commit and away they go. So I mean, this was so good that when we were developing it, we hadn't kind of finished everything, so we got like little bits done. And so we're working on the bits that weren't in the pipeline yet. There's a little room of like four of us.  Is it so good? So at that point that I knew that we had something special. So thank you very much. That's it. That's it for the talk. Thank you. So we have time for a few questions. Anyone wants to grill me? Very good. So the question is what does Cloud Foundry do for us in the pipeline? So Cloud Foundry is just our deployment target. So we use that as a platform as a service where we run stuff. Does that answer your question? Like it's the target where we deploy. Where we deploy to. That's a good question. So where we run Docker is still in flux. So there's talk of running it there. There's talk of running it elsewhere. So did we have to do what for Artifactness? Sure. So the question is, did we have to get approvals to store our data that's hidden in this wonderful green box? Is that an internal S3? So it's an S3 API on top of internal storage. So in that case, we didn't have to get approvals to put stuff outside onto public cloud. So this whole thing runs internally. Yeah, good. So the question is how much work was this and was it frustrating and it felt like it should be available somewhere else. So the answer is probably too much work or not enough. I don't know. It felt like a lot of work at the time. But the results are certainly there. I wish there was something that we could use straight out of the box. And there are some efforts. Shed it to Spring Cloud Pipelines. I know Marchion is doing a great job with that stuff. It didn't quite fit our use case. We kind of gone far enough down the road that we weren't able to use it. But it really depends on how much control you want to give up. So if you want to do everything for your developers or as much as possible, then nothing is going to end up creating Kafka and Elasticsearch as well. So if you want to do just deploy to PCF, then that's okay. So the idea that we want to do is on top of this is we'll add the security checks. We'll add whatever bits that are currently manual processes. In here is automated checks. But if we use something out of the box and is that availability, is that feature available? So it's kind of the trade-off. Anyone else? Yeah, yeah. We just end up calling different APIs at different points. But the idea is we do a CF push with a lot of the no options. So dash dash no, pretty much everything. And then we'll assign a temp route to it, which is really all we need. And then eventually we'll assign the real route to it, delete the temp route, and delete what the real route used to be pointing to. And then we'll clean up any applications that were there from failed deploys in the past.  In the same way we piped together a bunch of use commands, it's really the way we did it. Yeah. So the question is how long did it take us to get there? Yeah, so probably about three months from the time we'd finished the second pipeline to here. One of that, quite frankly, was spent deciding which direction we wanted to go. Like making decisions, I stand up here and say, we're using multi-repo. But getting to the point where we decided that that's the right thing we want to do, we want to do Jenkins, and we want to use the GitHub org plugin in Jenkins, and we want to use Docker. These sort of questions, and how much control do we give to developers, was really where we spent a lot of our time. The coding it up wasn't terribly important, terribly difficult. The hardest bit was kind of figuring out what we wanted to do. There you go. Yeah, so the question is, did we look at concourse? Yeah, so there are kind of two parts of the cloud team at RBC. So there's the platform team, which runs Cloud Foundry and Kafka and Elasticsearch, and they use concourse for that. For the development side, we didn't have enough experience, and I played around with concourse a bit, and one of the problems I found was that you have to build, and I don't know if it's better yet, but you have to build a lot of the things you get for free out of the box with Jenkins. Stuff like having a workspace that carries through all the way, like on all your Docker containers. Stuff like that you kind of get for free with Jenkins, and we had a ton of experience. I think between us, there was something like 30 years of administering Jenkins, so we had a ton of familiarity, so it felt silly to throw that away when we had so many other questions to answer. Yeah, go ahead. Yeah, the chatbot. So the chatbot isn't implemented, so this is all kind of theoretical, but I will talk about how the chatbot works. So the idea is a PCF service that would talk to, in this case Slack, and the idea is it would say, hey, your build's done, and then hey, your deploy's done, et cetera, and then for higher level environments, you would ask it to deploy that application. That would be your interface. So instead of going to the UI in Jenkins what you would do through a chatbot. So that's the idea. Anyone else? Awesome. All right, thanks so much for listening. Appreciate it.


I'm Pamela Meyer and I am an expert in deception detection. I've written a book called Live Spotting on the science of spotting lies. And I run a company based in Washington, D.C. It's called Calibrate. And Calibrate does deception detection training inside threat mitigation training. We also do interviewing and interrogation training. And we also do private consultation and work a lot with banks, insurance companies, and other financial service providers. We are actively working with a number of corporate clients to help them both figure out whether or not their employees have integrity or get to the truth in the workplace, as well as to train their employees in deception detection. And so every top C-suite executive today is so concerned that someone may be ex-filterating data, selling it to the competition, trying to harm their assets, trying to do something that may harm the company from within. If you talk to any of the sophisticated CEO today, that's what's plaguing them. They may have cyber figured out, they may have figured out how to protect the potential attack surface inside the infrastructure of the organization, but they can't keep tabs on every single human in the organization. And the human threat is really, really very serious. Converis is on the precipice of disrupting the world. It really is. Converis products really will help provide a world of much more integrity, honesty in the workplace, and the most important thing is trust. We should all be working with people that we trust, doing business with people that we trust. And I believe that the technology is going to assist us in creating a better world. The CEOs that are out there, that are really visionary, are very, very focused on the human beings in their organization. They may have it all figured out in terms of cyber security and protecting the attack surface, protecting their infrastructure, but they know that what keeps them up at night is the possible inside threat. So they're very focused on recruiting top-notch people and people who really do have integrity. To the extent that you have both pre-screening tools and technological tools to assist you doing that, you're going to end up building a much better company and one where people are going to want to stay.