Great, thanks everyone for coming. Today we're going to talk about decoupled open search. So really quickly who am I? I'm Adam, one of the lead developers at Previous Next. I've been doing Drupal since about 2010 and mostly involved in Australian government and higher education projects. So today we're going to cover the architecture of the project that I'm talking about and why we sort of chose it. Some of the search API open search sort of Drupally set up for it and then a deep add-in to the React technology or front-end technology that we used and that's my two sons. Cool, so first up what actually is open search? Well it's basically an elastic search and probably at least one time accidentally called elastic search. It was forked from elastic search in 2021 and it's developed and managed by AWS. The good news is all the sort of documentation, queries, Stack Overflow questions, all of that that you'll find for elastic search pretty much apply to open search. It was forked from elastic search 7 so you're all good there and it's an AWS managed service so you're able to run and scale a highly available open search cluster without really having to worry too much about how you manage it. So why did we go with this stack? Well we got a set of designs and they were completely full to the brim with search interfaces. All of them had different levels of content, filtering, sorting, rendering, multiple search apps on the same page, all this sort of thing. We even had stuff like landing pages where the filters weren't actually attached to the results. So a lot of different moving parts going on and these are some of the examples of the design. So we have landing pages as I said with the filters detached, we had global search results and all sorts of different things going on. We also needed to be able to have multiple apps on the same page and be able to apply filters to just one of them or maybe all of them. So how do we go about doing that? Well we went to, we went with React apps that were embedded in Drupal pages. We like to call that partially decoupled. We wanted to keep using Drupal's mighty CMS with all the fun tools that we know and love, the site building, the workflows, the out-of-the-box editor experience and just building what we know and love. But we also wanted to be able to iterate fast on these search apps and be able to build a design system that we could then use directly in our apps. We wanted really, really fast search and we wanted to avoid round trips to Drupal for querying and filtering and sorting and it also allowed us to avoid using views entirely. If anyone's ever used views for search interfaces, you'll know they're a bit of a nightmare and theming can be really, really rough. So we went with the search API open search module, which Vera and Kim Pepper wrote. So thanks very much, Kim. This just implements the back-end integration for open search, mapping your entity fields to the JSON documents that then get index and open search and manages all your index settings and field mappings and all that sort of thing. Essentially it's going to allow you to turn a piece of content into a JSON document and then that gets sent to open search to get indexed. So anyone that's used search API before would be pretty familiar with this kind of screen. This is where you're configuring your fields for your index and you're adding all your content entity fields down there. You're choosing which field types to use for each of the fields, how they're boosted and those field types are going to then determine how that content is indexed. You have things like full text, in-gram, edge-in-gram for full text searching and we'll cover that a little bit later or string fields for term filtering or alphabetical sorting. So I quickly want to cover some indexing tips and tricks and how to get really relevant results because often when you're implementing these kind of search interfaces, relevancy is one of the key criteria to get right. So yeah, we want to provide relevant results with minimal developer overhead and we want to also be able to provide contextual filters to users so they can filter those results and get more drilled down what they're actually looking for. So we went with an aggregated field. So with search API you can configure a single field that aggregates multiple fields into one. In this case here on the right there you can see we're concatenating the values of the title, the body, and the summary into one field and that's what we're actually going to use for full text search. We use the edge-in-gram data type and we'll cover a little bit more on what that actually is in a second but basically it helps us tokenize up values and makes content more searchable. And then also at query time we boost other fields in the index for better results. So again we'll cover more on that in a little bit. So what actually is an ingram or the ingram tokenizer will break up your content into words and then it creates ingrams out of those words at a given length. So if we say Drupal South with a length of two it's going to create all those little ingrams for us to search over. So you can kind of think of ingrams as like a sliding window that moves across the words and creates little sequences of characters. So on the term names for filtering it may seem a bit counterintuitive. Usually you'd use an ID to filter your content, makes it more robust, but using term names means we can use open search aggregation queries. So basically what that allows you to do is get a list of possible filter values directly from a single query and then you can use that to render out your filters. Again we're going to show more on that in a second. The caveat there you might be already thinking is well if I change a term name, that term name then has to be updated on everything that's tagged against that term. Thankfully search API actually has a little setting that makes, that gives you that for free called track changes and reference entities and it works flawlessly, haven't really had any issues there whatsoever. So this is an example of an aggregation query in open search. So we're saying give me an aggregation called organization size and aggregate on the field organization size. That's going to respond with a organization size aggregation. Again you can have multiple aggregations in one and that's going to give you back a list of what open search calls buckets with the term name and also a count of how many documents are tagged with that term. So really handy for building filters. And you can use these aggregation queries alongside filters as well. So you can build up contextual filters using this pattern and really praying to the demo gods that this works. But this is a little demo of our filters in action or the whole app in action really. It's going to work. Watch this. It's a live demo. So you can see as I go back to one, yeah, one really good thing to note here is as I go back to filters that I've already applied, the response is almost instant. And that's using a library called Tan stack query which we're going to cover more in a second. I'm going to oops. Play that really. No, I'm not. Never mind. Basically though, as you return to filters that you've already applied the response is almost instant because you're just getting cached results based on the query. So how do we get relevant results? We said, okay, we're aggregating all of our content into a single field. But what if we want to say boost content higher if it's got the search term in the title field or some other field? We can index those fields that we want to boost separately. We can index them as full text and then in search API is field interface that we saw before, we can select a boost value for that field. And then we can use an open search query called a should and we can combine should and must queries into our search into our search query. And that will basically give us more relevant results. So having a look at a full query.  But we'll go through this in a little bit more detail. So up the top, we've got our must query. So every thing that returns from the search search query, we're going to only match things that have COVID in the aggregated field. And in this case, we've called it text. That's what that little key there is. The next part, the next chunk of this is filtering on a term name. This in this case, it's the resource type field. And we're filtering on policy policy and legislation. And lastly, we have the should chunk, which is what I was mentioning before. This is going to say boost any documents with the word COVID in the title for text field. And what that's going to do is increase the relevancy of results that match that, but it's not going to filter out anything that doesn't match. So really, really handy for getting more relevant results. Cool. So that kind of finishes up. No, there's one more.  It's might be pretty hard to handcraft that yourself. So we use a query builder called elastic builder. Again, this works fine with open search. This basically lets you build up a object that represents your query, and then directly cast that to a JSON object that you can post directly to open search. All the links for that are on the slides by the way will be at the end. And these slides will be uploaded so you can get them from there. So looking at what a query builder function might look like, this is basically what we do might be a little bit hard to read but in essence, we build up a list of filters, and we add on our full text query, we add our should fields, and then we use that library, the request body search function from that library, pass all that stuff to it and then just call to JSON and that gives us the query object that we just post directly to open search. That ends the little tips and tricks part. So how do we actually get these apps into Drupal? Well, we just use layout builder. And this is an example of a landing page that we've got on our site. We've got two blocks embedded in the landing page. We've got the header block at the top. That's actually got a search app embedded into it, which is showing all the filters using those aggregation queries. Then we've got another search app down the bottom, which is showing our results. In fact, that's also using an aggregation query to show those little buttons filters or pill filters, as we like to call them. So these are just two block plugins that we've got in our project. And they are very, very simple. So we just have a build function, a little bit of config. And essentially, all we do is give a unique ID to our application. We use Drupal settings and we marry up the ID with the ID that we pass into Drupal settings. We pass in some properties that are going to be injected into our React app. So that's the heading description and index URL. And then we attach a Drupal library. And that's just going to be a very, very thin wrapper around Drupal behaviors, Drupal settings, and actually React itself. So this is an example of one of our library JavaScript files, literally just three lines or one line, really. We use this React component renderer, which is a little custom library we've built. I won't show you the code in that, but it's pretty simple. All it does is take that ID, marries up with that Drupal settings that we saw before that was injected by the block plugin, and then passes those settings as properties to a React app. And in this case, our React app is called all resources app. So on that note, we're going to dive into the React text. So this is getting into the more complex part of the talk. You definitely don't have to understand what's going on there. We're going to dive into each part individually, but this isn't just showing an example of that full React app that we were passing through into our library and marrying up with those Drupal settings that were embedded through the block plugin that was embedded in LayerBuilder. So yeah, we're going to cover all these technologies. Just a side note, I'm not actually a front-end developer, but I have learned a hell of a lot on this project. And because of how we've actually set up all these applications, I'm actually able to work on these apps without being a total expert. But big shout out to Jack Taranto for setting up this whole architecture. So first up, we talked about Tan Stack Query. This used to be called React Query, but now it works with View and other frameworks. This is how we get that really nice, those cached results when we're filtering. It's just a really powerful asynchronous state management tool for React, View and other frameworks. Out of the box, it provides really good caching via query keys. It'll do things like D-Drup requests. So if you have multiple apps on the same page that hit the same API, it's not going to do two requests, it's just going to do one, has auto retry for errors, and you can do really nifty things like updating stale results in the background. So if you do a post and you've got an updated list of results, you don't have to then, again, query those results to update your app on the front end. You can just do that in one callback and it's all handled for you. It's also got really good error handling as well. So this is an example of using Tan Stack Query and actually hitting open search. So up the top we've got, we're getting our index URL and our request body from context providers, which we're going to cover in a little bit. And then lastly, we're using the use query hook, which is from the Tan Stack Query Library. That has two arguments. The first is a query key and you might be thinking, hang on, there's an argument called query key in that array. That's actually something we pass in as a property and then we add the index URL and request body. That gives us a unique query for the combination of filters and full text search, which means our results are really nicely cached. The second argument to the use query hook is our callback, which is going to actually return the data and we'll have a look at what that looks like now. So this is our elastic fetch callback that we're using in the use query hook. We literally just get the index URL, the request body from the query key directly and just post that straight to open search and then return the results. All right, so context providers, if you saw that big application that I showed you, almost everything in there was a context provider. I mentioned the index URL and the request body before they were coming from context providers, but essentially they allow you to access data, any type of data from any component in your tree that's underneath the given context provider and you don't have to pass properties all the way through. You can use them for really simple data, like a string, which is an index URL, or you can use them for much more complex stuff like a whole set of search results. And Eric is doing a talk on understand react hooks later today at 20 past one, I think, and he'll cover this kind of thing in more detail. But we can have a look at one of our context providers here, which is the index URL provider. This wraps a huge number of our components. Basically, we just give it an index URL. When we call use index URL, we basically just get the index URL back. We don't have to pass that all around in our application. And the previous example that I showed you of the query that was hitting open search, I knew I'd do it at least once, that was actually in our search results context provider, which I won't show you because it's too big for the screen. In fact, everything in red here is a context provider. That middle all resources layout component is the only thing that's actually rendering anything in our application. Everything else is providing data that is then used inside that all resources layout component. So context providers very, very, very useful. And using a context provider is literally just this. We go use search results and we get our search results back. And any of our components underneath that context provider. Cool. So next up is getting to the sort of more limit, the higher limit of my knowledge on this topic, but I'm going to try and do it some justice. So Redux is our global store for our data. In essence, Redux has one store to rule them all, hence the DEF star. But our initial prototypes on this project actually went through that. We're going to have one store for each app. That's going to allow us to have independent search apps and we're not going to step on each other's toes if we didn't want to. Turns out, yeah, don't do that. It's a bad idea. So instead, we use something called slices, which comes from Redux Toolkit. So slices basically allow you to split that store up and you can think of it as slices of a pie. Split up the state into little features or applications and that gives you actions and reduces to fire just on those apps. So it allows us to manage that state like we needed to for each app independently. And again, Jack, thank you so much for setting all this up. This is kind of how you can think of a slice.  It's cut up into into separate slices for each application. And then those slices can be used in conjunction with the context provider to pass down that those actions and reduces and state into the lower level components and they can all work nicely and compartmentalized. So this might be an example of one of your apps. You have this slice provider, which will cover more in depth soon. You reuse the same components in there. The results per page, a pager, filters, whatever. And that's just going to work on just the slice and that's passed into the slice provider. It's not going to affect anything else. So creating a slice looks something like this.  So in our search apps this might be things like the current page, how many results per page, some of the filters that are applied. You could have filters applied by default and reduces, which are essentially stuff that you're going to apply to that state. And of course those reduces can be reused across multiple slices, but they're only going to be able to apply them only to one slice at a time if you want to. So setting up the store, we just combine all our reduces from our slices and then create our store from that. And how this is all wired up and the real magic behind all this is the slice provider component. So this is a little custom context provider. It basically wraps all of our apps components. You pass it one of those slices and then it's going to give you back actions and state for the app to use. That link there is to the much more detailed blog post by Jack, which I highly, highly recommend you give it a read. It's really kind of the bread and butter of what makes this whole project hum. And yeah, it's pretty, pretty amazing. So next up, how do we test all this stuff? Testing is super important. It's pretty hard to test all this stuff into end, but we can do some pretty amazing unit testing combining these libraries. So we use Jest, which is a kind of a unit testing framework. We use the React testing library in Jest for assertions and rendering and so forth. And we use something called MSW or mock service worker, which allows us to set up mock responses for OpenSearch. So this is an example of a test for our search app. So we render the app at the top and notice we pass it a mock index URL. That's going to help us in a minute with mock service worker. We wait for our filters and our results to render. And then we just match it against a snapshot and that snapshot is stored in the code base. So things like CI or other developers will be running these tests against the same snapshot. So very easy to know if anything broke. And they're very fast. Cool. And then the mock service worker. So this is where we basically set up the MSW. So we're saying in this example, we're intercepting post requests to slash mock slash all resources. It is reject. So it's a little bit hard to read. But that was why we set up that mock index URL previously. Then we're saying if there's a eggs key in the body, if you remember way back at the start of the talk, we talked about OpenSearch aggregations. That's the key that's going to come through in a query to build those filters. We say if there's an aggregation key in the query body, we return our mock eggs. It's just a JSON file. Otherwise we just return our results. And that's going to allow that test to then render the app with mock results without hitting like a real open search instance. Cool. So tying it all together under the hood. We had a content index by Drupal with search API. We have index that's queried by React at the front end. So how does that actually all tie together and how do we keep it really fast and most probably most importantly secure? So we use OpenSearch on Skipper. So Skipper provisions some credentials for Drupal that can then write to the index. And then our React apps go via a proxy and execute searches against OpenSearch. But the front end can only read that via that proxy. So it has no right access at all. So we have the proxy for a number of reasons. It makes it really really fast because the instance almost looks like it's local. So we have like a slash API proxy. That's what React is hitting that thing gets proxied on to OpenSearch. It's secure. The front end can't write anything to the index. Only Drupal can. And we don't have to expose any OpenSearch end points or anything on Skipper. Cool. And if you want to try that stack out yourself, we've got a OpenSearch image that you can spin up in Docker or anything else.  Cool. So recapping all of that, we use search API OpenSearch as I said. We inject config into our React apps via block plugins that are embedded via layout builder. We query via React apps using a read-only proxy that hits OpenSearch. And we smooth all that developer experience out with some pretty nifty React tooling. Here's a giant list of links. That's it. Any questions ladies and gentlemen? You there. I think there's a microphone coming. We are using the managed AWS service. It's hosted on Skipper. So Skipper will essentially, if you're using Skipper, you can provision one of these OpenSearch classes really easily and you get all that tooling, like I said, just out of the box. So it is using AWS's service under the hood. But it does things like provision the writer credentials, all that kind of stuff for you. As you said in the presentation, you use itching ground. And what if you have a requirement that a full phrase needs to go above all such results? How do you handle it? Because worth itching ground might have more instances in another entry where it's not a full phrase. Yeah, so we don't tokenize the query string. I think that's the most important part. So when we're actually doing the query, you have to specifically set an analyzer on your query string so it doesn't break those tokens up. But your question was how do you how do you ensure that the full phrase goes above everything else? Yeah, I guess you'd have to index that separately. As I said, you can set up as many fields as you want, right? So you might have your main text that's fully tokenized and then you can boost separate fields or separate strings with different fields. The second question is, one thing I've noticed in solar at least, boost values isn't the only thing that boosts it. There's behind the scene tools that boosts it like normalization and other things which you can't directly control in such API. How do you handle it in open search? Yeah, so there is, I did actually forget to mention that, things like the number of instances of this of the search term. So if a piece of content has the word COVID and the title and the body and all over the place, that's going to get boosted higher. Yeah, as you said, and internally in the black box, so to speak. Yeah, it's a good question and it is something that you'd have to iterate on a lot. I think there is always a lot of fine tuning and fiddling with little knobs and things like that to get relevant results, like really precise. But I think as soon as we added that should boosting, our clients were much happier with the sort of results they were getting and we've only just started sort of diving into that stuff. So I think, yeah, there's definitely a lot more that we could do there for sure. And the final question is, does facets work with open search as well? Yeah, I think aggregations are basically like a one-to-one to facets, yeah, Gibran. Have you tried direct field boosting instead of shoot boosting? What's the difference between shoot boosting and the direct field boosting elastic or open search? Nope, I haven't tried that. But yeah, I think this should helps you sort of boost stuff at the query time as well. So you can have different like boosts for different queries depending on your requirements, yeah. Sorry, how do you handle sediments and stuff like that? You spoke about search conflicts and understanding what person was off to. Yeah. Yeah, so Lee actually recently built Lee Rollins built synonym support into the module. So you can configure it at an index level if you want, but there's also more dynamic stuff that you can do which we're implementing where the problem with open search as far as I can see is that you need to index the synonyms as settings on the index. So they have to be they can't. Yeah, as far as I can tell why I haven't dove into it too much more than this, but you can yeah, you index the synonyms at the index level and then when you query it'll just automatically pick those synonyms up and use them. Yeah. There's also stuff like auto search suggestions and things like that that you can take advantage of. I just wanted to ask about the architecture and security of the cluster endpoint. So you are using a query builder to create like a payload from the reactor, right? And then you post it to a proxy and I'm just thinking like does it expose where does the proxy live? Is it in the middle way of the react or? No, it's an index on our cluster. Oh, gotcha, gotcha. Yeah, yeah. And then it kind of forwards the back to that's right. Behind the scenes. Yeah. Yeah, in the regards to this search AP attachments is the response the same in terms of you know, having the flights data extracted and being indexed is the response the same when it comes to the files being attached? I assume so. Yeah, I haven't I haven't tested that module out. But yeah, as long as the that index is other extracts the that's the you're talking about when it extracts AP attachments. Yeah, extracts the PDF content and index is that yeah, I can't see any reason why that wouldn't work. Yeah. Right. Thanks. Cool. If you have any other questions later on, I'm pretty easy to spot. So just yeah, hit me up. Thank you.


So there are 30 cyberpunk 2077 iconic weapons for these apartment wall collection and in this video guys I have a complete guide on how to get each one for the people out there who want his collection filled. How's it going guys, my name is dpj and if you do enjoy the video, leaving a like really helps out and if you like what you see and want to see more, be sure to subscribe. So these mega building apartment is home to be stashed in which you can store certain iconics which will appear on his 3 walls. Today I will guide you into getting each and every iconic for this wall collection. Now obviously be warned, this video will contain major storyline spoilers. Now everything will be timestamped below, I will try and do them in order in which you can obtain them. But below the timestamps will be in wall order, left, middle and right. Just in case there is a specific one you are chasing, also keep in mind if you have missed one of these iconics there is a chance to black market vendor who comes with the phantom liberty dlc may have them for you which we will cover at the end. Okay so let's get into it guys and we have the knee hand. So this is an iconic which is obtainable from the highest mission real early on. The knee hand is obtained from Siberial's body so do not miss this one. Next up guys we have the Satoru Katana, this one comes out of the same mission of the heist so not long after you have looted Siberia for that knee hand. T-bug you will lose connection with T-bug just before she loses connection with you. She will open the double doors for you to escape but if you don't take this route and if you go upstairs to the helipad area as you will see me doing on screen now guys, you will be confronted by two security guards, take these dudes out, go to the vehicle, open it up and you will find this iconic waiting for you. Okay so next up people we have Jacky's very own pistols. Now these are pronounced in a way I cannot pronounce but I will try it for you guys, entertainment anyway, I think it says a large hang on a Dorado or something like that. But either all guys these are Jacky's own personal weapons. And while to get these guys during the end of the heist mission, well obviously Jacky meets his unfortunate demise, Delamaine asks you where you want to send his body. Now I'm not sure what the other two options mean in this regard but just select send it to his family. Now not long after this, Mama Wells will get in contact with you about basically having a wake for Jacky. This triggers the side mission called heroes. So upon you going here guys and doing heroes you go to the El Coyote Cojo which you'll see right here on the map. When you go inside here you will speak to Jacky's mum. She will give you a garage key for Jacky's lockup. Once you go here guys you'll speak with Misty. You can clearly dialogue out and just simply rush back. Once you get back guys there will be a ceremony going down for Jacky. You need to progress past this part upon you doing this guys. What I did was I mean I tried going outside of this actual bar and skipping time by a few days that did not work.  So yes do what you got to do guys and go get this weapon. You also get Jacky's amazing bike for doing this at the same time. So definitely worth your time in doing this. So next up guys we have the plan B. This is Dextre the Sean's very own pistol. So how you get this thing guys is you have to wait until you complete the mission of love like fire. This is the last mission moving at one. Upon you doing this people you go to Dextre's body where it's found right here on the map and simply just grab this weapon. It's that simple. Okay so next up guys we have the orata. Now this is the thermal katana and this one is found during the disaster peace mission where you go in search of Eveline. This is found right near the furnace as you make your way down. Next up guys we have the butcher's cleaver. So this is found during the mission of mapped and plan. This one guys is found in Roland's butcher shop. After playside finishes butchering the chicken. Simply just grab this after this interaction. Next up guys we have Sasquatch's hammer. This one is obtained from the end of the I walked line main stone mission where you actually meet and can fight Sasquatch. I think previously you had to fight her and beat her to get this. Now if you can sneak around her you can simply pick up this iconic from behind her and progress on further but that choice is yours when you get to this point. Next up guys we have the Jinchumeru katana. I believe this actually stands for the loyal one. Okay so during the main quest after the prologue you'll come to a mission called play safe. This mission consists of you working with Takamura. The mission will consist of you having to take out snipers and then going on to find a net runner. Upon finding the net runner you'll be confronted by a boss called Sandeyu Oda.  So take him down.  Doing so or not doesn't change the outcome of the loot. You can grab from him. So do what you have to do and take him down. Next up guys we have the Mux shotgun. So the Mux shotgun is given to you upon you completing duty side missions. The main mission of Pyramid Song. Now to build up to this mission starts after you complete the missions of Double Life and Map Tanpalan. These are main stone missions. So completing these missions unlocks a side mission called both sides now. Upon you completing this mission the next one is called X Factor. Again guys complete this mission. You then guys have another mission called talking about a revolution. These are all duty side missions by the way. This can take a few days in game though to pop up so skip time if you want to rush this thing. So again here guys you need to progress past this mission. You'll then unlock another mission called Pisces. Again guys you need to progress past this one too. And the final mission to reap this reward is called Pumid Song. Now it doesn't matter if you're a male V or a female V looking to romance Judy. Once you get this mission done guys she will give you this shotgun. So if you're a female V and you did romance her after this mission is completed she will message you saying she left the weapon on the kitchen cabinet for you. If you didn't romance her the weapon will be left on a counter for you. It's as simple as these people. Next up guys we have the Overwatch. So the Overwatch is given to you by Panam during a mission called Riders on the Storm. Okay so to initiate that Riders on the Storm quest you first have to meet Panam. And it all starts with the main story mission of Goldstan. This is I believe the first time you actually speak with Panam. So completing Goldstan and where you actually get to work with Panam triggers a mission called Lightning Breaks. Continue on with this mission guys until it's completed. The next one you'll have is called Life During War Time.  This sets up the mission of Riders on the Storm in which you get to meet Panam and Gold to work. Upon you completing this mission guys she rewards you this Overwatch Sniper at the very end. Next up guys we have the Beijing Chong. Now this actually comes in the form of a crafting spec but it is one super powerful weapon. So to actually get these guys you need to complete one of the game's endings where you kill and defeat Adam Smasher. Upon you looting his body and getting the access token you can come to this point on a map guys which is basically a locked up stash. If you come here loot this room with this access token you can get this crafting spec craft it and place it in your stash and it will appear on your wall. Next up guys we have the Meloria. This is Johnny's personal pistol. Okay so this weapon is given to you via a mission called Chipping Inn. This is a side mission that will pop up after doing four of the main stone missions of automatic love, transmission, life during war time and search and destroy. At the end of each of these missions you will have dialogue with Johnny. This is the important part. You have to select the right dialogue choices where you befriend him so pick the obvious dialogue choices which would no doubt bring the two of you closer together. Okay so once you have that chipping in mission towards the end of chipping in you meet Grayson. Now it's important here that you not take this dude out but Grayson is the one who hands you over this amazing pistol aka Johnny's pistol. Next up guys we have the Arch Angel. So the Arch Angel is another weapon which is tied to that chipping in mission. In the end this one is given to you by Kerry Uradine but you need to get to that point first. So within that chipping in mission you also get Johnny's pistol as I've just covered but if you use the timestamps here I will quickly explain a game. So the Arch Angel is given to you by Kerry Uradine in a mission called Alike Supreme. But you have to select the right dialogue choices to get to this point. So there are 4 main campaign missions of automatic love, transmission, life during wartime and search and destroy. At the end of each of these missions you will have dialogue with Johnny. Here select the dialogue options to befriend him. This will open up another mission called Chipping In. Here towards the end of this mission you meet Grayson. Here whatever you do do not kill him. Here you will retrieve Johnny's pistol. From this point guys you make a deal with Grayson. Here it's important to make the right dialogue choices and select the dialogue option of You Got Lucky Today. You a key to a container which when you open up it's got Johnny's potion side. The mission then progresses where you take Johnny to his grave. Here you need to do as I do on screen now, select the same dialogue options and basically befriend Johnny Silverhand in every way possible. Let's say it was my real grave. What would you write? Here lies Johnny Silverhand. Sounds good, but what does it really mean? What did I achieve shouting Chippin' In to a bunch of angry pimplink? I remember what I thought on stage.  Either let down or used every last person who gave me their trust. But I've managed one thing for now.  But we have. Even stand to be in the same room with me.  And yet you don't seem to hate my living guts. You sorta deserved it. You're not knowing about me. I'm seeing things. I'm scared. Then moaning at Misty's about how you didn't want to die. Remember you whining for smokes in the middle of night? Complaining about not being able to kill me? Never thought we'd make it this far. Santiago? Not always lost yet. At least with Rogue. Can't pretend nothing's changed over 50 years. Can't just insert myself into her life. She was acting weird. Promise her I'd take her to the movies. Good idea. Ask if she's free some night. Thing is you'd have to surrender control. Again. Let's Delta. Nothing to see here after all. Worth coming out all the same? Absolutely. Thanks, man. Of all the heads I could have popped up in, hell it glided was yours. From here you open up another mission called Bless Dream Love. You need to complete this. You then get another mission called Holding On. Complete this too. The next mission will be Second Conflict. Complete this mission to unlock A-Lite Supreme, which should pop up for you 24 hours of in-game time after Second Conflict. So at the end of this mission guys, at the end of A-Lite Supreme, K. Udyne will hand you the Archangel Iconi. Okay, so next up guys, we have the Crash. This is given to you by a river after completing the mission of following the river. So this starts upon you completing the main stone mission of life during wartime. So do this. Now this next side mission you are waiting on could depend on your street cred level.  So the next mission you want here is called I Thought the Lore. This is where you meet the river. Upon you completing this guys, you need to wait 48 hours in-game for the next mission to pop up. That mission is called The Hunt. So within this mission guys, you are basically scanning a farm in search for somebody. But you need to progress past this mission to unlock the next. The next mission is where you get this iconic weapon. It's called Following the River. Here you need to wait 24 hours in-game. River will then call you and you need to accept his request to help. So it's this mission of following the river. This is where upon you completing it, river gives you the crash weapon. Next up guys, we have the Lexington X-Mod 2.  This is found within your mega building apartment base. So go here guys, as you can see me doing on the screen now. And if you haven't already, speak to Wilson. Here guys, you have to get first place in that shooting range competition. So I suggest you make it safe before doing it because I do believe you only get that one chance. Now I do believe you have to get a score of 40 or above to win. Next up guys, we have the dying knight. So upon completing the shooting to fill side mission for Wilson, he now becomes a weapon render for you. You can simply go to him guys and purchase this dying knight iconic. Next up guys, we have Skippy. So Skippy can be originally found next to a body right here on the map. Now once you have Skippy, you need to get 50 kills with him. After you've done this, he eventually will pop back up automatically and tell you he has info on his previous owner Regina Johns. Now upon you taking him back to her, she will after many days of you skipping time contact you again. From here, you can go back to Regina Johns and pick Skippy up. Keep in mind, he will not be as unique as he once was. He won't speak to you anymore, but he is now available for you to stash, which means it removes that mission marker from him, meaning you can put it in your stash to appear on your stash wall. Next up guys, we have the Bayako.  Upon you doing these guys, she will add it to your stash. Next up guys, we have the Seraph. This is rewarded to you from Padre, aka Sebastian Ibarra. Upon you completing all of his gigs, his gigs take place within the Galen and Haywood area. Next up guys, we have the Bloody Maria. This is rewarded to you from Al Capitan Reyes upon you completing all of his gigs. And these all I believe take place in Santo Domingo. Next up guys, we have the Blue Fang. This is a weapon you purchased from the weapon vendor within Jackson Plains, as you can see right here on the map. Next up guys, we have the Head Hunter. This is a weapon you purchased from a melee weapon vendor in Westwinders States, as you can see right here on the map. Next up guys, we have Iconics that drop from suspected organized crime activities. These what I would suggest you do is create a manual save before attempting each and every one of these. These are known to bug out. Okay, so first up we have the Bussaw. This is a crafting spec dropped by Yelena. So this is a suspected organized crime activity called Vice Control located right here on the map. Next up guys, we have the Sarm 11-6. This is also a crafting spec and it's dropped by Tom. Oh yeah, this is a suspected organized crime activity called Just Say No. But in order for this one to appear, you first have to complete the Vice Control suspected organized crime activity which rewards you the Bussaw which is what we just covered. So get that one out of the way and this one should appear on the map right here for you. Next up guys, we have the Moran Labe. This crafting spec is dropped by Anton Kalev. This suspected organized crime activity is called Westwinders States located right here on the map. Next up guys, we have the Headsman. This crafting spec is dropped by the Militech Mech. This suspected organized crime activity is Privacy Policy Violation located right here on the map. Next up guys, we have the Sovereign. This crafting spec is dropped by Shinobu Amaya. This is another suspected organized crime activity called Modern Labour Market located right here on the map. Next up guys, we have the Ying Long. This is a crafting spec dropped by Denzel Crier. This one is another suspected organized crime activity called Living the Big Life located right here on the map. Next up guys, we have the Breakthrough. This crafting spec is dropped by Olga. So this one is a new boss, new rules, suspected organized crime activity and this one is located right here on the map. And lastly guys, we have the Comrades Hammer. This crafting spec is dropped by Darius Miles. This suspected organized crime activity is Oreyo located right here on the map. So what about the Black Market Vendor? So this is a dude that comes with the Phantom Liberty DLC and then you can go to and buy Iconics that you may have missed. Now I know he sells weapons like the Sasquatch's Hammer, the Areta, the Nihan and a couple of others but there's a few on this list that I've put today that he definitely does not sell. But hey if you've missed any of these you can always go to this dude. Skip time by 24 hours to reset his stuck and hopefully guys he will showcase the one for you to purchase that you may have missed along your play through. And there we have it guys all 30 iconic weapons which have a place on these apartment building stash wall. Now some of these at the moment are bugged and won't appear on your wall but sooner or later CDPL will fix these and they should appear on your wall. But there we have it guys if you enjoyed the video leaving a like really helps out if you feel like what you're seeing want to see more be sure to subscribe and hopefully guys I will see you on that next one


That's a good start. Anyway, we have a lot of ground to cover here, so we're going to be very, very fast. And we wouldn't ask questions during the session. This is not one of my habits. But we'll ask a question or we'll let people ask questions toward the end. The topic of this presentation is really to cover different kind of orchestration and try to compare between them. So we're not going to take also a deep dive in each one of them, but it will give you the tool set to actually choose the right tool for the job and hopefully will give you a way to understand how they may. With me is Uri and Dan. And again, because we have a lot of ground to cover, I needed their help and they jumped into the, some of them want to curse me now, but it was an interesting exercise anyway. And the way we kind of structure the presentation is that obviously we'll try to really find this orchestration for those who are less familiar with that, but the bulk of the presentation would be really to try and first of all define how do we compare tools that don't really match the same standard, the same API and even give some sense to that. So that was the biggest exercise. And the bulk of the presentation really to go through each other tool, go through a quick introduction into that tool and what it means to actually manage an application, orchestrate an application with that. I think that the end result of that exercise would be that you will have better understanding of the classification of each one of them and which problem it aims to solve. You will not get the technical deep dive of understanding how to work with it, but you'll get a good sense of what it means. And if that, if we'll achieve that, then we've done a good job. In terms of understanding what orchestration is really all about, I think that the simplest way to think about it is the robots of IT and it takes, in my view, a very similar role as we move to DevOps into the robots in car manufacturing and it's basically to take manual process and automata. That's in a nutshell what orchestration really means. And at the end of the day we have different kind of orchestrations and why do we have different kind of orchestration? But they all seems to have common ground which is basically take a specification, a plan of what we want to automate and then the engine itself process, execute it and give me a result which is equivalent to the process that I would otherwise do manually. That's in a nutshell what orchestrations aim to solve and provide. There are different approaches to orchestration. In many cases it differs because of different assumptions of the architecture, for example. If I'm aiming to build microservices, then I'll build the orchestration in a certain way that caters to that. If I'm trying to be more agnostic, I'll build orchestration differently and that impacts the design quite significantly. Similarly to the scope of problem, are we dealing mostly with the installation and standing up in the environment or are we trying to solve and automate things till production? And that's kind of another set of differences. So in this exercise we wanted to kind of classify into basically three main categories. We call it the infrastructure centric, the container centric and the pure play. On the infrastructure centric obviously we have something like it. On the container centric we'll see Kubernetes and Docker and on the pure play we'll have Tosca and Clarify on one hand and Terraform on the other hand. So how do we compare tools that doesn't comply to the same standards and API? The method that we chose is to take an application, Node.js and MongoDB that all of you probably are aware of even if we haven't touched with it and basically run through the process of automating the process of installing them, configure them, sending them out and getting them into production and seeing how far the tool gets me there and what is the actual process in each of the tools to do that. Obviously we will not be able to cover all the tools and we actually spend only five minutes on each tool so we cannot compare all the tools in the market. We chose a selected list of tools in each category and that's what we've focused on. We haven't selected paths or other things on that regard. Now let's look at the actual test application and for that I'll hand over to Uri. My colleague here will explain how orchestration really looks like. Okay, thanks Nati. So basically we took a kind of simple Node.js and Mongo application to make it more production-like. We chose to use a full-fledged monocluster so you can see here replica sets at the bottom, monoconfiguration servers. There's also another infrastructure part of things where we basically provision a network and a subnet and hook the VMs into that network and then create an LBAS pool through OpenStack and register the Node.js instances in it. Here you can also see security groups. So basically all the infrastructure resources. Now let's just go through the setup process. So the first thing we need to do when we create such an application, deploy such an application, is create the resources, the infrastructures. So again VMs, security groups, networks, load balancer pools and all these stuff. Already here you need to do things in a specific order. For example, create the networks and subnets before you create the VMs. Same goes for the security groups. There's already some sort of orchestration here. Then we need to install the software binaries, Node.js and Mongo. If we're using Docker, that's just down on the images from some repository. If we're using non-dockerized workloads, we just do yam or iptget or unzip something. Then we need to start starting the processes, right? So we can't do it all at once because some processes depend on one another. We need to be configured after some other processes have started. So we start the Mongo D processes in the replica sets. Then we start the Mongo config processes. Then we start the Mongo S processes, but we need them to be aware of where the Mongo config processes are located. So there is some information passing here that the orchestration tool needs to do. Then we need to configure the replica sets. Here we need to pick only one instance and designate that as a master. Add the slaves to it. Then we need to add the shards one by one in Mongo. This is actually done as a command on a single Mongo S instance. It's pretty weird, but that's actually a good example for something that's not very symmetric and pretty hard to orchestrate. Then we basically initialize the data if we want to start the Node.js processes and register them with the load balancer. This is only setup. This is a very complex process that we saw for quite a simple application. This is only setup.  So for example, if some of the pieces of the stack fall apart, you only want to fix them and all other pieces and make sure that the dependent components are ready and know about this fix. Then we go into kind of post-deployment aspects like continuous deployment. If I want to push new code, for example, I need to do it, let's say, two instances at a time. If that fails, I need to roll back. If that doesn't fail, I need to push the change to the rest of the instances. If I want to apply infrastructure patches like an OS patch or something like that, I need to take down instances one by one from the load balancer, apply the patch, start them again. So these are all, again, pretty complicated processes in the same way that I modeled the initial setup process. You need to model these processes. That brings us to kind of a few common requirements that we want from an orchestration tool. So we want them to be able to be able to be in dependency management, right? So we need to be able to say to the orchestration tool, create this before this, tell this where this is located. We need them to be reproducible. So we need to be able to run the same plan over and over again and achieve the same end result. We need to be clone-able. We want to run the same plan side by side multiple times so we can create the same plan for QA, for pre-prod, for testing, for production. And we need it to be recoverable. Recoverable means that if my execution fails for some reason, I don't know, I exceeded my quotas or something breaks in the middle, I need to be able, even as a manual step, basically clean up the environment and start over again without failing. So with that, I'll pass to Dan, who's going to start to talk about the Docker aspect of things. Hi. Okay. So we'll be speaking about Docker Swarm and Kubernetes. We'll start with Docker Swarm. It's a native clustering system. It's written by Docker. Basically, it's a pool of hosts that all have a Docker daemon installed on them. And from the client perspective, you use it as if you were using a single Docker daemon, but it has features to schedule containers with requirements. The architecture is pretty simple. You have the swarm manager, and you have the daemons. As I said, the client speaks to the manager, and as we'll see, you can specify certain aspects of this scheduling. So if I compare it to all the points that we just mentioned, it's pretty low-level if you use tools like Docker Swarm. Basically, you have just this huge pool of instances, and you just deploy containers one after the other in the correct order. So to highlight a few points, for instance, when we create the replica sets. So let's say we have N replica sets for each replica set, for each member of the replica set. First, we simply just deploy a container that has MongoDB installed on it, and we specify the command. Now the things to note here are, you see, we say affinity and constraints, which this is where the real power comes from. So we say, do not place this container near any replica set instance or near any configuration instance because we want high availability, or for instance, do place it on the daemon labeled MongoDB because the image is already there, so it'll get deployed faster. Yeah. After we start these processes, for instance, in this situation, we want to configure the replica set. So for each replica set, we still have to somehow manually run the initialization code, either by a sage or doing some docker exec on one of the containers. So we will use a command such as docker inspect to extract the IPs of the containers. Okay, another, so another highlight, let's look at the Node.js application container. It's pretty similar. One note here is that we need to inject, for instance, a set of IPs. Now, it does have the linking, but it's still not very supported with regards to multiple hosts, and so we need to inject it. So that's another manual process that you need to have, like, abstract, like a layer on top of it to manage it for you somehow. For instance, there's nothing very new here. So say we have an AJ proxy container. We want to dynamically add and remove members. This is a certain way that you can accomplish this by having a script on this container and you just execute the commands. If we're looking about scaling, nothing very special about it. It's very similar. We didn't see the deployment process of what you would do, but it's pretty much, you just say, okay, we have three replica sets. Say I want a fourth one. I just execute the same commands with a new index, basically. So this was pretty fast, because it's pretty not reasonable to cover the entire process, so I'll just say what were my thoughts about this process. So the really nice things, first, is it's, like, pretty easy to use. If you know how to use Docker, you just start using it. If you know how to deploy the application with scripts so you can model this same process with containers and you just do whatever it takes to do it in the correct order, but there's nothing really new in regards to learning something. A strong feature is the placement affinity. Like, as you say, you can easily achieve high availability of different components simply by saying, don't put this where there are similar containers such as that. It's a strong feature. Okay, we did, we said nothing about setting up this cluster, so we did not handle infrastructure at all. So you can use Docker machine, for instance. You can use, we just saw a magnum that can set up a swarm cluster for us, so it should be pretty easy. What else? Okay, the process was, okay, many, many steps. So the replicas, so if you want, to add more members or remove members or add shards, we have to manage all these indexes by naming of the containers. There's no, like, something, like, we need to write this layer to manage that. It's pretty, there's no workflow or anything that you run the scripts. And yeah, so you see, like, requires other tools. Basically, it's all these other aspects, like, such as monitoring and healing and autoscaling and whatnot are handled by other tools, not by Docker swarm. Okay, I hope it's not, like, I hope there's added value here. So we'll move to Kubernetes. It's another container cluster manager. This one's by Google. We'll highlight, there are some key components to talk about. So pod, basically, it's just a set of containers that run together. They share the same network namespace so they can talk to each other using local host. They share volumes. So for instance, you'd have a Mongo S and an OJS application running in the same pod so they can speak with one another through local host, for example. This would be a use case. The application controller would be a way for you to achieve healing. First, you specify, like, how many replicas you want and Kubernetes takes care of the rest. So it's a common pattern to see a replication controller that wraps a pod with one instance and this is the way to keep it alive. Say one node fails, Kubernetes will take care of moving that to another. Networking-wise, each pod gets its IP. So we said, within the pod, they speak through local host but outside they get their own IP address, which doesn't matter on what host it's running on. This gets abstracted. And the service is the last key thing in Kubernetes.  Say we have the node.js. For instance, we have many instances of that pod. We'll wrap it with a service and it also gets an IP, which is static, and we can address it throughout the lifecycle of the application. So architecture-wise, I won't dive into it, but it's pretty similar. You have the manager node and you have the minion, which are hosts that we have agents running on them that do the bidding of the manager. This is how a replication controller will look like. For instance, you say we specify replica as five. I want five instances at all time of this pod, and for instance, this pod is a MongoDB replica set instance and it has some administration node running alongside, maybe to dynamically change the cluster as members get added and removed. Service, similarly, you'd say I want a service for all pods that have a label type node.js. When you get request to port 80, delegated forward it to some of the instances on port 8080. Okay, so we won't look into the files themselves, but as we said, we need to start config servers. So we'll do 123 kubectl create controller and we'll create a service because we want a static IP for these config servers, because the router needs to address them at all time with the same address. We'll do a similar process for the router. We'll start a service. The service in this case will be used by the Node.js application. So it's easy somewhat to model this step. Nothing very new to say about it. For instance, this is where we create the replica set itself. So in this case, we can model each replica set as a whole in a controller. Let's say we want five replicas. After we create the controller, we say we wait for all the instances to start. We can extract the IPs. Again, we need to manually, at one place, initiate the replica set and so forth. Regarding healing, it's something that gets provided by Kubernetes itself. Once we've wrapped it with a replication controller, if something breaks down, it simply gets rescheduled in another minion. Not much to say about continuous deployment. Besides that, it's pretty simple to, if you want to switch between versions, for instance, if you want, you can start a new controller with zero instances and move pretty easily from one to the other. And if something breaks down in the middle, it's pretty easy. You can just say, okay, forget the new controller. I want to roll back to the old one with a number of instances that were on it. So we'll tie things up with Kubernetes. Regarding auto-healing, you saw it's a pretty simple process. You can just wrap it in a replication controller. Load balancing, I actually did nothing. It was just telling Kubernetes to load balance a certain service, and it just works. Scaling, you saw, it's a simple matter if your application can handle it. The new increase, decrease the number of instances. It just works. So we didn't say anything in placement in comparison to swarm. That's because it's just not implemented yet. I saw that they're working on it. So it will probably get there at some time. These are pretty new tools. And with regards to orchestrating stateful services, as we just said, there are many steps. It's hard to automate such a process and to orchestrate it, so you need to do things on your own. Kubernetes itself does very little to help you in that regard. And with that, I will pass the stage to you. Thanks very much, Dan. All right. So this stage is about pure pro orchestration. So Terraform, anyone using it here? There was someone here. Yeah, there is one. Cool. Two. Okay, great. So Terraform is a relatively new tool by Hashicorp. It's a simple and a good way. Basically, it's a command line tool that you basically feed your plan or configuration into that. And that command line tool basically executed the configuration for you from the laptop you're running it from. So there are no servers or things like that. It basically has a few basic terms. There's a notion of a resource. If you work with Vagrant, that's pretty similar to that.  There's pretty extensive support for OpenStack resources, which is fairly nice. There are providers. So providers are implementations. Really the code that implements resource creation. So there's a provider for OpenStack and Nova Network, for example. And there are provisioners, again, very similar to Vagrant or to Packer. Provisioner is something that runs after a resource has been created. So that can be a script. That can be a file copy. That can be a chef run or things like that. Terraform has very nice support for modules. Modules are basically a way to compose configurations. It's like nested stacks in heat, for example, or substitutions in Tosca. So you can model individual pieces of your stack and share them between configurations. And there are variables and outputs, which are analogous to heat inputs and outputs, or to Tosca's inputs and outputs, basically allow it to interact with runtime and calculate the values. This is Apple configuration. So you can see here, for example, this is the type of the resource.  It's not really JSON intentionally. They want it to be human readable. But you can also feed just pure JSON into Terraform. It will know how to digest that. There are a few examples here. It's all kind of similar. The resource type, the resource name, and the parameters.  This is kind of an expression language or interpolation language that you can express, for example, that this subnet network is the resource of type networking network v2. And this is the name of the resource that we created over here. And this is the idea. This is runtime calculated value. And that's how I create dependency between elements in my configuration. So the way we modeled the Mongo application is we had a single top-level configuration file. There's a link here. We're going to share the presentation. And you can access that online. It's on GitHub. It creates all the infrastructure for the application that I mentioned, like subnets and routers and networks and so on. We actually did have a module here. We modeled a Mongo Shard as a separate module within Terraform. One thing to note here, there's no easy way still in Terraform to say I want x instances of that entire module. So you basically have to copy-paste it. So for every shard you want to create, you need to create a section in your configuration file. That's something, I guess, they're working on. And still we're left with kind of the hard issues of master assignment and registration of shard, which I mentioned before. There's no kind of cluster-wide operation to invoke provisioners in Terraform. So you can resort to a number of ways. I guess what the HashiCore guys are kind of encouraging, that's also in the mailing list, is to use console. Console is another tool by HashiCore similar to ETCD or to ZooKeeper, which you can kind of use to coordinate multiple instances through locking and acquire and wait semantics. We can use that to do that using scripts, using provisioners. You can use static allocation, so you can separate within the configuration the master from the slave. So you have a separate entry for the master and separate for the slave. That can work, but it's not really good because these things are dynamic and can change over time, so the configuration becomes stale as your cluster evolves. The third way is also to use a mechanism of local execs, but this also requires you to kind of use certain tricks on the local machine, like file locking and things like that to be able to orchestrate that properly. So if I have to summarize Terraform, the pros here, it's infrastructure and framework neutral. You can actually do anything with it. If something doesn't exist, you can basically create a provisioner or a resource provider yourself. It's pretty easy if you now go. It's very solid support for OpenStack, which is, I guess, where everyone's here. Very simple to install, very elegant. One of the things I really liked about Terraform is that it has a way of planning things. So essentially, you don't just create a plan and apply it. You can do a preliminary planning step, which will give you a list of all the things that are going to happen, and then you can see exactly what is going to be applied before you do that. In the same way, if you actually change a plan, Terraform actually knows to kind of calculate what needs to be changed in your infrastructure, present that also as a plan for you in a very elegant way and do that. On the other hand, configurations are not portable across providers. So for example, if you use Kubernetes or if you use Tosca, you can take that to any environment, because these tools work on any environment. They're not specific to OpenStack, for example. With Terraform, you need to create a separate configuration for OpenStack and a separate configuration for Amazon, and if that's a concern, you need to take that into consideration. It's kind of hard to mod non-infrastructure components, as we saw before, things that are not symmetric by nature. This takes a little effort and makes you go outside the tool. And all the operations are kind of part of that are done in the context of a single resource instance. If you want to run something, you only can run it in the context of a single VM or a single network and so on. So that's Terraform. I'm not to cover heat. Thanks. Thanks, Surya and Dan. So Tosca is really a neutral specification language for defining blueprints. As opposed to the other tools, it is a language, which means that I can create types. It's fairly extensible. I can inherit. I can create interfaces and so forth. And it has a lot of language characteristics that applies with it. The other thing that is important is that it does tend to look at the stack from a holistic way. So it will deal with infrastructure in a similar way to the way it will deal with software aspects of that stack. Clarify is an implementation of Tosca. And Clarify, and I think that's kind of the unique position as two main characteristics that I think are unique to Clarify versus the other tools is that it takes an integration approach to orchestration. Integration approach means that it doesn't assume that it owns the tools that it manages. It gives you a lot of set of plugins that could enable you to plug in configuration management tools as well as infrastructure tools, whether it's cloud, open stack, VMR and so forth. The other thing that is pretty unique is that it does focus a lot as opposed to the other tools on the production side, the post-deployment side. So it has built-in monitoring, built-in logging, alerting and also policy management and workflow that I'll talk about in a second. So those are the two main things that we need to take away from that. Another thing to note about containers and Tosca, this is still evolving. This is actually a suggested spec for containers is that Tosca because it is abstracted, it provides you also a way to abstract the actual software from the individual containers. And we all know that there would be more than one container in the world. So you could actually use Tosca to also be portable from the containers, not just the cloud infrastructure itself. And that's another interesting benefit to that. Now, if I look at the exercise that we have setting up MongoDB and Node.js, the way we modeled that in the Tosca blueprint is in the following way. I had one blueprint to manage MongoDB, and in that blueprint I have three kind of nodes. One node is to set the replica set. It's called Scalable, which means that for each node of that replica set I can define multiple instances of that. Then I have a dependency on that, which is the initialization node. Initialization node, it's all purpose enough. That's a logical node. So node in Tosca, a logical node. It doesn't have to be a physical node. It doesn't have to be a machine or a network. It could be just a software piece or a process that runs and ends, and it could still be defined a node. So in this case I can define a node that is a logical node that depends on the replica set, which means that from that respect it will basically start only when the replica set are ready, and that's how I can initialize that without external tool intervention, like in the other tools, and that's an interesting concept there. Similarly, I can do the same thing for MongoS. Same concept. I can have an initialization node that will trigger the initialization process after the MongoS cluster is being defined. The main takeaway from that kind of modeling is that I can actually automate the process almost from start to end without too much breaks in the process. And so if I measure it from an automation experience perspective, because of that strong relationship model, it's much easier for me to model the application so that I can create one blueprint and then say, give me the entire thing without doing any steps in the middle, like calling scripts or calling external tools. Let's see a few snippets of the blueprint itself. So this would be, for example, a way in which I can define infrastructure element. Again, I wouldn't go too much into the details of that, but in the same way that I define an infrastructure elementless security group, I can define a whole cluster of databases. In this case, it's a MongoDB shot. Note that I'm using something that is called Substitutable. Substitutable is the way in which I can create nested types, or nested or dynamic binding as they are called, which means that in runtime I can decide that a set and component, a set and node in the elements would have sub-elements that are dynamic. And for each application deployment, I can substitute that with another sub-element, whether it's a set, a different MongoDB server configuration or a different replica set configuration. In this specific case, we have a Substitutable element for the replica set, which means that every time that I'm creating a MongoDB instance, sorry, MongoDB shot, it will automatically create a replica set with it. So by defining multiple shots, I'm also creating the multiple replica set per shot. The other thing that I wanted to kind of jump into is the relationship concept, a concept in Tosca that is very strong as requirements. So for example, in the HAProxy, the way I express the dependency in the Node.js component is by expressing requirements and specifying the members in the Node cellar. Note that the relationship is implicit, and that will trigger a call to a certain script or flow. That will, at the end of the day, will inject the IP addresses of all the elements in the cluster into the HAProxy. In my view, it's more elegant, even though it's implicit. Last concept to discuss is the workflow. The workflow is really a way in which I can interact with the cluster after it's being deployed. And the reason why I think it's powerful is because a lot of the work is actually after the cluster is being deployed. Workflow gives me a way in which I can execute scripts on that cluster that has a context of the graph, which means I can execute different types of operations on any element in the graph, and the workflow is basically scripted in Python. The built-in workflow, for example, install, which is basically reading the graph, figuring the dependencies in the graph, and then calling the lifecycle events in each node in the graph. That's how I can set up an installation with the dependency in it, and it could be fairly implicit, meaning that it could be generic to any workload, to any type of graph. Similarly, we have uninstall, heal, scale, and so forth. Discovery is done through navigation in the graph. Once I have the context of the graph, I can actually navigate through that, and that's how I can discover elements, including external elements of other blueprints. I wouldn't discuss that right now, and it can be remote or local. So to sum the part of TOSCA into that, the main benefit of it is that it's portable, and it has a more holistic view on how it deals with the application or with the orchestration task that I mentioned. Basically, from a robot analogy perspective, the same robot can build more elements rather than doing a smaller step, so that means that you need more robots to actually fill in the gap. The main, if you'd like, benefit that Clarify brings as an implementation, and that's more implementation-specific, is all the workflow monitoring the post-deployment into it beyond the configuration aspect. On the con side, I think that the thing that is apparent to everyone is that the spec itself is still evolving, and Clarify itself is still not 100% compliant with that spec even in its current state, so we're evolving ourselves, and it's a moving target like the entire industry. Obviously, the set of tooling, given the level of maturity, is also limited. Then I wanted to actually move over to heat, and that's going to be shorter because most people probably familiar with the heat, and the concepts are very, very similar to Tosca, only that it's specific to open stack. That's really the main difference. So even if there are gaps between Tosca and heat right now, I believe that those gaps will be filled over time, so if your world is open stack and only open stack heat will be providing fairly compatible features. So just to give you, for those who are less familiar with that, a quick introduction to that. It's basically a way to model all the nodes and elements in open stack in a scripted language. Again, it's called HOT, with concepts that are very similar to that. From an architecture perspective, it's basically a REST gateway that can pass those blueprints and interact with all the underlying infrastructure that's a simple way to describe it, whether it's in there, whether it's Nova, Neutron, and so forth. So from a solution perspective, they were modeled that blueprint in this specific case, because right now it has less of a strong relationship. Again, these are gaps that are being filled in, and it's probably also part of my lack of knowledge not being expert in heat. Is that I model it as a few different blueprints that I need to execute separately or task stacks as they are called in heat. The first stack is the Mongo replica set, and once I set the replica set, the output gets me the list of IP addresses of that replica set. Then I need to call a script that will pick that output, pick one of the IP addresses, SSH to that machine using SSH key, and execute the initialization of that replica set. Again, similar to the thing that I'll do in the Mongo S side, only that because on the Mongo S, Node.js is dependent on that, that makes the dependency between Mongo S and the Node.js more complex because it's not just waiting for the Mongo S to start. It's waiting for the Mongo S to start, and then someone needs to initialize it, and then you could start Node.js. So the idea here is that the relationship can be a fairly complex thing, and not just a simple start machine and run some data you need to make. Note that the YAML file looks, resembles a lot the Tosca syntax. Again, the types are slightly different. In this case, I basically defined the security group, which is an infrastructure element, a server, a resource group, which in this case I can create more than one instance of that same definition. In this case, that would be the replica set instances, and I think that's kind of the main element that I wanted to kind of spend time on. So if I summarize it, I think that for OpenStack users, hit will come always fully updated with all the types and mapping of the resources that comes with it, and therefore it will be very good for setting up infrastructure within Hit. It has some software configuration elements, including integration with Chef, that gives it some capabilities to also do software configuration. Right now it's not a strong capability. If I take the elements into the con side, then obviously the same benefit is also a disadvantage, which is it's limited to OpenStack. So if my world is not just OpenStack, then that would be a limit. Also the post-deployment production require external integration with external tools, whether it's tools from the OpenStack ecosystem or tools from the outside world. It's still an integration process that you have to go through, and it's less trivial. I think with that I wanted to kind of look at the entire comparison in a different way. So we look at the tools as an alternative to one another so far. And basically what we try to do is show how each of the tools deal with the problem of taking an application and getting it into production. In this case I think, and it was even in this summit, we've seen that there is also synergies between the tool. Even though there are overlap between the tools, there are synergies between the tool. And I think the announcement in this summit around Magnum is a good indication to that, that we can actually, it makes sense, for example, to set up the infrastructure for Kubernetes through Hit, and then run Kubernetes to run container-only workload, and Hit to run the rest of the workload. So there is some synergies here. Similarly, by the way, Cloudify and Kubernetes could live very nicely together. Cloudify and Docker already integrated. That's part of the announcement that we made. And also there is some level of synergy between Cloudify and Hit. So the main takeaway on that slide is really to say that they are not necessarily just an alternative to one another, because they have different switch spots. Depending on the switch spot, they can complement one another because one thing will do a good job in certain area and the other one will do a good job in another area and that's where the synergy comes in. So this is the bottom line, right? How do I choose the right tool for the choice? So based on what you've seen, we've seen that there are more than one option to deal with the problem and very different approaches to deal with the problem. So if my world is microservices and containers-only, Kubernetes was built for that. So it will do microservices. It comes with a fairly opinionated way and how you structure your application for that specific task and therefore it will do a very good job there. If your world is more heterogeneous, in which you don't necessarily know that it's going to be just OpenStack or it's going to be just microservices and your world is more a combination of legacy, microservices, and new other stuff that doesn't fit that, then you would need a more of a pure-player approach. If your world is only OpenStack and you have a lot of infrastructure set up, then obviously things like heat would come into place. So those are the main questions that you need to ask yourself to come to an answer and say which tool are good for you for that specific workload. We can go even deeper into that questions and find which workload is that more stateful workload, less stateful workload, and that will also lead you to a different type of selections, but the bulk of it is really, am I container-centric? Do I believe in heterogeneity? And if my environment is heterogeneous, that will give me a different set of tools. With that, I wanted to conclude with the following words. So if you follow the OpenStack Summit event every year, and I did that for the 11th time this time, I'm going to get to Bar Mitzvah soon, working on it, then the main thing that you will notice is that every year there was a new next thing. Every year, like two years ago, it was puppet and chef, and it was Ansible and Salt Stack. This year it was a docker, and I can assure you one thing. Next year it's probably going to be something else. That's the only constant that I've seen, and that's the reality that we live in. And again, if I take Darwin's approach to that, it's not the strongest species that survives, it's the one that is more adaptable to change. Speaking of change, change is a way. It's coming over, and it will continue to come over. Just see the announcement of Kubernetes with CoreS. There's going to be some dynamics there between what Docker is trying to do, going up the stack, Google and other companies being threatened by that, and obviously they would never let one company control their destiny. So there's going to be some movement there, so there's not going to be one container or one, if you like, company driving that, even though right now that seems to be the case. And so it's inevitable that it will be in that case. Similarly, we've seen movement in the OpenStack community towards TOSCA, and much more openness within OpenStack towards collaboration and integration with Google and other clouds, so there's a lot of change coming down the road. And I think with that, I wanted to thank my colleague here, Dan, who had a lot of hard work into it, a lot of nights that it was actually spent on that. Interesting exercise for us as well. Uri obviously jumped into the challenge gladly, and that will let you ask questions. Any questions? No questions? So everything is clear. Everything is clear. So you talked a lot about TOSCA, but compare and contrast that with using an IETF, like a Yang model type system? Okay, so that's a very good question. The question is to compare TOSCA to Yang, those are not familiar with Yang. It's by O, U, G, I almost spelled it correctly. Y, A, and G. That's a specification that is mostly popular in carriers for configuration of network services. We actually find that there is a lot of synergies in terms of integration between TOSCA and Yang, and that's something we even announced this summit, that Yang, specifically, TELF, can do very good job on network configuration, but less of a good job on software stack and more application topology, and therefore you could actually integrate the two together. So you could use TOSCA to create the infrastructure, to create the network elements, and then call Yang to configure those elements based on the Yang template that most people already have existed. So that would be the different sweet spot. So even if there is an overlap, there's different sweet spot, and therefore they could actually be more synergetic to one another than alternative to one another. So next year we'll see a little more Yang in there. What's that? We'll see some Yang in your presentation next year. Well, actually, yes, that's a good point. It's a good point. Point taken.