Recent advances in rapid and affordable DNA sequencing technologies have revolutionized diagnostic microbiology and microbial surveillance. Availability of bioinformatics tools and online accessible databases has been a prerequisite for this. In this paper, we conducted a scientific literature review and presented a description of examples of available tools and databases for antimicrobial resistance, AMR, detection and provided future perspectives and recommendations. At least 47 freely accessible bioinformatics resources for detection of AMR determinants in DNA or amino acid sequence data have been developed to date. These include, among others but not limited to, ARGNOT, CARD, SRST2, MEGARS, Gene Finder, Ariba, Khmer Resistance, AMR Finder, and Res Finder. Bioinformatics resources differ for several parameters including type of accepted input data, presence slash absence of software for search within a database of AMR determinants that can be specific to a tool or cloned from other resources, and for the search approach employed, which can be based on mapping or on alignment. Each tool has strengths and limitations in sensitivity and specificity. This article was authored by Rene S. Hendrickson, Valeria Bortalea, Heather Tate, and others. We are article.tv, links in the description below.


I like seeing design in both big and small things, you know, from a, you know, a bridge and how it's created to small details in a building. And I think that's a great thing about architecture. It could span over those breaths between giant enormous city plans to real touchable physical material. It's not an act of just my own personal will that I'm putting on a building. It has to work for people and how they use it. And that's really how the building gets shaped. It's, you know, really asking yourself those questions, how is this going to liberate the users to be able to use this building in new ways? How is it gonna help the way that they work or see the world? And then, you know, you have to really understand that before jumping into the form making.


I just want to say a couple of things. One, we're the same age, we're both from Chicago, we're both Aries, you're a Missouri grad. That was in my top three, I just didn't go to Missouri. Your parents were immigrants, my parents were immigrants. I just wanted to say, you get your flowers, just everything you've built. I have a lot of respect. I know yesterday was Women in Sports Day, so congrats to everything you've done. Sean, Sean, Sean, what is, is this like a prank and waiting for like the other shoe to drop? That is the sweetest thing that I've ever heard. Thank you so much. It was a big day yesterday. I just want to make sure you got your love, that's all. I didn't get enough love, but now I did. Now I feel very loved and you are just, you are amazing.


Yeah, we on boss talk one-on-one. Here we go. How did you get your first role? Like when you got into acting for the first first time, I think she won't act bro because she is really asking you How long did it take you to get that like cuz I know a lot of actors be like man, I've been doing this for so long He was on last week I saw Yeah, this is a marathon not a not attract me so It ain't see the kids now that everybody want to be famous But I've been telling Amber them like yo, you got to put the work in it ain't all that one thing I've my I somebody dig up my first television Appearance at the Apollo was in 1989. Mm-hmm. And it was It was I Tell everybody Back then people was in the streets. I'm saying so you know the streets and When they had when they had picked me up It was one of those days where you wasn't doing nothing You was just in the wrong place. Okay at the right time.  We're actually going to see Eddie Murphy everybody's in the movie. Oh, you turn the Harlem night. Yes, right? so So we're going to the movies Shit So and back then with how they would get you they give you you you They mark the money the buy money. Yeah. Yeah, so they check your pockets and put I got no buy money on But I got a page on me and Which I still have to this day got page I need that's what I need Boy you bring back mirror works. He don't know nothing about that You put it was three oh four for whole I remember Yeah, y'all didn't get that. We did something I didn't trust me.  Yeah, let me I gotta stop I Remember the rotary phone. That's what I remember.  No, that was a good time though. Don't y'all never think y'all missed out on some JJ J&J Bebers I was out there on her and off in Shreveport. I go down there and pay my bill. Oh man. I was the man Oh, they knew me in there. Oh, yeah, I come to pay my bills. Oh, yeah, I come through man When it beep nigga, I had a phone. We had the big brick phone. It didn't work. It was gonna come It was in the Cadillac, but it didn't work The page is stopped by the pay phones. You know gone now. You don't got pay phone. No, when they gave me my property back my pay I was like What I was saying was They can never say I Remember the judge saying listen I'm gonna give you the benefit of doubt because she asked me to what do you do for living? I was like, I'm a comedian I Told her weird and she's like, no, I've never seen she's like, but listen I'm gonna put you on probation for a year and if you come back in my courtroom with the same charge I'm gonna give you the time this time plus that time. Oh three. Yeah, right. I promise The next week I want to go to Apollo, so when you see that clip, that's me going Okay That was my Puzzle that was me saying goodbye. Yeah, I thought it fall or fly. Yeah, okay. Okay. This is yeah so it's me at the Apollo night and it was me and Steve Harvey Yes, Steve. I've heard him talk about when he first went to the Apollo. That was during that time He was doing a joke about Mitch. Yeah, that's exactly what I heard. Yeah.  You just cut me off of all time. Wait a minute. They're alive top three I can't stand you. I'm gonna quit.   That's Cuz it's like there's so many different colors. That's like sender any genre But I'm saying It's the different types of comedy Bill Cosby. Oh you a bill Like bill coming in real hard, but that's not his number one. He just calling some names Hard from you know, I've watched No, no, you guys only sell a portion of bill You gotta say Richard prior wanted to be Bill Cosby That's dope. I never knew that never knew that he wanted to be Bill Cosby. His whole act Was a bill Cosby's act Really? It was still Bill Cosby's up and going Look look at every documentary every book gonna have me going back and look listen I'm only gonna tell you the facts He's whole and it wasn't till Bill Richard prior chain. It's okay.    Wow And that's that's why I said um If there's gonna be any it'll be bill Cosby Richard prior Eddie Murphy only three Daddy Top three Yeah, we on boss talk one on one one on one. Yeah, we gonna talk


The first of those last two talks are being presented by Sharon Tateron and Michael Lang who are from a newspaper that you might have heard of. Please make them welcome. Okay, I see. So hello, I'm Sharon Tateron. I'm a software engineer on the New York Times photo team. Michael Lang also a software engineer, same team. We're really happy to be here today to talk about how we build sustainable systems powered by Python. So first some background about us and our team. As software engineers on the New York Times photo team, we build applications that support ingestion and management of the photos received by our systems. This includes photos from staff and freelance photographers as well as external services. We usually get in around 15,000 photos per day, but sometimes up to 45,000 photos a day for big events such as the Super Bowl or Oscars. As part of our infrastructure, we have developed and continue to develop a suite of Python-based applications and microservices that we refer to as Photon. This includes microservices that are responsible for downloading images from FTP and photographer cameras, stamping image metadata, and moving images into the photo archive. This also includes APIs and shared libraries. The responsibilities that Photon app support continues to grow. So what we provide, for end users we provide resilient, highly available systems that quickly resume processing when faced with unexpected, yet inevitable errors and crashes. And for developers we provide flexible, maintainable systems that can be enhanced with ease. Systems that really empower developers to develop. What we found is that a better developer experience often leads to more reliable systems. When developers can build, patch, and iterate upon existing code with ease and more easily identify failures, new features and bug fixes can be released consistently and safely. This ultimately benefits the end user. We've also found that maintainable systems are sustainable systems. Systems that can be built upon and sustain over time as teams change and new features are added. To achieve this, our systems are powered by Python, both built in Python modules as well as open source Python libraries. So why Python? I'm going to take a little diversion here. Earlier in the year, maybe you could flip the slides for me.  And this is the entrance to Petra through the Sikh. Petra is an old city, 2,500 years old. Here's a picture of what it looks like. It covers a lot of territory. And so my wife and I decided that we needed some transport to get around. Vehicles were not practical there. So we had a choice. We could take a camel or maybe a horse. But ultimately, I decided on donkey. And the donkey could do everything. It could go up and down stairs. It could go all day. It didn't need to stop for any reason. And so that's why we use Python. It gets the job done. And we use a lot of Python. And Sharon will take you through more of our presentation. So what we're going to cover today is easy installation and configurable runs with set-up tools and click, code quality and type checking with Black, Flake 8, and MyPy, and feature addition with pluggy. Also, we're going to talk about resilient runtime with fail-fast and start-fast approaches and automated documentation with things. All the tools that we talk about today are either built into Python or modules, I can't say that right now, that are easily installable with pack. So installation and running. So first we'll talk about easy installation with set-up tools. Set-up tools enable building and distributing Python packages as well as namespace packages, which enables us to build and distribute sub-packages separately. Namespace packages are often used for loosely related packages and to indicate an organization name, application family, et cetera. In our case, our namespace is Photon. With namespace packages, each sub-package can be its own project and have its own repository, while also contributing to a shared namespace. In our architecture, each Photon microservice or shared library is independently installable within the Photon namespace. This allows us to locally install and develop one package at a time, deploy each microservice separately, and distribute each common library to our internal package repository individually, or all while maintaining a shared namespace. This is an example of how imports look from a shared namespace. Common and demo-util are separate projects, but both contribute to Photon. So one issue that we faced early on was whether we should use monorepo or a multi-repo. Monorepo's got some advantages, which you can see there and others. Multi-repo also has some advantages. We ended up, which is kind of a work in progress, but we have a lot of apps and we want them to be able to share, but we also have a lot of developers and we want them to independently be able to work on things. So we combine sort of mono and multi and maybe you can guess what we call it. Yes, it's the Monopython repo structure. So Sharon's going to give some examples of that as we go on. So in order to use setup tools for easy installation, first we pip install setup tools, then we create a setup.py file, and then we run pip install in editable mode in the project directory. This is an abbreviated example of setup.py and the setup function. After importing setup from setup tools, we pass an option such as name, description, and version, as well as entry points, packages, requirements, among others. In the packages option, we define the namespace packages with each prefaced by Photon to establish a namespace. For entry points, here we're defining a console script such that running a command that we've named invokes a Python function within our project. In our case, that function exposes a command line interface to the application and related utilities. The command line interface is based on the click library, which we'll discuss next. So click stands for command line interface creation kit, and it allows us to build command line interfaces by creating composable interfaces out of the box. In our architecture, each Photon application package has a command line interface entry point as we so defined in setup.py for running the application as well as supporting utilities. This enables us to pass argument to the application at runtime so that different configurations and scenarios can be explored with ease. So here we set things up. So first we pip install click, then we define the commands and invoke the commands. We've set things up such that individual commands are loaded from a command directory into an implementation of clicks multi-command class. If we run the entry point defined in setup.py, we invoke the multi-command class and we see that the subcommands listed correspond to the files in the directory. This allows commands to be lazily loaded and also any new commands can be added by simply dropping a new file into that directory. So here's an example of how we use click function decorators to define the command as well as the options for that command. Next we'll talk about how we do code quality and type checks with Black, Flake 8, and MyPy. So Black is an automated Python code formatter and it follows PEP 8 style conventions. It aims to reduce the smallest ifs possible, which means the fewest number of line changes. This tool adds productivity during development and during code reviews. Developers can really focus on just the content being added. It's effective for onboarding and knowledge sharing. Consistently formatted code is more readable across many developers. And it also allows us to standardize and format existing repositories with one command. No manual reformatting is necessary. So in order to use Black, we pip install Black and then we run it. So here's an example. We're moving the method call to two lines instead of one. We accidentally add a semicolon and we change the double quotes to single quotes. When we run Black, we'll see that everything has been reformatted. The semicolon disappears, the single quotes change to double quotes, and the method call goes back to one line. Since Black defaults to a max line length of 88 characters. Slide. Here we go. So next, we use Flake 8 also for code quality. Flake 8 is style guide enforcement and is actually a wrapper around three other Python tools. This includes PyFlakes, which checks for unused imports, variables, et cetera. PyCode style checks for PEP 8 style conventions, but we don't need to worry about that since we're already using Black. And McCabe checks for code complexity. This is another tool that really allows developers to just focus on the content of the code. So in order to use Flake 8, we pip install Flake 8 and we run it with optional configuration. In this example, we'll see that we add an unused import and then we're going to remove a used import. Once we run Flake 8, we'll see that we get expected errors. There is an unused import and also there's undefined names. So when we switch back, we'll see that everything has been solved. Let's see. It's hard to stop the video. It wants to keep going. Okay. So we learned a lot about typing in MyPy earlier as well. So to recap, we use MyPy for type checks. MyPy is a static type checker and it allows us to add type annotations and receive type hints. If there are inconsistencies in what's being passed to a function or what's being returned, there's built in Python support for type hints with the typing module which supports types such as any, union, optional, et cetera. It helps to identify common type-related bugs. So in order to use MyPy, we pip install MyPy and we run MyPy with optional configuration. Annotations are treated as common by standard Python interpreters, so typing does not interfere when running the application. So in this example, we change the parameter type from path to string and we get errors for incompatible types. Then we're going to change the return statement to a boolean and we'll see that we'll get errors since no return value is expected. So here we go. We're going to run it again. And then once we change everything back, we'll see that MyPy has passed. So in addition to running the three tools locally, black, flagate, and MyPy, they can be added as part of precommit hooks or also continuous integration build tests. For example, in our pipeline, a pull request must pass these standards in order to be merged in. So this is an example. It's a simple example of how we organize our applications. These are the server-based applications. We use a threading model. We use the threading model, I should say, pretty extensively to, in effect, create each thread. It kind of becomes its own little micro service within the application. And it's kind of a split input process output, which is an old model in computing. And so I want to remember this. Sharon's going to show a little more detail about examples, how this is organized. So another tool we use is Pluggy for feature addition. Pluggy is a plugin management tool. It enables plugins to hook into program execution. Pluggy is used internally by PyTest as well. Pluggy is particularly useful when defining divergent paths and when specific and sometimes lengthy logic is needed for particular use cases. It helps to avoid ever expanding if-else conditions and allows conditional business logic to be modular and self-contained. It's also useful for defining sequential steps in a modular way. Pluggy is really great for productivity as well. Developers can really focus on just the one feature being added by only interacting with the relevant plugin. And supporting a new use case can be as simple as defining a new plugin. So in order to use Pluggy, we pip install Pluggy, we import Pluggy, and we define a hook specification as well as hook implementations, which are the plugins. So this is how we set things up for plugins. Similar to the commands directory that we looked at earlier, we define a plugin directory from which plugins are loaded. Function decorators are used to define the hook spec and the hook implementations. When the plugin manager's hook method is called, it invokes each of the implementations and returns a list of return values from each plugin. In this example, we're using Pluggy to define separate image transformation, such as resizing or smoothing an image. A new transformation can be added by creating a new plugin class, specifying a function as the hook implementation, and dropping the file into the plugin's directory. It can be removed just as simply by deleting that one file. More implementation details of how we use Pluggy are available within our demo repositories. So moving on to a different topic, we're going to talk about thread-based, fail-fast, and start-fast approaches for a resilient runtime. We can always count on failure, so we must prepare for it and design our systems to bounce back, even in unstable times, even when external services are throwing errors. When we handle failures through retries, sometimes it's difficult to keep track of the state or bring the application back to a healthy state, and sometimes failures are masked. In addition, when we use robust libraries in our applications, such as the Google client libraries that we often use, they tend to already have retry mechanisms built in so that we don't have to. Fail-fast approach works such that the app encounters an exception, logs the exception, and fails quickly. This allows the application to shut down, resume processing, and while providing developers with instant visibility into errors. The photon applications are set up to attempt up to 40 retries over 15 minutes before entering a fatal state. To accomplish this, we define an event object for the threads to share. When a failure occurs, we set the event, and we use the event as a signal for all threads to exit. Then we use a process manager, we use Supervisor, to restart the application when it has exited. On the other side of things, we use start-fast. When we restart, we want all processing to start simultaneously. This eliminates differences in startup time between threads and prevents errors that may occur when all threads are not ready. To accomplish this, we define a barrier object for all the threads to share. We specify the fixed number of threads to wait on, and then reaching a barrier account signals all threads to release and begin processing simultaneously. We're going to go through a visual example of how start-fast and fail-fast works. First, we initialize the main thread, and we instantiate and initialize the module. I'm having a lot of trouble with this word today, modules. Then we start up the demon threads, and then we see that one thread is ready and waiting. Next, there's a few more threads waiting at the barrier, but we haven't reached the fixed number of threads yet. Now, all the threads are waiting at the barrier, so then they can all be released simultaneously. Now, all threads are processing, and they're all waiting on a shared failure event.  It sets the fail-fast event, all the other threads are waiting for. Now, all the threads fail, and they clean up as well. This indicates that the application is ready to restart, and we're back at the beginning, a clean slate. That takes a few seconds. We're pretty quick on our restarts of our services. One interesting thing about these services, I think, is that we arrange them in a hierarchy, and go to the next slide, like this. In effect, we have a hierarchy of microservices. We consider our applications to be microservices, but internally, they're also organized as microservices. This allows us to make our applications either simple or relatively complex, get self-contained, and let developers focus on them. I'm going to abstract a little bit from that. I think an interesting thing I've run into in my career, actually, we're both very experienced. On average, we have about 25 years of a professional experience. If you consider complexity and think about component count, and I don't want to dive too deep in what is complexity or what are the components, but in general, if you draw a line which is constant functionality, you're wanting to build your application somewhere in that sweet spot. You don't want it to have too many components. You don't want it to be complex, and you want to be in that sweet spot. The sweet spot is actually quite wide. When microservices first came out, I dived on them. I think I built applications with way too many components. Since I've pulled back and looked at ways to simplify that architecture, and that's pulled me a little bit more to the left side of the graph. Now, we can, of course, describe this graph a little more rigorously. Oh, actually, those are Maxwell's equations, but you can describe everything with Maxwell's equations, or it could be summarized, as we all know, as the number 42, which is significant to me this year because it's my wife and I's anniversary, and so we went to IMAX. That's the end of that, non-secretary. So, last, we're going to briefly touch upon automated documentation with Sphinx. Sphinx generates documentation from code comments, parameters, types, et cetera. In order to use Sphinx, in order to use Sphinx, we pip install it, we go into the docs directory, and we run make HTML, then we can open the index page. This can be done on the subproject level, but we also set up our development environment to generate docs from the top-level directory, such that documentation can be generated cohesively for multiple photon applications, common libraries, and other utilities all at once. This is an example of how the resulting documentation looks. So, in our code, we comment it pretty extensively so that we can produce this documentation. And this is after the donkey ride, relaxing in the Bedouin tent. So, we made available three repositories, which provides examples for how we set things up for our applications. You can find it on GitHub and interact with it. If you have any questions regarding it or comments or feedback, feel free to add an issue to the repository as you explore it. Also, we're hiring at the New York Times, so if you have any questions about that, feel free to find us and ask them. Thank you. Thanks. Thanks, Sharon and Michael. So, if you code Python U2 can be an award-winning newsroom. Hey, we're going to swap over our presenters.  Daylight saving does indeed finish this evening, which means if you do not adjust your clocks or your clocks do not automatically adjust for you, you will show up at this venue one hour before all of the organizers show up, and you'll be in a cold room with not much to do for an extra hour. So, please, use your government-mandated extra hour of sleep wisely and show up here at 10 a.m. Pacific standard time, not Pacific summertime. You know what I meant enough to correct me, so you didn't need to correct me. Okay. So, for our featured presentation, in 1989, our featured presenter wrote a programming language and named it after his favorite TV show. He's recently started writing a new parser for it, and he wanted to talk about it.


Chapter 7 Temporary Storage of Dead Bodies As a disaster usually overwhelms response capacity, it's often necessary to store dead bodies temporarily while awaiting identification and release to the family. This storage must be done respectfully and quickly, especially in hotter climates, to preserve and protect the bodies and improve the chances of identification. The storage process starts with the establishment of a mortuary or temporary mortuary with sufficient secure refrigerated storage. Each body or body part should be secured in a body bag or container with a unique code written on a waterproof tag and fixed to the body and body bag with cable ties. Chain of custody documentation is essential. Refrigerated containers with suitable racking are the preferred storage option and the optimal temperature for body preservation of 2-4 degrees centigrade should always be considered. Ice and dry ice are both problematic with potential safety concerns and should therefore only be used as a last resort. In the absence of refrigeration, cooler environments should be found that will delay decomposition. Finally, there are specific guidelines for temporary burial which is a good longer term option when it doesn't conflict with cultural norms. Remember, always store bodies with dignity and in a manner that will lead to more timely and reliable identification by experts.


If you get the skills you need to lead your team and accomplish your goals, let's find out. And congratulations on taking one more step towards becoming one of the great leaders of tomorrow. Today we're going to take a little bit of a break from the deep personal reflection we've been doing and focus on something a little more straightforward and practical. And that's determining the types of skills you feel like you need to have to be an effective leader and evaluating how good you are at those skills. Now the set of skills any leader has is unique based on their mission, vision, and situation. And it's going to be up to you to determine the set of skills that you need to be an effective leader. And there are some common ones that you see among great leaders like public speaking, time management, and problem solving. And I would recommend adding these to your list to get you started. At this point you may be asking, what's the difference between a skill and a trait? And that's a really fair question because the line between them gets a little blurry sometimes. The way I look at skills is that there's something that's more like an intellectual pursuit. Something that you can learn through study and practice. Whereas a trait is something a little more internal and personal. And what requires practice to develop a trait, it's something much more emotional and requires a lot of self reflection and behavior modification to really make into a solid trait. For example, if you wanted to build a strong trait of being organized and focused, you could work on organizational skills like time management and practice those skills in order to modify your behavior to really develop those traits of being organized and focused into strong ones that are part of your leadership style. So let's do a little exercise that'll help you figure out the skills you're good at and the skills you might need to work on. I'm assuming you're already pretty good at the job you're doing now. So I want you to think about the next job you would like to have. And I want you to think about the ideal person for that job and the set of skills that that person has that makes them so good at getting that job done. So once you've got your ideal person in mind for the job that you want to have and the set of skills you feel like they need to have to be successful at it, list those skills in descending order of priority so the most important skill is at the top and the least important skill is at the bottom of your list. Don't forget about interpersonal and soft skills too such as communication or negotiation because those can be very important especially in leadership positions. Once you've got your prioritized list done, rate yourself on each of these skills from one to five. One being you need a lot of work in this skill, three meaning you're about average in it, and five meaning you're among the best people you know of at this skill. If you want to gather some additional info about how you rate on these skills, go ask a few coat workers or even your boss to rate you on these skills and factor that into your assessment as well. Now look at the top 10 skills in your list. If you've got fewer than 10, don't worry about it, just work with the list you've got. And find the two skills in your top 10 that have the lowest ratings. Also find one skill on your list that you'd be really excited about developing further. So in the example I've done here, you might choose public speaking and giving feedback for skills to work on and you might choose negotiation as a skill that really interests you that you'd want to develop further. And what we're going to have you do is focus on developing these skills for the next three months. As we've talked about before, great leaders treat leadership as a discipline to be studied and reflected upon throughout their careers and even their entire lives. One of my favorite quotes about leadership comes from John F. Kennedy and it's leadership and learning are indispensable to each other. Taking a hard look at the skills you feel like you need to advance your career as a leader and then taking action to grow and develop those skills is the best step you can take today to begin that lifelong study and practice of leadership as a discipline. Tell us in the comments what you think the most important skills for leaders to have are. And if you've got questions about doing the exercise, let us know, we'll be glad to help you out. Come back later in the week, we're going to have a blog post that talks about ways you can develop your skills, not that you've identified the ones that you want to grow. And the best way to get that, click on that globe off to the right. That'll let you subscribe to our email list and that'll bring all of our content direct to your inbox. Thanks again for watching, I really appreciate it. And remember, the future is out there. Lead the way.


Howdy, everyone. Welcome to the second scale container day, second time we've done this. And we have a really cool lineup of speakers covering different techniques and different technologies for you today if you stay all day, not required to. We'll have three talks this morning, a break for lunch, four talks in the afternoon with a break in the middle of the afternoon. Starting us off right this morning is Elizabeth Joseph, who some of you who've been around open source for a while may know from her work on Ubuntu, where she co-wrote the Ubuntu book. And you may know her from her work on OpenStack, where she wrote an OpenStack book. And now she's working on Mezos and DCOS, and presumably writing a Mezos or DCOS book. Not yet. Not yet. OK. Well, anyway, she's going to talk to you about application deployment on DCOS and welcome. Thank you. All right, thank you for that introduction, Josh. So yes, I'm here to talk to you about continuous delivery using DCOS, which is built on top of, hey, laptop went to sleep. It's waiting too long. Come on, come back. So I'm going to quickly talk to you about what continuous delivery is, just so we're all on the same page before diving into this. So as Josh already explained, I've written a couple of books over the past many years that I've been working in open source communities. Sort of started out in Debian, moved over to Ubuntu, started working on OpenStack. And it's funny because in my career these technologies all sort of build upon each other. So I started off with Linux. Went into a cloud platform. And now I'm working on containers, which live on top of the cloud platform. So I've pretty much been going all the way through. So I spent about a decade working as a Linux systems administrator. And sorry about the flicker. We've got a bad VGA cable or something going on here. So I was working at HP when I was working on OpenStack. And after my team was dissolved there, I realized I really liked giving talks at conferences and getting to know people and sort of translating what we're doing in the infrastructure side to what people can do in their own environments. So I'm now a developer advocate over at Mesosphere. And we're the company that employs most of the developers of Apache Mesos.  But if someone is working on Mesos and they need to be paid for some reason, maybe they don't have a job somewhere else. We'll pretty much scoop them up and make sure that Mesos continues to be supported. And when I was working at HP on OpenStack, what we were doing there is we were running the continuous integration and deployment platform there. So I spent four years doing that. So this sort of coalesced in my talk today where I'm talking about CI CD based on my work that I did. And then how doing that in containers today makes a lot of sense. So first to define continuous delivery, just grab from the Wikipedia page software engineering approach so engineers can deploy software in short cycles, making sure that releases happen often. So this could be releases are happening with every Git commit. That's what we were doing in the OpenStack project. Every time someone updated the documentation or made changes to our infrastructure in any way, they went live immediately. So that continuous deployment is not actually that common because it's a little bit crazy. So usually it's more like it could be like once a day or a couple times a week and then sort of going up from there. And in the process of deploying the software, you're also testing it. So instead of releasing every six months or once a year, you end up having a development and feedback cycle that looks more like this. It might be hard to see here. But it's sort of starting out doing your project planning and then you're doing your development, testing, staging, and release pretty fast. And then immediately once you release your software, you can get customer feedback. So that's like the week of the release. You immediately get customer feedback. And then the next week, you're still working on things and still making releases to customers so you can continue to get that feedback. So the customers are happier because they're getting their features more quickly, or at least getting to try out some of the things in your software at a faster pace. And the developers are happier because they're actually seeing their work go into production and touch customers. And this is something that can't be understated. It's really important for developers in your organization to see their work in production, especially with people only staying at companies for a year or two. If they work on a feature and don't see it released for six, eight months, maybe a year, it's incredibly discouraging. Because by the time they've moved on to another company, their software is finally in production. And they're like, that would be satisfying if it had happened quicker. So they're getting closer interaction with the customers, and customer feedback can come in really quick. So this is becoming really popular, is doing this continuous deployment to get the feedback really fast. And if you're doing this with containers, I sort of see it as this two-pronged approach. So first, everything that we do at Mesosphere and for our CI CD pipelines, everything is run in containers. So this is the applications themselves that are doing the CI CD stuff. So I'll show you a demo at the end if we've got time. Hopefully we should have time. That runs Jenkins inside of a container, runs GitLab inside of a container, and then also all of the jobs that it's running to build the container that we're actually deploying, and then the deployment itself, all running in containers inside of a Mesos cluster. The other thing it allows you to do is organize everything really efficiently. So I have a picture coming up later that sort of shows you in a traditional system, things each have their own server. But when you split things up into microservices on containers, you're allowed to have much better resource utilization. So the way things sort of used to go is that operations would handle everything starting at testing. So when I was working on OpenStack, my team in operations ran the entire testing infrastructure. So every time someone made a commit to OpenStack, it went into our testing infrastructure, and then we pushed it. Well, we didn't have staging because who needs that? We just pushed it into production. We're an open source project, it was fine. So we controlled the entire testing infrastructure. And as operators, that's fine. It's great for us to control everything. We don't want developers running crazy things, but the developers felt really stifled. They were like, we can't use our own tools for testing, we really want to have more control over this. And this became particularly apparent in teams like the design teams and the documentation teams, which didn't necessarily want to have the same workflow as engineering. So what we'd have, which I thought it was kind of cool, but all the docs people had to learn Git, and they didn't think that was cool. Some of them did. Some of them really actually embraced having an engineering workflow, but it made it hard to get contributors to documentation people, because they were just not really keen on learning Git and learning Garrett and learning their entire development workflow just to write documentation. So in this new paradigm, we're sort of trying to move testing into what developers can control. And in order to do this, that means giving them control of their platform, but they don't necessarily want to run Linux and make sure a machine is upgraded and stuff. They just want to run their application, and that's where containers come into this. So you end up having to have an environment where you can support multiple CI CD pipelines. So the base part of this, all these little servers on the bottom are things that are run pretty much by infrastructure. So in the case of DCOS, that would be Mesos. And then on top of that, the developers themselves can run these individual applications. So they don't need to worry about hardware or their virtual machines. They just say, I want to run Jenkins. They click a button, as I'll show you, and then they're running Jenkins. So they just really need to worry about what platforms they want to use. So this can be an infrastructure of what I'll show you. I'll show you GitLab and Jenkins, but this could also be something like Artifactory and Travis CI. Maybe they're actually using GitHub and they want to add that into their infrastructure. So they're pushing everything in and out from GitHub from the stack. But whatever they want to use, it makes it a lot easier for them to decide what they want to use as far as an infrastructure goes. So if you're doing this on virtual machines or on bare metal, you end up with this industry average of utilization of 12 to 15% on servers. And obviously that's very bad. So you have one server for Jenkins, one for Artifactory, one for Cassandra. And then if you want to do high availability across these, you add in another server and then you're load balancing across those. So a better approach is using some one of these container orchestration and clustering software like Apache Mesos. So Mesos is what you would install. So you install Linux on your fleet of servers, whether these are bare metal or virtual machines, you just need to have Linux. In the case of Mesos, it runs on Red Hat and Ubuntu and CoreOS. DCOS doesn't officially support Ubuntu, but it totally works like people are running it. But Mesos itself, you install that on your base Linux install. And Mesos is what clusters all of the machines and turns them into this one big cluster that you can access and get resources from. It also does sort of intelligent scheduling. So when an application needs certain resources, Mesos, depending on the application, there are frameworks inside of Mesos that know like if an application like Spark or Hadoop needs certain resources, you can intelligently divide those up across your infrastructure. It knows it needs a master and agents.  So these frameworks inside of Mesos are sort of intelligent with making sure both masters aren't on one machine and other things. It's scalable, it's been used, Mesos has been around for seven or eight years. It's being used by OpenTable. OpenTable is a really cool one so you can go to OpenTable.com and scroll all the way down to the footer and highlight an empty space in the footer. You can see which Mesos agent it's running on. That was really cool. I learned that during one of their talks at MesosCon. And then Yelp and Uber, Twitter, how they solve the fail-well problem was using Mesos. So it's a pretty big thing in the industry running a lot of the internet. One of our founders is a guy from Berkeley, Ben. He's super cool. He's one of the reasons I started working for Mesosphere because he's super passionate. Still codes in the weekend even though he's this fancy startup founder now. And then on top of Mesos, we created DCOS. One of the things we saw is our customers were solving the same problems over and over again. In fact, some of them actually created open source projects around solving the same problems like Yelp did. They have an open source project called Pasta. And that pretty much does a lot of the same things that DCOS does, which is doing all the resource management, tasks scheduling, container orchestration. So it definitely calls out. It uses Mesos on the bottom, but Mesos can be kind of a bear. It doesn't handle container networking and logging in metrics and stuff. So DCOS adds that layer on top that handles everything you need for your orchestration. And making sure your containers can talk to each other because container networking is kind of terrifying. And one of the really cool things it has is this universe catalog of preconfigured applications. So when I installed Jenkins and GitLab later, these are applications that you just install from our little app store, but they have some same defaults. So I'm not really gonna change much in the Jenkins defaults or the GitLab ones, but it pretty much gives you a base amount of RAM and storage space and CPUs to run a super basic install. And that's all preconfigured, typically by people who know what they're doing. So we interface with the Jenkins community and the GitLab community and the Artifactory folks to make sure we have some sane defaults in these packages. And in a lot of cases, they actually contribute the packages themselves to DCOS. So to show you a picture of how this all comes together, the bottom there, you've got any sort of infrastructure. So you've got, maybe you've got physical servers, maybe you have an open stack cluster, or maybe you're going to public cloud. So Amazon, Google, Azure. And that's pretty much all you need these to do is run CentOS or CoreOS. And that's what those lowest infrastructure stuff does. And that's what you install Mesos on top of. And then DCOS actually includes the installation of Mesos and then it includes all the container orchestration and everything that I talked about. So this means you can also move it to, if you start off in a cloud, if you decide the cloud is too expensive, you can move it to another cloud. If you decide cloud is not for you, you can move it on-prem. And it avoids a lot of the vendor lock-in, which is something I care a lot about as going through this open source stuff for so long. And then you interact with DCOS through, there's a web UI. I actually need to upgrade, date the screenshot. It's hard to see anyway. But I'll show you that when I do the demo. And it also has a command line tool, which you'll also see. And then it also has an API. So these are all ways you interact with it. And so if we look at our side from before, where we had the traditional data center with this not great utilization. When you're using Mesos and DCOS, you can see all these little blocks are now put across maybe three servers instead of five. And so they're all containerized so they can all run alongside each other. They're all playing nicely together. So you've got a few nodes for Jenkins nodes and Cassandra is split up across a few servers. So you're able to have an infrastructure that's really packed in tighter and you have much better resource utilization with all these things running in their own little containers. It also means that the entire, like you can run Jenkins and Cassandra on the same server, right? But then you need to install all the dependencies for both of those on the main machine. With containers, you can just ship the dependencies with the application and then you can move them around to all the machines you want. And you don't have to worry about libraries getting stale on these old virtual machines or servers that you're running. You just move the containers around. When it comes to actually deploying apps, I'll talk about this in a continuous deployment sort of way, the scheduling for launching a new service.  You instead call out to the frameworks and Mesos that sort of know where things should be, but you also have the containers themselves. So you do have to provision the containers and handle dependencies there, but it's not on the system level anymore, it's just in a container. And then for deployment, as I mentioned, there's like a UI for DCOS and a command line tool and stuff. So you don't need to give developers access to your actual servers. You need to give them a restricted user account so they can launch specific things through one of the user interfaces. Being a sysadmin, I used Nagios for a zillion years. I was actually still using it on my home network until a few weeks ago when I upgraded Debian and the Debian packages went away. So I'm a big fan of monitoring, but one of the really cool things about using these new containerization things is we have in DCOS something called health checks. And if a service dies, you can have it automatically bring it back. Or if a service dies five times, you can say, okay, after five, don't bring it back again because something's wrong and I need to look at this. But there's different ways you can do health checks. So you're no longer getting a page if MySQL dies. It just restarts it and lets you know, like, hey, this thing died, but I restarted it. So your blog still works. I mean, you should probably figure out what the problem is, but it can automatically restart that stuff. And you don't have like, I used to use all these tools. I can't remember the one I was using a while back was, but there's all these tools you can use for like restarting things automatically. And system D, of course, does some of this restarting now as well. But health checks are really nice because you're able to see like, across time how your systems are doing. Service discovery is something that I really like in DCOS. So with the olden days, you've got your DNS servers maybe running locally on your system. And you're trying to manage all of these host names and where your services are. Maybe you're manually load balancing. DCOS takes care of a lot of this stuff. It has automatic service discovery. So when I launched Jenkins later, Jenkins will have an internal service name of Jenkins.marathon.mesos. So that's its internal DNS name. You can always get to it by that. And it's really nice to know that you already have a DNS name already set up automatically for you for every service that you launch. There's also load balancers built into it. We're gonna use a load balancer when we launch our application later to see how it can automatically serve traffic out to the internet for you. And then for persistent storage, we're working on a bunch of stuff using like for the external volumes, things like Rexray, it will mount the volumes for you inside of containers. And then you can also run things like Ceph and Gluster on this cluster. And those are like self-healing file system, network file systems. So the point of all this is that we have this huge cluster and now developers can with a single click install whichever application they want. Once it's installed, it's served across the entire data center. You can do load balancing and the applications are pretty easy to run. So the developers can run their own infrastructure and it's fine. They don't need to worry about updating Linux. They don't need to worry about owning servers. If their Jenkins dies, it can pop up on another server and attach to the network storage and be up again and it's okay. Question. Yeah, so the question is if one developer launches Jenkins, how does Mesa handle another developer launching Jenkins? That's all okay. They can run alongside each other. And in fact, that's one of the really cool things about using something like this is you can launch, if you're doing an upgrade, because upgrades are terrible, you can launch two Jenkins simultaneously, just clone the storage backend and say, hey, I wanna bring all my data over here and test it with the new version. And then you can test it with the new version when you're ready to do the cutover, you can do that. So it's actually much easier to do an upgrade. So you're not doing an in-place upgrade anymore. You're really doing a migration, but you're able to test it super easily because you can able run a bunch of different Jenkins is all at the same time. Right, so the question is how are they referred to internally? I think we just depend them with a number. I'd have to look at that though. Yeah, so yeah, so you can run multiple instances of these all alongside each other.  It goes into a version control system. From there, it goes into your continuous integration system where it's gonna do the testing. Since we're using containers in the testing, it's going to build a container and that needs to ship it off to a container registry. In the original version of this talk that all our sales guys give, it sends it to Docker Hub and GitHub. But I like open source, so that's why I'm using GitLab for everything here. It then does container orchestration to actually deploy into the production environment. So it goes and takes the container that it just built and then loads it into production and then you've got a load balancer in front of that. So putting in the actual details of this. Your git push sends it to GitLab, BitBucket, GitHub. That's where your code lives. Then it gets sent to Jenkins and you can upload it into GitLab, Artifactory, Nexus is another registry for container images. From there, it can use Marathon or VAMP, which I'll talk about in a few minutes to actually do the deployment because there's a bunch of different deployment ways you can do deployments. And then DCOS and Mesos does the actual container orchestration work. And then you've got Marathon LB, which is our load balancer, which is what actually serves things to the world. So we had a customer, I wasn't involved in this project, but what they were doing is they were building a DEB package for all of their deployments. So their process was building a DEB package, pushing it to their Debian repository. There was some sort of syncing problem. I think they had like a load balance across their DEB package repositories. They'd wait for the repository to be ready. They'd boot a new AWS instance. They'd turn the instance into an AMI and the AMI is what they actually used to launch the server. So this process took about 30 minutes, which may not seem like much, but once they put everything into containers, they stopped building Debian packages. They just immediately shoved everything into a container and ran that container through the whole CI process. So that's the container they tested, that's the container they put into staging and the same container went into production. And they said it takes like less than a minute now to run this whole thing, which was a huge win for them because they were a big company with a lot of changes to their software. So having to do a lot of this launching and creating an AMI and updating your Debian package and all that stuff was just a lot of overhead that they don't have to do anymore. So that's all sort of like the basic stuff with the CI CD system in containers. In DCOS and Mesos, we have some ways to do some of the more advanced strategies. And the ones I talked about in the abstract for this talk were Canary and Blue Green. In case you're not familiar with these terms, these are deployment mechanisms. So Canary is where you release a small number of nodes with the new version to your customers and to whoever is using your software. This allows you to test them on a customer and see if anything goes wrong. A lot of big companies actually do this. Like Google will do small rollouts of new software to just specific users. And then if something goes wrong, they can pull it back and they've only impacted a small number of users and the rollback is really easy. So that's a pretty common way to do these deployments now instead of just pushing everything into production all at once. Another one is Blue Green, if you have the resources for it. It actually runs two production ready infrastructures side by side. So you'll run one that's running in production and then the other one is the one that you upgrade and when you're ready, you flip over to the other one. And now you're running in production the new version. So if you need to revert, you just pop over to the other one which is still running the old version but if everything goes well, everything goes well and everything is running. And then you can upgrade that other one so they're both at the same until you're doing the next upgrade. So, and yeah, it's, I have these linked, I'll provide the slides later but there's really good articles on Martin Fowler's site about how these work and how they came about. So in DCOS, we have a couple of ways of accomplishing these. The first way is just doing it totally manually. This is what a lot of customers end up doing eventually once they understand Marathon. So Marathon is what handles actually the deployment of things. You have Marathon definitions to install services and we can actually look at one for Jenkins in a few minutes but this allows you to set a custom configuration. You can tell Marathon like launch this many systems when I do a deployment and allows you really fine tune control. The trouble is Marathon's kind of complicated and it's a little complicated to write this stuff yourself but we do have a documentation site that you can check out to sort of get started on blue green deployment ideas for Marathon. There's also a lab that one of my colleagues wrote about two years ago and two years is like 20 years ago in Mesa's time. So I don't think the lab actually works anymore but you can get some ideas from the Marathon definitions in the lab to start crafting your own to do one of these like blue green deployments or play around with the canary stuff in Marathon. If you don't actually wanna do all of this yourself, there is a piece of open-source software called VAMP that I quite like a lot. It still hooks into Marathon but it does a lot of this stuff automatically for you. So it has a concept of blue green, it has a concept of canary builds, the deployment. So you're able to do it more automatically and there's a lot of changes in the interface you can make. So there's documentation on all these various components and how they work together and then Mesa's and DCOS specific documentation. VAMP also has Kubernetes support. So if you're using that you can also use VAMP. Of course it doesn't hook into Marathon but it hooks into Kubernetes own scheduler. And then there was a really good talk by Julian over at Microsoft about using DCOS and VAMP together at Mesa's Con in Europe and Prague last year. And it's from October so it's still pretty fresh. He's got an open-source project that has all of the code for his talk and I'll share the slides again but that's the YouTube video. He does a really nice job of explaining how you would do this deployment with VAMP. I was gonna do a demo in my talk but then I'm like why would I waste time on that? He already did it and it's a great presentation. So I've got like 20 minutes to get this demo done.  So I'll explain what I'm doing as I get started because I wanna actually make sure this goes well so I have my notes here, all right good. So I'm gonna just make sure I'm still logged in. Yes, all right. Someone around this command, can you see that? Do I need to make it bigger? Pretty good, all right. Ah, thanks from the back. So I just typed DCOS node and this will show me all the nodes on my cluster. So this is the CLI. You can also go into the web UI over here and you can see the nodes and this pretty much has the same view. You can do DCOS service and right now I don't really have anything running. I've got marathon running which is just the scheduler and then metronome which handles like one-off jobs and tasks. It's kind of like cron for a cluster.  So I'm gonna install marathon LB, blah, blah, yeah, whatever. So I'm just gonna install this. So then if I go back over to here, if I look at my nodes you can now see, hey, there's something happening here. I go back to my dashboard, you can now see that of my CPU allocation two of 16 are now allocated to this container. So the next thing I'm going to do is I'm gonna install Jenkins and I'll show you this by going to the catalog through the UI because I actually wanna make some changes and in order to do that on the command line I actually have to edit a JSON file but it's a lot easier just to do it here if we're doing a demo. So here I'll show you the catalog first. So we've got a bunch of certified packages and then we have ones that are handled by the community. So Jenkins is one of our certified packages. That means we have a partnership with Jenkins to make sure it's running a decent version. Oh, I was gonna do something else first. I wanna find out what node to run it on. So I don't wanna run it on our public node. So where's Marathon LB? Hey, it's still deploying, why is that? All right, good. So let's see this one. So Marathon LB is running on our public agent and I don't wanna put Jenkins on the public agent. So if I default, Jenkins will just land on whatever container I want or whatever container it wants but I'm actually going to pin it to a specific one because if it reboots and goes to another container I don't have like shared storage on the back end set up for this demo. So if Jenkins pops over to another server it's not gonna have its storage. So I'm just gonna make sure it's not being launched on 236. So I'm gonna pin Jenkins to a specific server when I launch, or specific container when I launch it, or agent. Jenkins. So I'm gonna do that by going to the storage one. So here, storage, this is where you would go in to help to figure out where the host volume is. So by default it just goes to temp inside of your container which is not good but this is a demo so whatever and I'm just gonna add it to that host name. If you open up this JSON editor you can actually see what the JSON is and this is your Marathon definition. So if you wanted to do this manually or wanted to upload this from the command line or you wanted to edit it anyway this is what you would do. I mean normally in production people don't use the UI very much. It's used for playing around and getting approval concept but in reality people just have a whole stack of these JSON files and that's what's in their infrastructure. So I'm gonna go ahead and run this service. And so that's Jenkins running and you're gonna see the resources go up on here. I'm also gonna install GitLab. This is one of our community maintained packages. So I'm gonna review, oh I was gonna find a node, yeah review run this one. So I'm gonna make a couple of changes on here. I'm going to add a virtual host so this is gonna be cd.princesslaya.com and I'm gonna change the port because my cluster is running on a infrastructure that doesn't actually have port 50,000 open and I'm just gonna grab one of these to, I'm gonna just put it on, I'm gonna pin this one to a specific agent as well because again I don't have storage in the backend setup. So I'm gonna go ahead and run this. So these services are now deploying. I also didn't set up any SSL certificates and something that I learned about Docker is it really doesn't like using insecure registries. So some of the demo magic I did before getting here was telling Docker on all of my agents before launching anything that it's allowed to grab from an insecure registry that I set up. But we're also gonna have to do that inside of Jenkins because Jenkins also needs to pass along that information to Docker in order to actually do the deployment from an insecure registry. Obviously in productions you would use a secure registry because you'd actually have SSL certificates set up for all of this stuff inside of your infrastructure. So of course this is a demo so this is gonna take longer to deploy than normal. But it's coming along. Let me see what I can show you inside of this, inside of this here. Yeah, so we got our dashboard. It's showing how things are going. Jobs, this is what metronome powers. This will be able to run one-off jobs. So if you have a cron job or something, you can set this up. We're actually having it running internally to send out an email every week reminding us that all hands is coming up on Monday and to submit questions. But this can really do anything. It can run all kinds of one-off things. And then the catalog. There's all kinds of software in here. It's pretty good. Artifactory is another continuous delivery sort of thing. Got Cassandra and all these beta ones are just sort of in waiting. But yeah, you can deploy SAF on DCOS and all kinds of other good things. All right, so Jenkins is running. So I did not give a public address to Jenkins but what we can do to open up Jenkins is actually run it inside of the DCOS UI. Come on, internet. And the address is like service slash service slash Jenkins appended to just the regular address that you have inside. So if you look at this address, it is the same, yeah, we just have a service address for Jenkins. And GitLab is running. So I'm actually going to go to do this one first because if someone went to cd.princesslaia.com before I did, they would have been able to set my password for me. All right, but the first thing I'm going to do actually, I have to do a tiny bit of configuration on Jenkins in order to handle the insecure registry situation I have myself set up with. Guess what? We don't have the latest version of Jenkins. Gonna fix that. This is kind of boring but all I'm going to do is set up an environment variable to pass along my insecure registry instead of Jenkins once this loads. I have to hit advanced a few times. All right, so that's what I need to do in Jenkins. Now over in GitLab, I'm going to log in as the root user which you would never actually, ah, phew, okay. So I'm going to create a code repository, a Git repository in here. You would typically set up your own user and not use the Git user or the root user. And I think I call, yeah, I call this scale website.  I'm going to make it public and I create the project. So actually now if anyone wanted to, they could create a user on this instance and submit a pull request to my project but you don't need to do that. So I'm going to just clone this repository and it's empty, of course. So I've actually got all the files that I want to go in there staged over in another folder. So I'm actually just going to copy all these from the other repository over into my scale website repository and then I'm going to show you what I put in here. So the idea here is I want to launch a website in this continuous deployment system. I'm just going to use nginx, it's going to be super simple. So if we look at my Docker file here, all it has is grabbing nginx container and then just copying my index.html to the place where we want to serve it up. Oh, it's all the container does. It runs nginx and serves my website. The index.html, super simple, tells you hello. I've got a Jenkins file in here that's actually going to define what my pipeline should be doing. Hey, it's got some space there. All right, so it checks everything out from the Git repository. It builds the Docker image, it logs in and pushes it up to the container registry inside of GitLab and then it actually does a deployment of the Docker image once it's all been built. Yeah, yeah. And then I want to show you my marathon.json. Again, this is a definition for the application. So this will be my website that's running. This is what it's going to pass along to DCOS once it actually deploys the application. Caps lock. All right, so I'm just going to get add all these files and I'm just going to do like an initial commit here. So we look over, your repository's empty, but I'm going to do a Git push. Hey, come on. And now where my repository is not empty, so just push them up to the Git repository. The registry is actually our container registry and you can see it's empty right now, but it won't be in a few minutes. So we are, we have our, everything's all set up. So we're now going to go over to Jenkins and actually set up the pipeline. So we're going to go to the new item. We're going to call this nginx. Going to do a pipeline and this will be our continuous delivery pipeline. I didn't add a testing component into this because I wanted to make sure that the demo actually got done before we all had to leave. But adding, adding a testing, adding testing into this would just be extending the Jenkins file to actually do a test. Like you might check to make sure links are all working or something on a website. And then what we're going to do here, we're just going to, we're going to set up a build trigger. It's going to just pull Git every minute. It's craw and style. And then it's going to warn me that I'm running this every minute. Probably wouldn't want to do that normally, but I need to get this demo going guys. So here we are. And then pipeline script. We're going to use Git. And because remember I added the Jenkins file and everything in my Git repository. So I'm going to, I don't actually need this root guy here. And then I, so that's my Git repository. I need to add a credential to Jenkins in order. So because when I push it to the image registry it's going to need to log in. So I am totally insecure using root over not SSL again. And then I'm going to call this, I give this the ID of GitLab. So this is my credentials for Jenkins. No, don't make it worse. And that's my pipeline to find. So within a minute it's going to go ahead. So I'm in the nginx project now pipeline inside of Jenkins. And in the build history we're going to see it pop up in a moment because it's going to look at the Git repository. It's going to find that Jenkins file in there. It's going to be like I'm going to use this Jenkins file and I'm going to run my thing any minute now. Yeah, so this is, I mean at this point in this talk we're sort of getting into like Jenkins-y stuff and GitLab stuff. But, and of course you can run other things, Artifactory or you can use GitHub as your code repository. You can use Docker Hub if you want to as your image registry. But GitLab's pretty cool, so there we go. So what it's trying to do now, so it's starting this job. It's looking for an available agent on the Mesos cluster because there's this plug-in that we ship with Jenkins that interfaces with Mesos directly. So Jenkins is able to run something on the Mesos cluster and it's able to spin up its own containers. When I was working on OpenStack we had a tool called NodePool which connected to a bunch of virtual machines across the world and different data centers. And that was fine too. But we did have to require this whole like NodePool tool in order to manage our connections. With this for a really simple like continuous integration deployment pipeline, you can just use the existing Mesos cluster that you have. So I'm gonna open this and see what it's doing over here. It's probably waiting on something. All right, so it found my code repository. It's doing a checkout. So we can start seeing here Jenkins is doing its thing. So checkout is looking good so far. Yay, so now it's going to build the container image. So if we pop back over to the log screen here, you can see it's pulling down the nginx image that it's gonna use. That's how it builds that. And then it's going to publish it. In the publish stage it logs in to GitLab and then it pushes the image up there. And then it goes to the deploy stage. In the deploy stage, it grabs the image from GitLab and then it pushes up into DCOS. So now we have this nginx service running and that is the one we just deployed directly from Jenkins, which is pretty cool because now it's in your list of things that you're running. And then if we open that up, oh man. Okay, just give it a second. It's totally gonna work in a second. There we go. So this is just a little website we put in to GitLab in the code repository a few minutes ago and now it's deployed. Of course, you probably want to add a testing step in there so you'd have an after build. It would probably have testing and then publish and whatnot but wanted to make sure I had time here. So where am I time was? So I've got maybe a couple minutes for questions so. Yeah, all right, any questions? Hi, not so technical question here but just how do you enjoy using the DCOS GUI? And we've sort of been fiddling around with Jenkins and more and more where I work and I just, I was looking at the GUI and it looks really nice to use and how do you like it? Yeah, I like it a lot actually. It's funny, I can actually, I can try to find one DCOS 1.8. The GUI used to be really, really dark and it looked a lot, yeah. So this is our old GUI here, actually it's hard to see but it used to be like dark. That's actually very similar to how Marathon looks so I actually just, I can show you because Mesos and Marathon also have their own GUIs which is super good for debugging because DCOS sort of tries to make things simple so this is the Marathon UI and then you've also got a Mesos UI that are totally separate so it allows you, these are lower level ones you can sort of dig into them but the DCOS UI, like it's super nice. We have a really amazing design team and we just released 1.11 yesterday. We released like twice a year so yesterday was a big day. So we released yesterday and the UI is actually even better than it was. Like the login screen is really slick so yeah, I love the command line so GUI is not really my jam but I actually quite like it. They did a really nice job. They also, I think in the 1.9 release if you try to launch something and it fails, there's like a grid which will show you, actually I tweeted about it the other day, that will show you like what it actually failed on. I was trying to prepare for this talk and my whole demo was breaking all over the place. It was a bad day. But it actually show you like why did it fail? Did it fail because it didn't have enough CPU or it didn't have the right port? Yeah, here it is. Hey, look, there's my cat. Okay, where did it go? Well, now I lost it. Oh yeah, there we go. Yeah, so this is like a resource offer screen so it'll show you, it's kinda hard to see but you can see those little check marks at the bottom. It went through every agent on the list and in one of them it, you know, the role wasn't right. That meant it was a public agent and we wanted to run it a private agent. One of them, some of the other constraints were right. Everything, all the CPU and stuff was good but the one I wanted to run it on didn't have the right port. So it has this resource offer screen which is really nice for like just first glance like debugging why am I not able to launch this thing anywhere. So that's pretty cool. Question. Yes, did you say earlier installing DCOS also automatically installs Mesos? Yes, yeah, so when you install, yeah, DCOS it brings in Mesos automatically. It's part of the installation, yeah. Anyone else? All right, well I'm around all weekend so come say hello and ask me questions whenever. Oh and I can show you the registry here.


This lecture is part of an online mathematics course on group theory and we'll be covering a few examples of groups, mostly of order 120 or 168. So first we'll look at order 120 which is 2 by 60 and there are plenty of groups of order 120 but we're only going to be interested in the ones that involve the icosahedral group of order 60. And there are four obvious ways to write down an interesting group of order 120. We can take the group SL2 of F5, 2 by 2 matrices of determinant 1 over the field with 5 elements. So the number of elements of this is 5 squared minus 1 times 5 squared minus 5, that's the order of gl2 F5 and then we have to divide it by 4 because we're taking numbers of determinant 1 so this ends up as 120. Or we can take the group that the binary icosahedral group, so you recall that we take the map from the unit quaternions to SO3 of the reels and we take the icosahedral group here and take its endless image here and because this is a double cover, this is twice the order of the icosahedral group which is 2 times 60 which is 120. Thirdly we've got the group S5 of all permutations of five objects and this is order 5 factorial which is equal to 120 and another obvious thing we can do is we can just take 2 times the alternating group A5 which is order 2 times 60 which is also 120. So we've got four groups of order 120 and we can ask which of them are isomorphic to each other. Well one way to sort them out is to look at the Seal of 2 subgroup. So if we work out the Seal of 2 subgroup of these we find here it's the quaternion group, here it's the quaternion group, here it's the dihedral group of order 8 and here it's a product of three groups of order 2. So we see that these three groups are indeed distinct although it's really easy to confuse them because they're all quite similar as we will see. On the other hand these two groups are in fact isomorphic so these are really the same groups as each other. So if you've got a group of order 120 you can ask which of these groups it does it correspond to. For example we can take the group of all symmetries of an icosahedron so I guess again the group of all rotations of an icosahedron has order 60 but you can also do things like reflections of the icosahedron so this is another group of order 120 and if you work out its Seal of 2 subgroup it's z over 2z cubed so we can strongly suspect it's isomorphic to this and in fact it is because the symmetries of an icosahedron has the symmetries plus or minus one and then it has the rotations of the icosahedron so it splits as a product like this so it is in fact 2 times a5. There are three ways to build a group out of a group of order 2 and a group of order 60 you can think of the binary icosahedral group as having the group of order 60 kind of sitting on top of the group of order 2 because the group of order 2 is a normal subgroup this one you can think of the group of order 60 as having the group of order 2 sitting on top of it because the group of order 60 is a normal subgroup and this group they sort of sit side by side because they're both normal subgroups and you remember this is very similar to what happened when we looked at groups of order 24 here we had the binary tetrahedral group that had a group of order 12 with the centre then we had the symmetric group s4 again of order 24 and finally we had the group 2 times the tetrahedral group which sort of you can picture is looking like that so the the tetrahedral group and the icosahedral group kind of behave very very similarly they can both be combined in three interesting ways with a group of order 2. There's a one particular application of these groups is to take the binary icosahedral group so let's take g to be the binary icosahedral group and we know that g is a subgroup of the group s3 of unit quaternions so we can look at the quotient s3 modulo g and this this group here is just a three manifold and this group acts fixed point freely on it so this is a three manifold compact one that's rather famous one called the Poincare homology sphere and now there's you can form the quotient of s3 by any discrete group there are quite a lot of them for instance you can take a cyclic group or the binary tetrahedral group so this is always a three manifold and its fundamental group is easily seen to be just the group h so we can obtain lots of three manifolds with finite fundamental groups so the fundamental group is the first homotopy group and it also has a homology group h1 and the homology group is just the abelianization of the fundamental group pi1 now the group this group that the binary icosahedral group has the property that the abelianization is just trivial this is quite easy to see from the fact that the group a5 is simple so the abelianization must be contained in the center of order two and the abelianization can't be of order two because the binary icosahedral group has no homomorphisms to a group of order two so this group has vanishing first homology and Poincare when he started looking at three manifolds he originally conjectured that if a three manifold has the same homology groups as a three sphere in other words it is vanishing first homology group then it is a three sphere and a bit later he found this counter example well so this group has the same homology as a three sphere but it has different homotopy groups so its first homotopy group is order 120 and he then made the notorious Poincare conjecture that any compact three manifold with the same homotopy groups as a three sphere must actually be a three sphere and this was proved a few years ago by Perlman continuing work of Hamilton as a very difficult result so now we'll discuss the groups of um well a group of order 168 so we've seen that the first simple group has order 160 the next simple group has order 168 and there are two obvious ways of writing down groups of order 168 we can take the group SL3 over a field with two elements and this is order two cubed minus one times two cubed minus two times two cubed minus four which is seven times six times four which is 168 and we can also take the group PSL2F7 so it's a seven not a four and this is order seven squared minus one times seven squared minus seven and then we should divide it by six and then we should divide it by two because we're taking the projective group and this again has order 168 and these two groups are actually isomorphic although this isn't terribly easy to see if you try and write down a homomorphism for one of these groups to the other you'll find it's not at all obvious how to do this I mean you know this is matrices over a field with two elements and it's rather hard to get from that to matrices with fields over groups over a field with seven elements this sort of isomorphism is one of the many things that makes the classification of simple groups rather tricky because there are four groups of small order there are all sorts of accidental isomorphisms between groups for example you might start trying to classify simple groups by trying to work out what their characteristic is because they should be you know some sort of matrix groups over fields of some characteristic and examples like this show that it's kind of tricky to figure out the characteristic of a group because this group has two different characteristics it's a natural matrix group over fields of characteristic two or seven anyway we can write down the some things acted on by this group first is the pharno plane so this is the projective plane over the field of two elements so it's actually done by sl3 of f2 and you can draw a picture of this as follows it's got seven lines and seven points and here are the seven points of the plane and I've also drawn in the seven lines except there are only six of them so I'm going to draw in the seventh line like this so this is actually a straight line in the plane although it looks like a circle because there's there's no way to draw it all with straight lines and if you check the if you look at this graph you can see it's got a lot of automorphisms and if you look carefully you can actually see if it's got 168 automorphisms and is in fact isomorphic to these two groups here another way this group turns off in as this is the automorphism groups of the Klein quadric which is x cubed y plus y cubed z plus c cubed x equals naught so this is a um a quadric that means a degree four curve um inside um the two-dimensional complex plane and this quadric also has autumn automorphism group of order 168 which isn't all that easy to see you can easily see a subgroup of order 21 but getting the remaining automorphisms is kind of difficult um this is an example of a Howitz surface so if you've got a compact Riemann surface s of genus g greater than one then Howitz showed that the automorphism group of s has order at most 84 times g minus one um and there are no examples for g equals two and for g equals three there is this example here so for this surface has genus g equals three and its automorphism group is exactly equal to 84 times three minus one which is 168 and it's the unique Riemann surface of genus three with an automorphism group this large um so uh i think that's all i want to say about these groups what we'll do next lecture is discuss simple groups in a bit more detail and give the Jordan Holder theorem saying that groups can be decomposed into simple groups and the number of simple groups that occurs is unique


Hey, it's Anfa and today I want to do something very different. I want to make a feature request You see I really like EQ 10Q equalizer But what I would really want is to be able to you see make a few bands say not just few peak and now I would like to be able to using two knobs just shift the whole thing to the left or right or scale it up and down so like have like a A knob that will squeeze this to the left and just move all the bands respectively So I could basically Program a formant filter it and then shift it around that would be really useful for sound design Eternity later Yeah, it doesn't mesh well together yet, but I'm gonna figure this out so yeah Please implement something like that. Maybe in EQ 10Q maybe in calf equalizer allow to like Control with two knobs just you know the bottom frequency of the whole spectrum and which would scale the whole all bands up and The top which would scale all the bands down and then with just these two I could automate some really awesome effects or maybe even do formant processing with vocals and whatnot that would really open up a lot of sound design possibilities and allow for more use of DQ's as sound design and creative Tools not just for mixing Mmm, maybe what would be really awesome is to be able to control that via MIDI input So I could play notes and the notes would shift all the bands accordingly so that my equalization stays in tune With harmonics of my instruments that would be really cool, too Anyway food for thought Thanks for watching and I'll see you in the next video and The live stream which is in today's every month Sunday. Bye