Welcome. My name is Adam Miller for those who don't know. I work on the third engineering team at Red Hat. I do a lot of things in as much as I can. Most recently it's been around container build system and automation. I'm going to talk today about the multi-architecture version of the layered image build system. I want to preface that with the fact that I really, really tried to have a live demo for Yalta Day. It's just not going to happen. So today's topics, we're going to define containers. I like to do that briefly just for anybody who is new to the topic space, who doesn't have experience in it, knows what on earth this stuff is, what we're talking about. Go ahead and specify the difference between based image layers, images, and then why Fedora containers. What's the motivation? Why are we doing this as a project? Why are we spinning our wheels on this? And then why multi-architecture containers? Why does that matter? A quick fun history lesson about the container build system within Fedora and then now to kind of break out where we've come from to now where we are today and then where we're going. Then we'll get into a little bit of how it's built and one of the main components built with is OpenShift, so we'll define it and we'll go ahead and very briefly describe what that is. I want to define release engineering just because it helps frame and identify why the system is designed the way it is in certain aspects and why certain decisions were made and what the motivation behind some of that was. And then how does it all work? How does it all work today and then how is it going to all work for multi-arch? Because right now the multi-arch stuff is inactive development in the Fedora staging environment so we're still pushing towards the goal of having it in production. So very quickly, what are containers? Containers are generally referred to as operating system level virtualization and what that means is that we can have multi-tenant isolation of processes and what we can do is we can more or less forklift software and confine it in a space and make that it think that it has a view of the world that it doesn't actually. We can lie to it and say that when it queries the prop file system it has its own specific view because it's been named space to offer. We're providing constraints around it in order to control it and have finer grained restrictions on it. And it also forces the ability to kind of wrap that in whatever kind of abstraction we want and I'll get into a little bit more specifically how we're doing and the implementations we have but we kind of get to wrap it in ways that we want to execute various different platforms on one host. So containers are not new and I will make the argument that the original container was the Cherute in 1982 and you may or may not disagree with me on that. I will completely admit that it was a very unsophisticated container but it allowed us to basically take a piece of software and lie to it about the environment in which it was run, the context in which it existed and executed. It thought they had a world view of the system that wasn't true and that's more or less what we're doing with containers. Now they're exponentially more sophisticated these days so we have copy and write, we can do quotas, we can do IRA limiting and all sorts of things. Brief, non-exhausted history of sophisticated unix-like container technology. I would wager and tip the hat that this started in 2000 with FreeBSD Jails. Then Linux vServer and O1 Slayer Zones and O4 OpenVZ and O5 O8 LXC happened. IBM created a toolset for LXC which basically was a user space toolset that allowed us to talk to and have kind of a really interesting grasp on kernel features in Linux. And that's where I like to think that things got interesting because that is kind of what was the catalyst to a lot of our current tools that we have and a lot of the change in the way that we are trying to run systems and implement a lot of this stuff. So 2011, SystemD N-Spawn interestingly enough did nothing with LXC but used a lot of the same back-end technologies. And a fun fact I would like to tell N-Spawn apparently was originally created because they wanted a way to be able to test SystemD without having to repeat booting systems. So N-Spawn is actually kind of a containerized technology. And then until 2013 DotCloud released Docker, later renamed themselves to Docker Inc. I think that was probably the big one that got everybody really focusing because it provided very low barrier of entry, standardized image format, and then a way to actually like move these things around and pull down these images, run them, destroy them very ad hoc very easily. Until 2015 Run-C was really some of the purview of the open container initiative. There is now this documentation out there that defines what a container is, how it should be run, and Run-C is a reference implementation of that. Then until 2016, ContainerD was a Run-C orchestration daemon. All of these things are now how the Docker is actually built or Moby if you're following the upstream component. So, layered images versus base images, when we get into container world, you have layered images and base images. And base images are built from effectively nothing. They start with basically a blank slate. You define a root file system within them with everything you need. If you want to break it down, you can actually have a single process in that. Languages like Golang that are statically compiled are kind of making that popular in some use cases where you can actually have a base image that is from nothing and just runs this one process. That's valid, but for most people who are transitioning to a container world, what they start with is a base image that's based on an operating system distribution that they're used to using. So what we have in Fedora Space is we're going to have a Fedora base image that you can then build upon. You know, CentOS or REL or Debian or OpenSUSA, they're going to have those as well for those people who want to use alternate other distributions than what Fedora is doing. But there's a distinction because you have the base image that is effectively your starting point for building what I would call more sophisticated applications. And that's not like kicking the shins to saying that your shell running core utilities isn't sophisticated, but if you're trying to run a service that you're generally going to write that and build it on top of what the base operating system provides. So your base image is going to be the thing that your distribution release engineering group is generally going to produce and then layered images of what the rest of us are going to build on top of to create things. And I like the example here that shows that we can basically take a Fedora 25 app, put it on a Fedora 25 base image, and we can move that entity around from a Fedora 25 host to a Fedora 26 host base OS and just run it unmodified as it is. So the reason I say that's a distinction because what we're talking about from the Fedora layered image build system is we are building images that are layered. The Fedora base images today are still built in Koji using Image Factory and that's outside of the scope of this talk, but just know that there is that distinction and we have them separate. So why? Why on earth are we doing this? So why Fedora containers? So we want to enable the ability to deliver Fedora content faster to users. And the next line item is partially, it's a very vague statement. We want to automatically generate release artifacts with security updates and that's very far and wide reaching in the sense that we want to do that across the board. But for the application stacks and the service stacks that we want to deliver, that falls under the purview of this build system because it allows the ability for all of these new deployment types and all of these new, you know, you've got your green blue, your red black, you have these different strategies for deploying things. And instead of making the users pull all of our content and build all of these things themselves and replicate, you know, having a build environment and then having to track dependencies so they can kick off and make sure that they always have the latest version of everything. We want to be able to do that on their behalf so they can just get to use it just like they would use, you know, a DNF update and traditional system. And then also lowering the barrier of entry for contributors. So we currently do a lot of legwork to repackage things over and over again in ways that are already packaged. And that makes a lot of sense for a traditional operating system for a traditional Linux distro because of the fact that you really don't want to have applications clobber the system. You don't want, you know, let's say something that you pip installed or NPM installed or Ruby gem installed or insert thing here. Installing, you don't want that to clobber your system. It makes it very difficult to manage updates to maintain, you know, a good manifest of your systems do, you know, relatively standard practice for maintaining a large environment. However, if we take that away from the end system and we put it more into a container and that container is an entity that is then rolled out into what you would consider an immutable infrastructure to where if you want to do a change or you want to do an update, you don't actually change something on the system. You rebuild that artifact and redeploy it. And then you just simply restart whatever that process is, running a new context of whatever the thing you're trying to deploy is. So there are some obstacles to actually be able to allow things to be distributed under the purview or under the flag of under the brand of Fedora without being packaged originally. And we have some plans on how to do that. And a lot of that will actually kind of fall under, you know, the reasons that we need to do that will fall under the release engineering component. I'll kind of run back to that. We don't just support one architecture today. So that's like the first and foremost is if we're going to do something as an official Fedora initiative. I like to believe that we will focus and pay attention to all the things that we currently say we will. But also, I think one of the big ones right now is a very large trend in what's going on in technology is Internet of Things. And not many Internet of Things dev kits or packages or, you know, things like pieces of hardware that IoT folks are working on are actually 664. There's just not a lot. I mean, most of them are doing something considering a little problem.  Well, I mean, you know, throwing stones and whatnot. What I'll call the magical arm revolution. I don't know if you spend much time on various RSS feeds that I do. It just seems like they're at least in headlines taking over the world. You know, they still, wait, we don't have hardware, but whatever. They'll ship it next month, every month. Yeah, so in other architectures matter. I mean, there's a risk five bring up that's kind of being grassroots, bootstrapped. There's been kind of a rekindling of a handful of people's interest in MIPS. There are other architectures out there. There's a lot of things that people are interested in. And we don't know what the future holds. If at some point in the future somebody decides that, you know, architecture X, Y, Z is going to take over the world, it just seems silly that we would build a system today that will lock ourselves out of being able to adapt to that quickly. So quick history lesson. If anybody's been to any of my talks before about the later image build system, some of the content I've gone over already probably looked familiar, and this definitely does because I love to poke fun at and or throw Matt under the bus. But the way that the later image build system happened was Matt Miller said that there's an open source layer image build system and we should deploy one. And he's right. There was one and we should deploy one. However, there was a misunderstanding by both of us that it was done and just needed to be deployed. And we just needed, from the infrastructure standpoint, needed to like make some antipole playbooks and roll this thing out. We did not, either of us, realize that it was under active development and that we needed to join in its continued development and help get it somewhere for the finish line. So we did that. And then phase one, I would say phase one is probably about 18 months ago. We had single node builder. We finished that in a few months. Image format, v2, registry v2, manifest v2. That broke the original implementation. So when v2 rolled out, if anybody is familiar with any of the, so what I talked about previously with the way that there's like these standardized ways to have an image created to transport that image and have the metadata about it, that's these things. Well, the specification went from v1 to v2. This was incompatible. Universal truths that were true about v1 were not inherently true about v2. And very large portions of the build system had to be rewritten from scratch or heavily refactored for the components that were still legitimately useful, but just were broken. There was a lot of refactoring. So phase two, let's go ahead and bring it up into the future. Grab the v2, manifest all the different v2 formatting. Do a scale out deployment because now that we have a better understanding of what we're doing, we can go ahead and scale this out into a way that will make sense for the future. Automate tests that can be tied to the output of OSBS. This is actually none. Nobody uses it, but we have. There is a testing mechanism. I would like to very much so thank Tim Flink, because he worked really hard on this for a very long time, and then nobody used it. So, I'm sorry, Tim, but thank you very kindly. And Relinch is able to promote images from the candidate registries to production. We have a formal process for that. There's Koji tags, and things are tracked in a way that are relatively similar to the way that other artifacts in Fedora are. For a while there, there was kind of like a special off to the side thing, and we're getting more in line with the way that release engineering does things. Phase three is what I would say is happening right now. We have image registry scale out that is done. We have multi-master image registry. It's running on top of Gluster Storage. Search and advertise for image registry. I know if you go to registry.project.org right now, you'll get a web UI. I'm sorry. I'm actually helping to track down an individual while I will not name and throw them under the bus on tape, but I'm trying to track somebody down because we just need one patch, and they had said that they would do it and hopefully we can get that done and merge soon. CVE and security metadata for updates. That's something that we still want to plan. What's very interesting about containers is there's basically a tar ball with some metadata, and there's certain metadata that is specified that you need to have, but there's a lot that's not. For example, how do you version these things? That's an unanswered question. There's no specification for that. You define a tagging strategy that makes sense for you and your team and you go with it, but it's not universal. It's not ubiquitous. Everybody's not going to agree on it. You can't just go to somebody else's registry, whether that be the Docker Hub or key.io or somebody else's and search for a thing. That's Postgres. You will probably find a couple of dozen Postgres, and they might not all be versioned the same in the same format in the same way. That's an interesting challenge and we're kind of working on that stuff. Phase four is going to be OrbStrader and worker architecture, and I'll talk about what that means and why that is important, and then multiarch. Phase three isn't inherently done yet, but phase four has already begun because of the necessity to get moving towards the multiarch truth, and that will also allow us to take out certain components of the build environment that we don't require. And because it doesn't inherently affect phase three, we're not going to put a stop to some of that work. Very quickly, OpenShift is a container platform built on top of Kubernetes. What is Kubernetes? I'll talk about that in a second.  The only one that we simply care about is the build pipeline. I'm sorry, the only three that we care about, build pipelines, image streams, and triggers. So a build pipeline is basically a primitive that allows you to pass in some information. OpenShift turns it into a thing they call a build. They build it for you and they spit out some stuff. One of the inputs into that is an image stream. So we talked about how the container registry has this standardized format for how images can be transferred and moved around while an image stream basically is an alias for that that can monitor it for changes and then logs an event. And what's interesting about events, we can trigger actions off of them. So that's where the triggers come in. So one of the things that we're going to have is triggers that are used for our various aspects. And we could potentially use that for things like generating CVE data and that kind of stuff. So we're interested in those things. Fun diagram. If you look at the green section, build automation, that's most of what we're interested in. If anybody has questions about all of this stuff, we can talk about that later. The main reason I put this up is because I did want to point out that there's a lot going on here. OpenShift is a very powerful system. Kubernetes is a very powerful system. There's a lot of capability there. And what's interesting about the layered image build system is we only really are using a very small fraction of that capability. But a big reason that we use this is because of the image streams, the triggers and the fact that it is a cluster that can be scaled out. So as the build system that Fedora is using becomes more popular and we need to put more hardware behind it, it's pretty easy. We can just keep adding more nodes behind the orchestrator and it'll span to our heart's content. And I say that and I think there's, what, thousand nodes? I think there's thousand node systems now and we have a three node system. So we've got room to grow. We'll be fine. So really quick, Kubernetes overview. You have master and nodes. And this is what I was talking about. So when you run a container inside of Kubernetes, you have this idea of a pod. Container runs inside of a pod. And that's where our builds will run. So we schedule this unit of work inside this pod and our builds run there. And then the master is where the orchestrator happens. So when we need to scale out and we need more build capacity, we need to be able to run more of our build pods. We can just add more node hosts behind it and scale out there. And the reason that this... Nope. Ah, I put them out of order. Sorry. We're going to skip that slide. I'll come back to it. So OSBS is built on top of OpenShift Origin. So if you look inside of the OSBS thing, OSBS is the OpenShift build service. It is a thing that puts together various components to create a custom build type that we feed in OpenShift. So OpenShift is one component. We add some things to it and we create what we call a build root. And that is where the build actually happens. So we take advantage of the build primitive. We rely on OpenShift for the scheduling and orchestration. We enforce the inputs come from known, valid places. So if you try to create a build that just does a curl pipe to bash, we will probably fail that build because we have actually resources that we green light. Isn't that it? Yeah. Well, thank you. Okay. And then we have a build root. Our build root is the unlimited Docker runtime. So for everybody familiar with doing RPM builds in Koji, this is going to be kind of a similar idea of a build root for that. This is the minimal set of tools that are required to be able to actually run the other set of tools that are needed to perform the build that you have to ask of the system. And we create that and that is the baseline of every build and every pod that goes on inside the cluster. So we firewall constrain at the Docker bridge interface and that's how we actually do our isolation for various inputs is there are no entrusted inputs. So for example, you know, DistGit in Fedora space is going to be an accepted base, official Fedora repositories, that kind of stuff. Inputs are sanitized. If for any reason you pass something that is malformed or misinformatted, I'm not going to say it's perfect. You could potentially get around to what we've set up, but so far for the most part has been good. Okay. Yep. Atomic reactor is another component. Single pass Docker build tool has a lot of really fancy features that we really like and the main ones are the fact that we can change the base image and that we can inject repositories. So right now for any reason that we need to do like a build override or something, we can actually inject another source of information to be valid. So it's the build system. So here's Fedora's implementation as it stands today. So we have Fedora later in between containers. They interact with DistGit just like RPM maintainers do. They send a container build to Koji. Koji then kicks it off in OSBS and then that magically happens in the registry. Let's see if people take pictures all the way. Cool. Okay. This is what it's going to look like. This is what it looks like in stage right now. It just doesn't work yet. So this should all look familiar. This piece down here is the same. This piece over here is the same. But what has happened is we now have an OSBS orchestrator cluster and then we have worker clusters. And we will have a separate worker cluster for every architecture that we care about. What's going to happen is Koji is going to kick off a build in the OSBS orchestrator. That OSBS orchestrator will then in turn kick off a build for each of the architectures that we care about. And then that information will flow back into the orchestrator OSBS and then that information will flow back into Koji. So Koji will remain our source of truth and we will get to a place to where, just like today when you kick off an RPM build, it will build for all the architectures.  And this allows us, I probably should have found some space to say like other architectures down here, but I just picked a handful of them for demonstration purposes. I'm not saying that these are free, we will support, but yes. Is it going to be necessary for everything to be RPMs before you can build a container or will you be able to pass them again and build a container for something that isn't necessarily packaged as RPMs? Today, yes, it's required. In the future, so that was actually a few slides. Yeah, so the lower your barrier of entry for contributors, that's what I was talking about. I probably should have spelled it out a little bit better, but right now, yes, it has to be all RPMs. In the future, one of the goals is for that to be... I mean, the limit is not in OSBS, you can upload it into Lucaside Cache and unpack it into Docker file and use it in there. Right, no, no, yes. So it's not a technical limit in OSBS or in the layered image build system. We can technically do that today. We currently lock it down to not allow that. And that's actually, so this is a good moment to do the really quick... What is release engineering? Main software production pipeline that is reproducible, audible, definable, and deliverable. And those attributes we want to maintain true into the future, those are things that we want to keep and not throw away. And one of the big problems and one of the big challenges with allowing things to not come from RPMs is the ability to recreate them in a way that can be audited later in the future. So what we do is in Koji we have... So for the OSBS, the Open Chip Build Service, what that happens there is when Koji kicks off the build, when Koji kicks off the build, when OSBS is done, it has a successful build reported from the worker node, the orchestrator will then do what's called a content generator metadata import into Koji, so all of the metadata that would potentially be needed to recreate that build environment, that build route with those specific set of inputs is stored in Koji. So Koji is still the source of truth. We can realistically, at endpoint time, once the Ansible Playbooks are set up and everything you can fix are good, we can turn key destroy OSBS on the back end and just relaunch it and then we can just keep building. That's the goal. However, the rebuildable part and the metadata to import back into Koji, if we say that we allow things to be able to like pip install or go get. So if you did a go get, those are pulling things from random places and those random places on the internet can disappear. So we lost the ability to truly audit and we've also lost the ability to recreate. So what we need is a way to have content streams that are curated. And what's interesting about that term is we have a wiki page somewhere in Fedora that talks about that exact topic and a way to solve it. I just don't know whatever happened with that project. So at some point in the future we will find a way to bring that project back to life or we'll start a new one because this is something that we need. Can I just add real quick that this is also partially there's been work on the OSBS side to support pulling content other than RPMs directly from Koji. So if we have a system that's either building or importing Go artifacts into Koji, they could then be pulled directly into a container build in such a way that it would be audible that a reference to those artifacts would be included in the metadata. So for the video one, this line of questioning started because he asked if we are required to have RPMs built two, the comment was there's currently work on going to allow from the OSBS upstream to allow for content sources other than just RPM repos to be injected into the build environment for a container. So we could theoretically get to a point where we can then have that input. There's another guest. Cooper, this is a bit more of a question about the actual architecture of OSBS here that we'll see the answer later. But is there a reason you decided to go with a multi cluster approach for the different architectures as opposed to, say, having multiple nodes with different architectures? So the question was, is there a reason for having multiple clusters instead of having various nodes with specific architectures and just labeling them appropriately and having node selectors. For those who aren't familiar, node selectors based on labels. Yes, the implementation overhead of patching all of Kubernetes to understand architecture awareness was difficult because otherwise Kubernetes would just kind of randomly schedule things to run on nodes of the wrong architecture. And that wouldn't go well. So yeah, it was basically from the OSBS upstream of, I think, five people who don't actually have a long tenure of hacking on Kubernetes itself kind of evaluated do we try to implement multi-arch awareness into Kubernetes itself or handle it higher up the stack and the choice was just to handle higher up the stack. That's interesting. I think that problem actually got a little bit better with a couple of recent releases. Okay. Okay, so the comment was that I think the problem actually got better within a couple of recent releases. So this work, the architecture design for this work started back in January, if that helps frame some of the time frame of information that was available. I think it could be switched over very easily. It could be switched over easily if there's a better way of doing it. Oh, I don't know. Because of some of the ways that the bookkeeping work manages between these, it would be a decent amount of work to pivot. It seems to me, at least, that it's designed to I mean, a lot of the metadata transport is happening. I was just, it was up my brief thought that it could be switched over by itself. It probably could be. I just don't know to the best of my knowledge, it would be non-trivial. From what I've looked at in terms of the implementation of the orchestrator to worker communication, basically the orchestrator just like gives instructions and then kind of like keeps an earmark of I'm waiting for metadata I'm waiting for the metadata upload from each of those and then it reports back. I mean, I think the idea of work is all built with standard file that they were Okay. There was a, yes. Can you, is there an example diskit repo or package name or whatever that package container works in just to like, poke at it and see? Yeah, actually we have an entire container namespace in and that's not like a standard that has a spec file. No, it doesn't have a spec file. It's a Docker file instead. What's that good example? Um, figure We'll just state for the next session. I feel like we're in the question and answer space. Yeah, yeah, it's fine. We have wholeheartedly translated Q and A and that's totally fine. There were like two slides left and they're not important. Container, yeah. So here, container engine, why not? Do-do, files, sources. Oh. What? Yeah, what? That might be a bad example. Well, no, because I know this is a thing. Like, now here we go. We have the developer container engine here saying that it's not in figure. Yeah, it is. But, all of diskit is in figure now. There's nothing that But you really want to know more about this Yeah, that's not a good one. It's running an instance of how to get. Yes, question here? We're still on your question. Was that good? Yes. Okay, cool. It's also, cool. Yes. I maintain that. It's a little outdated. I need to update a couple things. It's not perfect, but we do have container guidelines. So if you go to container guidelines. So we have a review process and guidelines for all that, just like we have for RPMs. The verbosity of our guidelines are considerably lesser than that of our PM.  So we just kind of try to work amongst various communities and gather as much consensus as possible and say that that's actually that. Yes, that is actually a really good transition and a good plug. If you are interested in that directly after this, we have a workshop on how to become a container maintainer in Fedora. Josh Berkus and myself will be doing that right here in like 20 minutes. Yes, Troy. You got your container and you have three arches. How does somebody say I want arch 64? You just pull it. So you can use something like Scopio or Docker or whatever and you just pull it because we'll have manifest list implementations. So if you are on an x86 machine, it'll pull x86. If you're on an AR64 it'll pull a AR64. So if I do a Docker pull. Dan, where you at man? I'm sorry, what Troy? So if I do a Docker pull let's say I just have Docker. So the original question was how would somebody know how to get the appropriate architecture once we have multiple architectures building? The answer is the manifest list in the registry should make it such that you will just Docker pull F26 let's say F27 you will Docker pull F27 slash HGTBD and then it will negotiate with the registry what architecture you're on and pull the appropriate one. I think it actually provides metadata in the post now when it does the get it would be a parameter. Cool, question? I have a question so do I have to maintain one Docker file per architecture? The question is do I have to maintain one Docker file per architecture? The answer is no right now it should just be like with a spec file, you just have your one and we feed it through. The kicker there is if we find somewhere in the future that there are oddities that don't work on one architecture and we have to find some workarounds that could change but no the plan right now is just the one. Question? So the question was how do we get using Builda instead of Docker and the answer to that is we already have a plugin for it we just need to switch to it and that is on my roadmap for Fedora's OSBS instance I will speak for nobody else's OSBS instance but yeah we will be doing that and if Dan, if that is not done by DevConf I will owe you an explanation. I have kind of a safely related question so it seems like a lot of containers from packages the Docker file is literally just like install package and possibly set entry point depending on which entry point is there any is there a reason that you need to in those cases we need to have Docker file and all is there a reason we can't just use some of the APIs that like Builda uses for instance to kind of automatically produce. So the question is some of the Docker files are basically just DNF install and then provide entry point for whatever that thing that was the statement and the question was is there any reason that we should necessarily do that as opposed to just allow people to use things like Builda and just define what they need and build it. I mean like it seems like in those cases right you can kind of almost assume an implicit Docker file but not actually need to maintain a separate repository that has virtually the same Docker file across the steps. Well yeah I mean basically okay so what we need to do for that scenario is we would have to define the set of things that we want to just be the DNF install of that package and then an entry point and then we would have to have a uniform defined entry point that we would have to then supply for each one of those things. We could do that it's just a matter of somebody defining what that set of RPMs is that will be containerized somebody writing those entry point scripts because it could just be a command but that is supplied by the RPM or it could be a script that we have to provide and then somebody writing the system that would tie it all together but the way that it's set up now and the idea behind it is that we have this one uniform process that will cover all the bases that we need I'm not against changing it that's up to the atomic working group to do as they please might need me personally like yeah why not let's change the world a lot of this stuff is I mean all of this stuff is made up as we go so I mean if you have proposals for how to change it I'm all in like it's just this was kind of proposed the way to get moving and that's how we're doing it and the other thing too is like so let's say you do have HDPD as you are starting in your entry point and then you want to use that as like the base image for to build another layered image to inherit that one that's another thing too is you can actually create a layered image from a layered image and you can daisy chain up that and you know that just because of your container background but that's a point for a lot of people that affords us new possibilities and those kind of things but yeah I think that it would make a lot of sense but I think ultimately in a lot of ways once we have the fresh maker integration into this stuff that won't matter because fresh maker is going to be doing all that stuff for us anyways oh I'm sorry fresh maker was from the previous talk it's a magical system that's part of factory shadow that automatically kicks off automatically triggers builds within the environment based on changing of content yes great we have six minutes we can make it generate it doesn't today so multi multiple questions yes right now for our container definition our arch is a tag yes it's a label and right now we've been putting x8664 in there yes is there going to be a separate definition file for each architecture or for packages that are available in architectures can we define a generical he answered that earlier well I didn't explicitly I didn't call that out explicitly so the statement was currently in our guidelines we have a definition of the architecture and currently we define as x866464 the question is in the future we're going to have to define that explicitly for each one the answer is no I think we will go the route of rpm in the sense that if you we will remove the requirement to define that we have to patch coji container build to remove the requirement of that and it will default to some value however if you need to specify only certain architectures we will then in that case introduce that because just like in you know rpm so we can do an exclude arch or explicit define arches we'll want to do similar thing for container builds just because that in reality sometimes for example like there is no docker on powerpc64 there is on le on little indian so pbc64 you can't build kubernetes or openshift today because they rely on docker so I mean there's just there's going to be certain things that the other department question you almost answered my question once the limits on those things do you have to have openshift be able to build before you can build on each of those architectures the question is what is the limit on those things do I have to have openshift be able to build openshift right now doesn't build on powerpc64 le it does I have it ok it technically does it's not supported yeah I know a lot of what we're going to do is not supported that's kind of what we do in fedora oh ok yeah 32 I mean nothing but x8664 is technically supported by openshift origin well in centos we got the merge64 right I mean I mean I mean in fedora so let's actually ok so you're going good I've got s390x I've got how commercially supported does it work as long as you've got it to build yeah so we've got AR64 it's the first phase so we've got AR64 RV7HL I686 powerpc64 little indian x8664 and then what is it s390 s390 is it dot yeah so if you go to the f27 build you'll have s390 because I think you just went to an f26 build I did you're right yeah so f27 and you'll see s390x in there as well yeah s390x so we're building for everything right now that's also we might talk to you about the problem make it pen on centos me yeah I've got a handful of patches to the build scripts to make it work I didn't end up having a patch openshift itself but the like all the like 10,000 lines of bash because apparently everybody in kubernetes and goling just hate to make files we're like it's the lineage that's been working for you for 20 years we hate everything what's up with these patches right oh god yeah actually just I mean since we're talking about it yeah it's actually it's an impressive thing but it's like a whole there's a whole namespaced so let's say like binaries yeah so if you look at the functions there's like function definitions that are namespaced there's like OS colon colon build colon colon binaries from targets and stuff and like there's a lot of this and it works well like I'm not gonna knock it except it's just difficult to debug when it fails because I mean anyone in the room who's ever attempted to debug an overly sophisticated bash script will attest to the fact that it's just hard sometimes and then I mean kubernetes does this like openshift does this because kubernetes does this and why reinvent the build system because the project you're based on already has one so yeah I don't knock them it's just like I've fought this thing enough that I have a little personal care about the pain yeah so the answer to the question is we will build on everything that we can we have one minute any last questions cool thank you


So yes, I'm Chad, I'm a graduate student in Tallinn, Estonia, Tallinn University of Technology, and this talk is about a proposed sort of foundation for these ledger systems that we see in blockchain technology. And I'll just start off by explaining sort of what I mean by that this is probably a bit different than most of the talks. And the thing to start off with the thing I'm going to assert is that these distributed ledger systems have two parts and the two parts that they are sort of orthogonal to each other they interact a little bit, but not a lot. And the two parts of course the distributed consensus protocol and the ledger that you use the distributed consensus protocol to maintain. So, think about the second part in this time care about the consensus protocol we're going to talk about the algebra of these ledgers. And in particular, we're going to see that the algebra of these ledgers is to be found in something called a monoidal category. So we'll have a sort of correspondence between a notion of ledger and what we call a symmetric monoidal category. And before we start again we're going to talk a bit about what algebra is because this is a bit subtle, and I'm going to lean on this later we're setting up an analogy. And so the first point is that some so if we think of addition, for example, there's a sense in which seven plus three, say and three plus seven are the same thing. So they're both 10 was sort of denote the number 10 and they're equal in that way but in another sense seven plus three and three plus seven are completely different things and we just have to look at them to figure out why we sort of got these terms and the terms are different but they might be the same. For some notion of the same. This is sort of what algebra is about or at least that's the perspective that we're going to take here. So in particular we're going to have sort of our, our ledgers as things like three plus seven and seven plus three, and we'll be able to talk about when they're sort of the same like when this sequence of events on the ledger would be the same thing. So we have to talk a bit of math first but you don't. Mostly we have to look at these pictures because these pictures are good enough, they're sort of a fully formal or rigorous syntax for the things we'll be talking about. And I think they're a lot more intuitive so we're using them pretty heavily.  And then we have to draw them from F from a to be in our category. And if I have morphisms where the thing that one is to matches the thing that another one is from that I can compose them to get another morphism is why I could sort of stick F and G together to box wire things like this in the middle. And then they satisfy some rules so this composition has to be associative and it has to have a unit, whoops, which is just sort of the wire with no box and so on so a category is a bunch of things that you can compose. We're going to talk about categories with a bit more structure in particular monoidal categories, which are ones with a monoid structure on the whole thing. So this means that if I have a morphism F and morphism G I can put them sort of beside each other to get one that does both of those things in parallel if you like. And then a monoidal category is symmetric if I can cross these wires in a way that makes sense. So that's very quickly what a monoidal category is. And the way that we're going to read these diagrams and in our monoidal categories is as the whole category really is as a theory of resource convertibility, where our objects the things that our morphisms are from and to are like collections of resources. And our morphisms are going to be ways to transform the collection of resources that they are from into the collection of resources that they are to. And so, for example, we might take our sort of basic objects to be just like bread dough water flower and oven to set up a sort of naive theory of resource convertibility for bread. And then we might have our morphisms generated by one that allows you to mix water and flour and transform it into dough here on the left. And one that allows you to bake dough in an oven to get back some bread and the oven, you're in the middle, and one that allows you to need dough to get more dough on the right. And so that's sort of the thing and once we have this we can. So I'm hiding a lot of algebra here we're going to generate the free monoidal category on this data. And in that we'll be able to construct things like this this will also be a morphism there. So basically we can read it kind of intuitively as describing the transformation of well two bits of dough and another into two bits of bread and an oven by baking one piece of dough in the oven and then baking the other piece of dough in the same one after another. And so that's how we'll be thinking of these morphisms in our monoidal category is as transformations of resources. The syntax part the three plus seven is not the same as seven plus three part. And the algebra part will be that to the axioms of a monoidal category, more generally, our resource theories will make two of these string diagrams equal. In case the sequence of events they describe is sort of the same sequence of events in the sense that it would have the same effect on the resources involved. So for example in these two large diagrams on the left, we see we're mixing and needing water and flour sort of at the same time in parallel twice and then we're baking the resulting dough in sequence one after the other. Whereas on the right, we mix and need and then bake and then after we bake the first look we mix and make the second look so these are strictly speaking different procedures, but they would have they would do it the same thing. That's according to our theory, they might not do the same thing if you actually baked bread but that's okay it's a model. And so in this way we can read these string diagrams I'm going to the word I'd like to use is material histories these string diagrams tell us about the material history of some stuff it's a little slice of that. And from material history is exactly what a ledger is, in particular, a ledger like say the Bitcoin ledger is the material history of all the coins on it. That's the point of a ledger. And a transaction as well a transaction is just a string diagram and we can sort of append a transaction to our ledger expressed in this way by sticking it on the bottom. So I have three big ones and I wanted to do something with the bits of bread, maybe the oven on the bottom while I could stick more string diagram on the bottom by composing another sort of string diagram of mine. That's how we would do that by composition. And then like I was saying with that picture before equal transactions are sort of these material histories that would have the same effect, according to our, our occasional theory, and similarly a ledger is just like a big transaction so two ledgers would sort of lead to the different states of the system if they were equal in the theory. So that's, that's main idea one probably the most important idea is that the algebra of these ledger systems that we see in distributed ledgers distributed ledger technology is to be found in what we call symmetric and thing to sort of. Okay, so one thing that we care about in distributed technology, especially is ownership. You know, most of the systems that I'm all of the systems that I'm aware of care mostly about modeling who owns what or who is responsible for what. So the second part of what I'm going to talk about is how you can model ownership in a nice way in these symmetric monoidal categories, which I'm just going to call resource theories. And in particular we'll get a nice graphical calculus that will be a lot like the one we saw before with diagrams like this, but with additional information allowing us to denote ownership and change of ownership in a sort of rigorous way, which we'll talk more about at the end for now we're just going to use the diagrams. And so to set this up we're just going to assume a set of things that are capable of owning things. So these could be public private key pairs that's very cryptography thing but I'm just going to call them Alice Bob Carol and so on. And we're going to associate each of these with a color because our, we're going to use the color in our diagrams. So Alice is blue blue and Bob is yellow and Carol is red for me. So I want to go to the next slide there. And so the idea is that we're going to start with one of these resource theories one of these symmetric monoidal categories and we're going to construct a new one by coloring everything in, or sort of by overlaying color on everything and our diagrams, where a color corresponds to ownership by one person and something is shaded blue that's going to mean that it's owned by Alice. And formally this corresponds to us having objects like x Alice and why Bob that sort of x owned by Alice and why by Bob, and also we'll have combinations of these so we could have one object that is excellent by Alice and why by Bob and so on. And morphisms are just, you know, again, there are there are colored in morphisms from our original resource theory. The idea being that anything you can do to resources you can do to resources that you want. That's kind of the, that's whatever she means. And so a blue morphism will be Alice transforming some of her resources, as opposed to just a resource transformation happen. What we've already introduced in doing this is a difference between sort of Alice having an X and Alice having a Y and Alice having an X and a Y, it's sort of a silly formal difference but we have to mediate it. And this, this is something that crops up in for example the Bitcoin system where there's a difference between having two $1 coins and one $2 coin because you can't pay Bob $1 with two, sorry with one $2. And so we'll introduce morphisms resource transformations that allow each person each sort of different color to split apart and combine the collections of things that they own. So just just the intuitions given by these diagrams here below. And they have to satisfy a number of axioms and these axioms are just things that you would hope are true but will corresponds to a sort of canonical mathematical structure. The two on the top here are asking that, well, we'll just read what they say is that if Alice does F and G to two collections of her things separately and then combines them that's the same thing as if she had combined them first and done F and G to the different parts after. I'm realizing that the one right is the same diagram that should be the diagram on the left upside down and that would say something about splitting and doing things. Combining and splitting must be associative and so on. This is just what you would expect to be true about collections. And then we need some axioms to deal with empty collections to be sort of fully formal it turns out to work out much better if you have empty collections as well. So this is just at any time from from nothing Alice can manufacture a collection of nothing and vice versa can get rid of. Are we doing for time pretty good. We've got these colored string diagrams denoting sort of material histories of things owned by people, of course ownership changes over time so we're going to introduce some new resource theory more morphisms that allowed the owner of a thing to change so for any two colors of people and a sort of thing here X, we introduce a morphism that just, you know, in the way we read this is that x belongs to Alice and here the color changes so now x belongs to Bob, it's like a change of ownership thing. And this has to satisfy axioms as well just to make sure that it's coherent with respect to all of these collection management axioms we introduced earlier. So if I have some X, if I have some things X and some things why I combined them into one collection, I have some x and y and then I give that to Bob, this ought to have the same effect on all those things like that. It's the same thing as if I had given them up to Bob first and Bob combined them, and so on and so forth for empty collections and splitting collections apart. Whoops. And then the last axioms will ask for is that that sort of here stating that the ownership of a thing is somehow immaterial and that if I perform a transformation of some things and then give them away that's the same as I had given them away and somebody else had done exactly the same things after. And finally we ask that. I know this is a lot of axioms. We'll talk about what they mean mathematically. So we have axioms where if Alice gives something to Bob and Bob gives it to Carol that says the same effect as Alice just giving it to Carol that's here on the left, and then if Alice gives something to herself that's the same effect is nobody doing anything. That's your bottom right. So when we have all these axioms and all these equations we can represent or we can sort of represent and derive things are in our equation theory, like this, and this is sort of the point of all of this, where we might read this diagram on the left, as the transforms well you know an oven belonging to Alice, two bits of dough belonging to Carol, and an oven belonging to Bob into bread belonging to Carol and ovens belonging to Alice and Bob. The one on the left describes the sequence of events in which Carol gives one piece of dough to each of Alice and Bob, then Alice and Bob bake that dough in their ovens before giving it back to Carol. Whereas the sequence on the right describes another way of doing this where Alice and Bob each give their ovens to Carol Carol bakes the dough into bread and then gives the ovens back. So there are different things, but they should really have the same effect. And indeed in the equation theory that you can find the details in the paper. But in the equation theory we've sort of been building with these colored diagrams we can derive that the corresponding morphisms are equal so like really formally these have the same effect, which I think is kind of neat. So that's, that's sort of the, the main thing I suppose I guess I have five minutes left and we'll spend the rest of the talk talking a bit about more of the math so we did this thing with all of these colors and all of these color components to be a bit normal we could also do this by asking for a resource theory, we'll call it x, just a symmetric model category, a set C of colors and we'll define the colors of X C of X to be well the free product category on X cross C. And with some additional morphisms given here in sequence calculus form subject to 18 equations just corresponding to all of these things I've been talking over for a while now, you know these can all be written down fully formally in the notation you see here as opposed to the diagrammatic one but I'm not going to put that on the slides. So, right. And once we do this, our equations and our additional morphisms and our whole setup is largely characterized as health that so for any symmetric model category X, this is our original resource theory, and any set C of colors, you know, we've sort of our axioms insist exactly that there's a thing that we call a strong symmetric monoidal functor from X, our original resource theory to see effects. And the idea is that anything that is blue, anything belonging to Alice is sort of in the image of this factor, if that means anything to you. And then that's sort of our collection and splitting apart axioms. And then almost all of the rest of our axioms tell us that for any two of these functors these sort of owned by Alice functions and owned by Bob factors. That was what we would call a monoidal and co monoidal natural transformation between these two functors. And that's that sort of this thing where what we call the components of the natural transformation are given by this change of color and our axioms ensure exactly that it has these properties. So if this means something to you that's great, if not the point I'd like to make here is that these are very canonical things it's kind of remarkable that we can use this canonical structure to model this our ship thing in a nice way. And then something, one final math fact is that the, so there are two axioms that we don't need to get that previous proposition, the bottom two on this slide. And these axioms turn out to give us exactly that there's an equivalence of categories between our original resource theory X and our new colored one with ownership, see effects.  And the sort of the point of this is that we know we haven't made a mistake in adding ownership, we know that we haven't accidentally destroyed or created any of the categorical structure. It's sort of the same thing structurally but with colors, which is nice to sort of good. That's all I wanted to say. If you found this interesting I'd encourage you to have a look at the paper. It takes a while to get used to sometimes, but beyond that I guess we can move to questions. Thanks for your time.


Hey everyone, Drew with GeoArm and I am back to talk to you about setting up smart arming on your GeoAlarm control panel. It's very easy to do, but first let me give you a little background as to what smart arming is. Smart arming is the opposite of unvacated premises.  Well your system can, since that is maybe you accidentally set it up as stay mode and stay mode which means no motions are enabled or anything like that. What it can do is automatically convert itself to arm away to where all of your motions become active. This is a useful feature if you forget sometimes, but otherwise you can just leave it off. But that is a feature that's here and I'm going to show you how to go ahead and get that set up. Hit that hamburger icon, one, two, three, four is my code that's configured at the time of your set up so yours is probably different. You're going to go ahead and hit security now. You can see the actual smart arming feature here and it will show you. If you enable it, what it will do is it will wait one hour and if there hasn't been, I got a program in a motion sensor first, but you would enable it here and when you enable it you would set the time. So the time would be one hour. So if for one hour it noticed that there's no one moving inside of the house or there's no motion, it will automatically after that one hour trigger it into arm away mode or smart arming. But otherwise if you don't want that feature enabled you can just leave it off. Ladies and gentlemen that is how you enable and disable smart arming as well as a brief explanation as to what it is for the Geoalarm Control Panel. Please leave any questions or comments below. Take care and have a great day. 7-6, Geoarm, the worldwide leader in DIY security.


Hi, so I'm not an expert on PyTorch, just to clarify, PyTorch is one year old, so I'm maximally three months, seven months, somewhere between vacation days, so using it. And yeah, today we're going to go through this model, an actual consequence sequence, it's fun. And let's see what we're doing. So who saw this problem from the post from Facebook, just like each of you who saw it, who read the papers that I posted? That's very little fun. Let's try to get through as much as possible. So there's this problem where you go to the coffee shop in Singapore and you're not Singaporean, you try to order coffee, you try to order kopi and you try to imitate the Singaporean, what you'll get is, oh it's not, it's not on this picture. So kopi is actually coffee with milk, it is not black coffee. If you want black coffee, you need kopi-O, which then it also comes with sugar. If you want black coffee without anything, you have to say kopi-O koso. And the list goes on and on. This is just six out of a hundred other paper forms. So for those who have not read the papers, this is just a crash course. Most likely it will take around five minutes. So who knows what's this? Perceptron. Perceptron, who said that? Is that an Autobot or a Decepticon? Sorry? No, it's not a transformer. Although a transformer is also another thing of architecture. But all it does is it takes the input axis, tries to summarize them with some function, most likely a summation, because it's simple, multiplies it with the width that it comes with and then goes through some transformation so that it fits into this curve is actually a sigma curve, which means it just wants the output to be detain 0 and 1, all minus 1 and 1. So who knows? This one. Anyway, this is called Elman-Nap. This is, in short, this kind of architecture or this class of architecture is called recurrent neural net, where first you will take the x, which is your input, and then you go to some healing state which then is nothing. It goes past it to the next one, takes the x, and then it passes it to the next one, and you try to predict the y as it goes on. So there's no input output here, the output is actually some sort of your input as you train, and as you go on, you know that it can go on forever, but because of GPU, there will be a cutting-off point, and then this is how you predict. So this is one way to structure recurrent neural net, which is a very, very old one called Elman-Nap. But this is also something that we'll see here now. So anyone understand why is it previous healing? That's very important. And the difference between this and the previous one? No? Okay, let's go back next, okay? Previously, there was nothing, right? You only have one state. Everything is an x, a feature, it fits in, it goes in. But somehow now you say this x is related to this x, you're trying to learn the ways between here, between here, before you get to the y. So that's why it's recurrent, in a sense. It just kind of passes on. And what's really important is this healing state is what it's trying to learn, and it passes on to the next one. But you realize that somewhere here, it's very hard to see what's there. So that's the weakness of recurrent neural net. Something that's very helpful, it comes from trust, block from down there. So previously we see this, right? But if the input, it goes to the next one, it gets the previous and then, this is how it actually looks like. As an input, you take the previous hidden, it takes some sort of and learn and hidden information, somewhat in blue. And then it pass the blue and red, it comes out with a mixed hidden information, blue, red. And then it goes on and add more until you get the final output, which then, this is what you're learning. You want to know how are the different inputs related so that you can predict the output later. In short, this is recurrent neural net and you're free to go. Oh, no. This didn't work. It worked in class, where the students just kind of left. So, okay. For sequence to sequence, it's basically recurrent neural net in a different form, but it does, it is meant to input an output sequence. Previously we had recurrent neural nets going like this, where you don't know what's input to input. The output is mixed in. So, after evolution in 2010, 2014, 2011, they came up and said that, why not we input something through as an input and then generate some sort of a vector in between and generate output? It kind of worked well for everything that has a sequence, machine translation, text generation, Python interpretation, this is actually a very nice paper if anyone read before. And that's what it does. So, you have the inputs previously you go through and now you also go through, but it goes to this encoder and it compresses into a single vector. At the end here. So, after you see this end symbol, it becomes a single vector. And from the vector, this becomes a stud state of a decoder that tries to predict an output. Makes sense? Anyone lost at this point? No. Maybe this is too simple. Yes, before anything, data lunch, because no data, you get nothing. Although, although we can't value the value of data, I think the value of data is priceless. Yeah, scientists spend around 80% of the time just comparing data and managing data analysis. That's what I do. And it's very simple to manage data. First, you take data from somewhere. You then throw in some pandas, S-frame, dust, whatever, get the data into some shape, then repeat until it's done. It's a nice recipe. So, I have uploaded a very small data that I just copied from a few websites. As you can see.  And that's what we want to do. So, the left one is called a window. It's our input and this is our output. And that's what we are going to do. So, it's cool. Am I going too fast? Am I going too slow? I'm going to cheat because I have a full notebook here. Just in case. I will try to speak a bit. Actually, you don't have the notebook to cope with me, but then what will happen is I put a lot of question marks that I will try to fill in and most likely it will help because he knows Python here, right? Who doesn't know Python? You should try to apply to the program after you learn Python. This is just some I hide this stuff. And what we are using is actually a torch library. You can install and pip install. You go to PyTorch. You will see the instructions kind of clearing in your face. So, this line. Do you guys see the code clearing? Is this better? So, anybody know how to pass and pass? So, this line is very, very irritating. Because of this line you have to go through every variable and say .cuda just because it's kind of difficult. That's why you say something. There is a way to not do this. Yeah, you don't have to do this. There is a way to not do this. Yes, yes. You can set, based on this used cuda you can set a variable equal to a type either being a cuda float or float which type of that type. But if I want to change it to a tensor I must create multiple types. No, yes, but every one of them we got type with the typing. That's where... That's where tensorflow is easier where you just set a flag and everything runs. But... So, remember this because this will pop up everywhere when you debug. Very irritating. These are just some conventions that most torch users use. I don't know why they call it F because they like to write algorithms and all that. That's what we're supposed to do. So this is just an introduction that we went through. These are two tables that we should try to read. This laptop has no GPU. Yes, but it was still run because the number of rows is... Remember the number of rows in the data. Yes, that is affecting. I just cut off the last one. But it's small enough to fit in. And the vocabulary is small. So this is the task. We have an input we want to back to output and hopefully it does this spoiler alert. It cannot do the last one. So let's go down. Oh, it's actually here. Is it okay if I sit down and type? Using Jensim, this is actually a very nice library that you don't need to manipulate your own dictionaries and you don't need to do like back-of-words or text. We'll see it later. The pen and data frames look as such as just not very explained. And this is what you do. Remember in the recurrent neural net, there's always a start and an end state. From what we see, we need to start and then at the end of the encoder, the start of the encoder must begin. So this is what we're going to do. We're going to assign this symbol as a start and this symbol as the end with the index 0 and 1. And the first thing to do is to manipulate the pandas data frames to what we want. So let's do something like this. Please correct me if I'm wrong because I'm typing out of brain. I think I can do this. Food has capitals. The simplest way to manipulate your data is just to lowercase them. Although there's a lot more, you can refer to my pie. Is it Python mid-up or Python mid-up? Python mid-up. The first way is to lowercase them and then this word tokenize is a function for MLTK or you can use any tokenizer. In this case, you can actually dot spaces that also work. So the idiom is actually kind of simple. You have a data frame in the column. Because string.lower is a function, you can do this. And then we apply it. Like this. So there's a lot of code floating around doing like this. Please don't do this. Apply word tokenize. Sometimes you want to remove the stop words and you have stop word list. You go like that. Stop down the list. And then you dot apply lambda x x. Just for your information, this is the right way to tokenize and then remove the stop words. Some people have been like doing it in those steps and doing apply again and again. Apply is actually a very, very expensive operation. So try not to use it. Sorry. It didn't work so well. So yeah. This is the right way to tokenize and remove stop words together at the same time so that you don't use to apply. So yeah. That's all. That's what we get. We get a list of tokens which is the list of words that we want to include in the end symbol. So the problem is this is very nice. Actually you can fit this to a referral that you'll accept it and say that this is very nice but then torch will go kaboom because they like numbers and they don't like correctness. So the first thing to do is to convert them into numbers like this. And that's where you use Genshin dictionary. This is a hack actually. So if we look at this and I just add. If I just do a dictionary of the English sentence values anybody know how to print a thing. Thought. So with the dictionary you can actually fetch the words. So let's say zero and then you have one. You see there's something wrong. Anybody spot what's wrong? Yes. Because what I wanted was to get zero index here and one index here. And if you just let the dictionary side it will just go in some sort of hash because of how hash is just written. So one way to force it is to just say that I have a document of one word S and a document of one word with the end symbol and the unknown words. Unknown words don't occur in this exercise at all but then I'm just putting it in for fun. And that's how you hack it such that you do this. That's right. And then afterwards you add in all the documents that you want. Which is the English sentence. That still happens. And voila, we get the order. So that's one way to hack. Is this too boring or I think this is actually quite important because so we get English sentence like this and then the very very fun part is just now we created the vocabulary. We just feed in any English sentence and do a dog dog to IDX to get the IDs that are in the dictionary. So if you have this English sentence right, maybe do English vocab dog, dog to IDX Oh yes, who knows what is wrong here.  Not true, who knows what's wrong with the reaction. Yes. I commented this like sorry. But yeah, that's what you get. So that's one thing here. This is actually a very new feature. So if you can't do this and you see this attribute error just do pip install-u-genc you'll get it. And we requested for this. Like this is very helpful. So now you can do vectorize. Like everybody is more effective. But this vector is a list of integers which is not very helpful. Because PyTorch wants everything to be or some sort of PyTorch data structure. So you need to convert it into this variable question mark. Who knows how to solve this. I actually don't because I'm pretty sure it's wrong. Like there's some long tensor because it wants a tensor and then there's the last one. And I get it right. I'm gonna cheat. Anyone knows what's wrong? You know, this is the part that PyTorch should just remove the Python part. It's kind of unintuitive why is it not there.  So just remember this by heart or cheat by me. But never mind. Is this still very boring or interesting? Just check. And what we really want is to remember our inputs are pairs of sentences. They want to fit into the model. So that's what we want. And if I just add a cent pair here I should get something. Where there's an input and an output it always starts with a zero always ends with one. Because that's our style and simple. And in between it's some coffee something I can remember because I know what street is. It's always coffee. And yes, the sequence to sequence model. The general idea is to take two neural nets and transform one to the other. And that's where encoded and encoded comes in. The simplest kind of neural net you can quote in actually this is not the simplest but a very simple encoder that you can quote is just to take the input put it through an embedding layer then it becomes embedded which is the output of the embedding to the GRU which is an RNN. Because the GRU just imagine is an RNN you see there with some superpowers that can remember but remember this previous hidden this is what makes an RNN an RNN so this is very important and what the RNN always produces this output and the hidden. So question, where does the output go? Where does the output go to the encoder? Partly correct. For this case it actually goes nowhere because it's very simple. Then where does the hidden go? I sense the aura but yes the hidden. The output The output will be the encoding but it does nothing in the encoder yet because it just get passed on to the next one. But then it is the hidden that really moves so if we look at the slides remember it's the blue thing that moves and blue things are hidden for now. So where does the hidden go? Anyone from the back? Nothing. We know that this is the previous hidden so this hidden is going to get looked back here to the next stage and this is why it's recurrent because it's looping all the time. It's loopy. Just remember this graph or I should not try this one. So we have the input right so this is how you create a class of architecture or network itself in PyTorch you always import from the module because all kind of network encoders are all modules. There are ways to do this separately but remember always use modules. They will always use this super self in it because there are other things that you can initialize but I'll skip it. What's really important is the input size and the hidden size. That's why you need to decide how many nodes you need to put here. And 20 doesn't matter for now because we're only building the architecture. As we move on you see that we're just taking in the hidden size and we say this network contains hidden size of this and then we have to create an embedding. Anyone know how to do this? It's really simple because this part is more PyTorch. So neural.embedding embeddings take two things right it takes the input and the hidden size. And then if you go on the GRU does the same thing it takes the GRU and then input. I think I'm wrong here. Yes that's where I'm wrong here. No, no it has to take the hidden size because that will determine so the input size is the number of words in your sentence that's going to be the input size. The hidden size is the number of nodes in the embedding. You have to know how many nodes are there. So it has to take the hidden size. In this case I'm making the hidden size of the embedding and the hidden size of the GRU standard but you can separate it out and say hidden size 1, hidden size 2. So I'm just being very lazy and I say hidden size all size of hidden all embeddings and hidden layers are all having the same size for now. So does it make sense? So embedding always have a size but it cannot be size 0. So it must be size more than 1. 1 normal. Similarly remember when we move forward our output in the hidden is just trying to move the GRU again. So I have self.gru which is here and it takes the same thing the input size and the hidden size. No question here. So typically for LAP pipelines you take the input and you change it to some embedding like word-to-word or fast embedding. So what's the difference between learning to wear fast bags like this and embedding like that? Yes, there's no embeddings in word-to-back that will give you a complete over. That's a bad job. But the thing is there's a few ways to get embedding right. One is you can don't train this at all and make it fixed. But what I'm saying is I'm using the embeddings as part of training to train a new set of embeddings for the given inputs that I have. So if you want to use fast tags or word-to-back what happens is you remove this and says all of these weights are fixed and you don't train them then you wouldn't need the variable you wouldn't need the embedding layer you would just need to use I think pre-trained layer or pre-trained module.pre-trained something but that's where you can replace this with something else. And then remember that we always start with some sort of hidden state for now let's just say we create a one-one hidden size who knows what the size is self-proclaimed shouldn't be taking the embedding Yes sorry you are right. Anyone knows why but let's go back again so it's very simple once you have drawn this graph we just ask a professor to draw the graph for you and then as an engineer what you do is you make this graph into a code so you put the input so this is initialization but then when you move forward you need the output of the embedder which is here and the previous header which is here sorry I mixed that up with the initialization so when you create the embedder it is going through the embedding and then it just shapes it into a single it took me 10 minutes and killed my brain nobody knows what spinous one sorry it's a flexible size you are not stressed by embedding that yeah why don't they use now a question mark never mind it's just very strange so it will never blow up because it doesn't do anything it's an object so the code is a little more complicated because somehow the default tutorial at the redo but then I'm going to skip it I'm just going to tell you it's a nonlinear unit that we see in this slide here this F here it's just a transformation you can put any transformation you like you can alternate your transformation every other loop if you like it but that's just for you so most people just keep it fixed so similarly we have the same thing remember the decoder is just some sort of extension of encoder but then in this case we have to predict an output just now the output was pretty much useless so the input goes into the embedding it goes through the transformation it gets to the GRU takes the previous hidden which comes from here produces the output which then these are just they are just numbers right what you need to do is a softmax a softmax kind of choose what is the best fitter for the case so in the case of machine translation you have the input vocabulary and output vocabulary the output vocabulary is what you want to get the output here so this the softmax converts the numbers in the embeddings and then chooses the best fit and fit it to the vocabulary does it make sense most likely it doesn't make sense now unless you see the numbers all you have done this before so you'll get through later that's the really cool thing about PyTorch you can just kind of inject half way and say print something you can't do that in TensorFlow unless you use eager mode that will never be deployed at some point I don't know but yes same thing right it's very simple you get this graph from a professor you just code and it's like talk from embedding this is kind of strange because in this case in the input we have the input size that's our input and the hidden size but for the decoder the input is actually the hidden size because it comes from the encoder and then the output is going to be your output size so you kind of swap it around so you have your hidden size here which is your input and then your output size and then the GRU does the same thing it takes from oh there's a relu later it's not moving these are just energization so you have hero, that GRU at any point of time if you think me typing the code is too boring sound out, I can just skip and copy and paste but yes so this is the part that I always forget because it uses gru basically the software gives you a certain number the linear will output the output that you want but I forgot what's the input most likely is the size of the output so these are all energization so it must be all sizes it's nothing to do with the movement of input out it will blow up later if it's wrong then I'll copy and paste so the graph again we have embedding going to relu and then from GRU so this is the thing we have embeddings with the strange minus 1, 1, minus 1 now who knows why this happens actually there's no good reason why you should do this this is just an interface of high touch you can look at this discussion with the link here but just remember this because this could have been a single factor 1 times 1 times minus 1 equal to just minus 1 which is any size that you want yeah actually there's a reason why this sequence lag and all that I think it could be a lot simpler in terms of its interface but just remember this I can't read that much the relu is very simple it's just an F every time you have nonlinear functions always call from the F and relu we can change this to a sigmo stick with relu so after the relu it takes into the GRU so just type GRU self.gru takes the relu and the previous sequence output and then it goes through the softmax this part who knows how to type this you are initializing the hidden steps the hidden state from the start how do you type this anyone it's a one line answer don't type this copy and paste because this is exactly how you want to do from the encoder encoder and decoder is just an imaginary boundary that you draw but to the RNN to the network itself it doesn't really matter so it's exactly how you want to initialize this you could initialize it differently for the encoder and decoder but the simplest way is to initialize it the same way so that you can debug it easier so this is the cool part when you train a model you don't need to do wait till the end until you print something so I'm going to go through this remember we have the hidden side that you have to terminate for every kind of layer for now I'm fitting the hidden sides for the hidden sides to all the same there's a learning rate of the optimizers anyone don't understand what's optimizer and learning rate just double check good and then the best size is just how many times you want to see the same data again and again sorry the epochs is how many times you want to see the data again and again the best size is how many data points you want to put for time you write the rate it's what you're optimizing on for now you're just going with the negatives not lost and then the maximum length is a heck actually when you decode you can keep on going forever right if you look at the slides again I'm always going back to these slides because they are good it goes dot dot dot dot dot how do you know when to stop anyone anyone yes yes the slash s is very important actually the start symbol has no meaning and you can drop it don't tell your boss because your boss will ask you why I don't understand the output and the input but it works but anyway just put it there the slash s itself the end of symbol is actually more important than the start so that's why you need to set the max length so that you have sanity and then you can actually look at the slash s after some experiments you realize that actually this can go down further because at some point you'll just keep on producing and have to say this is the end this is the end and so on and so forth so what we have is the same thing input vocabulary, output vocabulary I'm just doing this so that it's easier to see remember encoder takes the length of the vocabulary which is the input size and the hidden size the decoder takes the reverse where the hidden size is the input is actually the size of the encoder and then output is the length of the output vocabulary That's where this part comes in, where you need to do a lot of cool-down again. Although it's not necessary if you've found the correct thing to do. Optimizers, you can change this. Optimizers always come from the OPTIM module. There are several. SGV is the most common. And then what you do is you take the network. Every network has a parameter. But then we didn't declare this. So where did the parameters come from? Answers? Yo. Okay, the parameters actually come from this line. Super God in it. It kind of created many, many things that you don't see. But then it's natural to create the parameters because that's what we are training. Remember, the parameters are the colorful words that we see just now, right? From the hidden states. And then the learning rate as per the SGV. And then we create sample data. So in this case, I'm just randomly creating, picking up batches of sentences from the back of sentence pair I have. And across each batch, which is E4. And I have a type error. Yes, it's wrong. I knew it. It's the dot linear. So, okay. When you see this, learning experience. When you see this, every time you see this dot mm error, most likely the linear can only go into the decoder level. So if we go back to the folder, this is not normal size. What actually is it? Yeah, here. I don't know. I'm going to check too. The linearity actually takes the hidden size also. Output size. I'm not sure why it takes the hidden size. I'm going to just leave it there first. It shouldn't blow up this time. But, logically it should work too. It works. So let's just end the line here. For sanity, understand what's the training data. So the first one is a batch. This is the first batch. And this is the second sentence. All the first sentence. We ran from a pair of sentences. Which is our something-something goby. And this is strange. There's 25 here. Never mind. I don't know why it takes this. So when we do the training, it's just trying to extend where it is here. So first we set the hyperparameters. Then we start the training. We iterate through the batches. For every batch, we initialize optimizers because we go back to zero. And then the log loss will be zero also because that's what we're calculating. And then we add an up later. And then here we go through each batch which is the start sentence and the end sentence. In this case, they're called variables because Python like the word variables. The encoder hidden is input variable. What should we do at the start of the encoder? Questions. Partly correct. So what should we do now? Remember our... This line here where this initializes hidden states that we don't remember. So that's partly correct. You can actually initialize it randomly. But for now, we're just initializing zero. So we should do the batches encoder.initialize through states. And it doesn't blow up. And then we iterate through each state, right? Because this is just initialization and there's nothing going on. It's just initializing all the input states and output states. So the question is this. Why did I already initialize in the class? But now I have to initialize it again. Actually, in the class I initialize nothing. There's no real numbers going in. So this is the real part where after the class you try to initialize and put the numbers in. So this is the really cool part about... I thought you can just do this and see. We look through and then we see that this is the last variable. This is the last variable without thought size. And then we see that the size is four. And then we put the variables in and then now we initialize. So in this case encoder outputs and all other things in the class itself has empty values. It's only when you're here when you start training and putting the variables. So at end we have this again. Remember we go through the... Each state, we start with zero. After the first state we have the first word itself and then we try to put it into the encoder. So how do we type this line? I'm just going to type it. Remember our input is the input variable. Remember our graph? The input variable is just the first word that we created here that we read from the data. And then the encoder hidden is the one that we have initialized here. And then it runs and then we iterate through. I'll be sharing the full one. Anyway, so the cool part is here. Whenever we move along the states we don't really know what's inside this encoder output and this encoder hidden, what are they really? So if we just look through and in the end we have already reached the last sentence and then from the last sentence we can see this. So in the last sentence we see that the encoder has... This is just printing what's in the encoder. The encoder is embedding of the input size and the hidden state 10. And the GRU just has the hidden and the previous hidden, right? So if I continue typing the data back which is here and the final sentence and the first word and the first part of the copy part which is the finished part, we see this. But if we move on, we see that the last sentence in terms of this is what we're interested in. We don't really know what's the encoder output, right? From here, we just see that the graph puts in the encoder output to the hidden and then just keep on looking by itself. But if you look here, this is just four words, five words and we look at the encoder output there are actually five rows here, right? See that? So that represents a state as we move on and you see all these zeros here we see the size is 20, 10 because we put a maximum of 20 and the 10 itself is the size of the embedding and the size of the hidden state that we put across. So that's the nice part of PyTorch where as you go on, you can debug by looking at this. You can also check what's in the hidden state which is very interesting. The hidden state itself is this. Let's go back here. When you have 0.6154 and the hidden state, remember the hidden state is going to be passed to the next state that means the hidden state is actually the previous state, right? Kind of mind-wrapping but you see that the hidden state is the previous state that was existing here. It's exactly the same. So RNNs, we have settled them already but then these are just the encoders. If we move on to the decoder part, it's the same thing that is going to be touched upon first So after we print out, after we encoded we want to move on to the decoder itself. It's the same thing here. You need to initialise a variable where we use the strange part touched on the variable and initialise the start so it must be encoded output. This is actually wrong. So sorry, this part of the code actually comes from the tutorial of PyTorch. You initialise it with the start state because you always create a start but actually this is not needed because you could easily initialise it with the output of the hidden state here but let's just follow and play on with it because remember since we added the start symbol to every sentence we should start the start. I think this was skip one and just say go on and then you get input and remember this is where the decoder state is wait, why is this good? This should be done. Just go through this, that will be easier. So the start of the decoder hidden state is actually the end of the decoder hidden state stuff and when we decode we get input and input and input and then we see all these strange syntax coming on like why are you doing this when you have the data and then pick the top K the top K is actually the best which is the soft place at the end but if you look at it this is what it's doing if we finish the loop get to the PyTorch and there's the output if we look at the top K we see the output the last word is actually this and it's mapped to 106 columns is the vocabulary size of the output so it's trying to map what is in the output here it's trying to map the decoder output to the words itself and what you see here is a lot of probability that it's belonging to that index so this top top K1 actually picked the best and that's why you see the best decoder gap of the world is actually wrong it's actually the yes because this is I didn't train anything remember we were training on how many batches we were only training on batch size of 2 with 30 columns so that's like 60 times so that's the coolest part of PyTorch anyway let's finish this so after you pick the best output you update the last PyTorch and then when you see this symbol you break although you can continue by the sensor so just break you propagate the loss backwards see this loss here is lost within the batch and then before the batch your loss is 0 you back propagate the loss and then the encoder and the decoder optimizer needs to step to the next step so that's all that's the sequence to sequence but let's put everything together put it in the function get some nice library basically I'm just printing out like how many seconds and how many minutes things go through and show a very nice graph and this is exactly the same thing I've seen just now but with more handy like you see printing out loss printing out loss so we set the hyperparameters again we reset and we start training remember the input size is to distinguish the vocabulary length of the input and then output is actually the vocabulary length of the output this is exactly the same thing as we see just now but I just put it in the function so that you can see clearly and it goes up and yes this is the most satisfying part of your job if you do AR because this is coffee but let me explain what this is going through so remember this number here is the criteria that we are training on this is the criteria we are training on which is the log loss I don't know why it became positive but anyway so this is the loss, the lower it goes the higher the accuracy of the system this percentage is because I had a fixed kind of iteration and I said that you go through so many epochs so many data points per batch that's why you can calculate this easily normally when you train you want to train this until your GPU breaks and it goes on and on and keep on talking and get some coffee and we'll finish soon I guess at some point you see it's getting lower which is good wait the graph is missing there should be a graph somewhere I think I may stop talking I'm not going to retrain the next thing that you need to really do after you train is to put in your code to save this is actually very cool this is called fString in Python you can actually just put the variable inside the curly brackets and then you can just save and it will save so getting the model to translate now this is the part that you say I'm just a I don't know crook and I'm telling you something it doesn't work it just shows it's not working so let's go through this because this is the last step the decoding part I should call it translate it takes in the model that you train with the input that you want to translate and then the next line is just for sanity again I'm just going to use the same one I put the input variable and the size the same thing that we did initialize the state like how we did previously for each sentence we initialize it to zero and then we move on again then the variable itself is exactly how we initialize our encoder itself and then here is the part where you look through each word in the you look through each word in the input variable in Python this could have in this yeah this is bad Python we should not look through this way because it likes that but yes this is just looking through each word and then as usual we have our encoder that takes in two input one is the input variable and the encoder here then we look again we store the encoded output and then what is this doing encoder output EI it says this and it overrides yeah this is the same thing where it just says take the encoder output repeat again and then put it back in encoder header which actually this whole thing is not used at all which we had raised my point in the graph the encoder has no use if I do this it should not affect anything so the encoder input same thing we start with the start index we put in the input and go to the encoder remember the hidden state is the first hidden state of the encoder is the end state of the encoder and then we want to keep the list of the words now the same thing that we did when we stepped through training the encoder takes the input of the encoder and then we do this top and that's where you get the numbers out here but this is the special part when you get to the input what we want to do is to not get the output of the hidden state just now but you force the input of the next word to become the softmax of this this will give you a clearer kind of way so every hidden state is kind of fuzzy you don't know where it is but since you are sure what you can do with the input of the next state with what you are confirming as the correct output of any word so in this case this is some sort of teacher forcing but it's not really because it's not teaching we are just forcing the output to give you the fastest output as fast as possible because in this case we know that the beam gets smaller and smaller as we go through as everything in the encoder output becomes