Okay, my name is Dan Walsh, I'm going to be doing a talk called containers and production. This is a little background, I run the container team, I'm the lead engineer on the container team at Red Hat, I've been working on Docker and containers for the last couple of years and I put out a paper, it's an internal paper, but eventually we'll be putting it into the blog talking about running containers in production, basically thinking about containers going forward. One of the things, containers talk about DevOps all the time, so developer and operators and how this new DevOps model where we're constantly putting out packages and the whole release cycles and things like that, but to me at this point Docker is concentrated almost exclusively on the developer side. Especially when you look at, I'm ignoring orchestration at this point, but just looking at what the tools have done, they've been really concentrated on the developer. When I think about containers and production I think, I'm trying to think, we're trying to look at from the operator's point of view, what are operators going to do when they get containers and production? What do they look for? So it's really sort of rethinking about containers. Most container work up to this point is concentrated on the developer, Docker build, Docker commit. It's all about building these containers, building your applications, and the standardization of OCI format, new tools can arise, so up till now everybody's been building containers with Docker, as I said, Docker build and Docker commit, but really I see there's a standardization of the format, image format, and I'm starting to see other groups starting to build containers or container images using other tools, and I think that's good. I think it's good to have innovation and stuff looking at how you build containers. So I've seen the Ansible right now has playbooks. They're allowing you to build containers with playbooks. OpenShift has a thing called Source Day Image, and they're building container images that early today. We used to talk by Fedora talking about how they're doing layered container builds in there, and then there's another project that's been going on for a while called Docker. But the real goal here is to allow people to build these applications, and as long as the output of these applications is the same, I think that's good. It also means though that we don't have to ship a Docker engine and a Docker fully instrumented for building containers in production. But really why do I care about that? What we're looking at here is the copy on write file system. So we look at containers, container, the whole Docker infrastructure has all been looking at adding copy on write, because when I'm doing Docker builds, and when I'm doing Docker commits, what is that? I'm basically taking what I had before, I put a copy on write file system on it, and then I continue developing the applications, and I'll eventually save that application to a new image for me, and that's all built around copy on write. When I'm running containers in production though, I don't want my application to be able to modify itself. I really don't want applications to be able to modify slash user. So in production, I would argue that we want to move towards a state where we're not necessarily using copy on write file systems in general for doing that, and that opens up some opportunities. So if we look at the process of copy on write systems, things like device mapper and butter FS, both break shared memory. So right now, if you run device mapper containers, which is the fault for Fedora and all Red Hat based operating systems, we run with device mapper. The second default is AOS, we're going to talk about Oval AFS, and then you have butter FS. Well, butter FS and device mapper break security, sharing memory. What happens is these are block based devices, and there's a problem that the file system and the block device don't communicate well together, and the bottom line is the kernel figures out that a shared memory is the same. So if I load glibc and another process loads glibc, the kernel looks at the file system and makes sure that the file system image, I'm simplifying this a lot, and the block device is the same. So it can say, oh, that's already loaded into memory, so I can use it. As soon as I start using device mapper or butter FS, the kernel gets confused. So we can no longer figure out that even though it's using both systems are using the same low level image, the kernel can't figure it out, so it ends up loading shared memory. The problem with that is if you run in 10 containers in a system that are loading the JRE into the system, if I run them on standard file systems, the kernel will figure that out, and it will load one JRE into memory. I'm talking about copy on write file system, right now you load 10 of them. So we end up instantaneously using tons more memory when we're using copy on write file systems. Another copy on write file system is Oval AFS. Oval AFS actually doesn't have the shared memory problem, but it has other problems. Nice thing is if you've been following the kernel patches, we've been working very hard to fix a fundamental problem with Oval AFS, and that's SC Linux. SC Linux support wasn't there, so we lost a lot of security. If you went to an Oval AFS for your back end, you've had to turn off SC Linux support for separation of kernels. My team has been working heavily on that. We have patches both going into the Linux kernel for Oval AFS and to the Linux security module, and Paul Moore is in the back somewhere here. He's the one. We're merging into the SC Linux part of it, and those should come probably by the end of the year. I'm hoping to have a raw high kernel out in the next few weeks that people can start to play with Oval AFS and SC Linux support. But a real problem with Oval AFS is it's not a politics-compliant file system. So there's some fundamental problems that people run into when they use Oval AFS based on assumptions on politics compliance. It really was, so the one we hit originally was a program that opens up a file descriptor and then reopens the file descriptor or opens the second file descriptor on the same file. They'd open up the first one for read-only and then opens the same file descriptor, second time for read-write, and it actually ends up with file descriptors to two different files. Because when I open up a file for write, it actually does a copy. Yeah, it copies the file into a new place. So RPM did that. So we ended up with RPM having two, you know, thinking that it opened the same file twice, but it actually opened up two different files. So there's other problems, things like she had sockets that basically there's problems with Oval AFS. But I think in a lot of cases people are going to start moving to Oval AFS once it gets a little more at the corner. So another thing, copy and write file systems sometimes suffer from write. They don't perform as well as XFS. You know, if you're using a standard file system, you're going to get less performance on a copy of write, especially when you go to larger scales. And as I said earlier, in production, most images should never be modified. I don't expect, when I ship an Apache version, I don't want Apache being able to rewrite Apache. Just fundamentally, you don't want applications able to write to the slash user when they're running the production. So read only container security is much better for security. If I can only write to certain directories, I can write to my executable path, so it's always considered good. Lastly, copy and write does not support network storage. So you can't run a copy and write file system on top of NFS, it's top of the list of steps. So you lose the ability to share your images from a centralized side. You're going to see some cool stuff coming up with that. Each consists. So a fundamental problem with Docker or a container model right now is, imagine I have a Kubernetes cluster that has 10 nodes, and also there's a huge spike. I always like to use the Sports Illustrator's swimsuit edition. So the Sports Illustrator gets so many hits per week, per week, per week. And also the swimsuit edition comes out, and they go like 100,000 times more hits on their website at that point. Well, that sites out in two weeks they go roll back. So you need to have the sudden expansion of hundreds of service. If you're running a container environment, what you want Kubernetes to do is suddenly launch, say, 100 new nodes. Well, each one of those nodes, in order to run your application, has to now go out to Docker or some registry and start downloading the images. So it has to install. Or you have to pre-provision all this stuff. So just think about that. As you expand out, you have all these images. Just image data moving back and forth before the application actually starts to run. If I had a shared file system, and I ran the same image on these multiple sites, as soon as the site connected to NFS on the server, your application's there and you ready to run. So just think about that from a scalability. Scaling out, if I could use my images stored on a network storage, it would be awesome. Another thing about shared storage for my images is all of a sudden I have a vulnerability. I have 100,000 installed base. Shell shock happens. Now I have to go out and do 100,000 image updates. If a lot of those machines are all sharing off the same NFS, I just update that image on NFS and it's in fantasy here, but it gets updates. I'm like, so do we want to get rid of our cloud file systems? Some ways, in production we want to, and I have Daniel Ruiz, who's one of our leads, always comes screaming into me about peck containers.  And you run that container forever, okay? So we can't fully get rid of copyright file systems, but in production, in a lot of cases, I don't think you need to run any applications on top of copyright files. So we've been working with this, the atomic registry, atomic or open ship, the name goes back and forth all the time, whether we're calling it atomic registry or the open ship registry, it's basically taking the Docker registry service and adding additional features onto it. So what we've done, what my team's been doing is working on a service which is watching container images to arrive. So when someone does a Docker push to the registry, we have a service that's watching, sitting out there watching for messages coming in and telling us that an image has been pushed. At which point this application will explode the images onto an OS tree. Why are we using OS tree? So we don't have to use up as much space. So you do a Docker push from your server, the application goes out to the registry, this application will watch for that and then we'll explode it onto an OS tree file system. The first feature we wanna add is the ability to do an atomic scan. So what we have is hundreds of developers out there pushing applications to a service. We have an application watching it. That image comes out and we explode it out. We do an atomic scan, figure out whether or not this application has vulnerabilities. If it has vulnerabilities, we can basically deny the push. All right, so basically in order to do that, we have to explode the image onto this and then run the scanning on the exploded image. But what happens if we share that exploded image after it's been exploded via NFS? Steps. Blaster. Let's look at it. Where is my notes? I'm going to solve for engineering to turn it red hat and I'm working. We wanted to demonstrate a little bit of the software that we've been working on, which is called the Openshift Explorer, which is an image explorer. Basically what it can do is it can watch the Openshift registry or the atomic registry for changes to images and explore those images onto this so you can inspect them by making a building over the scanners. So to start with this demo, I've just got a standard Openshift deployment of your virtual machine and a default Docker registry point. So nothing really special going on with that, other than that I've configured it for persistent storage so that I can use that storage inside the Explorer. It saves us a trip. So it might be, I've called this and I've gone into the directory. It's just a go program, single binary. So it does it really quick and it builds a binary and a Docker image that we're going to use as the base of the Kubernetes pipeline. So we'll just build that image and we're going to deploy it using Openshift so you might deploy a different service than Openshift or any kind of Kubernetes service. So after we build the image using a Docker, we have to give this program a few privileges. Namely, we have to create a service demo. We have to create a deployment config for this pattern so that Openshift will create a replication which already keeps it alive. And it also adds privileges and most expeditions like for us to do that just for this demo is to give that service account a faster admin. So it's going to overkill, obviously, production using something more restricted for the purposes of the demo. It's the easiest way and we can give it a big upgrade. So going back to our Openshift console, you can see that Docker Explorer pod has been created and it's got one replica. So it's running now and if we take a look at some of the images we've got, basically this pod is watching the image streams and is going to explode them onto disk in specified locations. So it's actually pre-pushed and in which this busybox image. And we want to see if our Explorer pod has exploded. So I'm going to the pod now, looking at the logs and you can see it has a message exploded that it exploded default busybox. At least if we go back to our favorite VM, we can take a look at the explode directory and see the contents of that image have been unpacked into this specified directory for us. So we have a few folders here, digest images and repo, I'll explain the repo in just a little bit.  So something like busybox might refer to a checksum of an image which would be located in digest. So digest contains the actual file system and images just contains this file link which points into that digest folder. So it's a way for us to update the tags but also keep images around. So you can see this link contains this checksum which points to a specific location under the digest folder. This log checksum in this real fast folder contains the actual contents of the busybox image. So this is a pretty first-level system. You can do quite a lot once you have access to the real fast. You can run scanners on it, basically do whatever you want. One of the more interesting things we can do is share it. So I've got an export configure to just share this with world read only. And I'm gonna activate my NFS server so that I can browse this on my host machine and we can see what kinds of things we can do with this system once it's been shared between two machines. So back on my host here, I'm gonna mount this head slash mount and you can make sure I have the address right. Doesn't look like it's connected. Let me check the IP. That's changed. Fix that really quick. And that looks like it's now in write-up. So now I have access to these real fast folders and slash mount, you can look at it and play and do a tree on them. So you can see I have the busybox tag and it's really fast to exploit it here as well. So I'm gonna configure it for a watch on this tree so we can take a look at it later. And I'm gonna go back to the favorite VM and I'm gonna push an image that I've tagged to be on the OpenShift repo. Specifically, I'm gonna push an Alpine Linux image. It's pretty small and I should push very quickly. And so once we push this image up, we should see it's immediately exploited on the host machine, not just in the guest VM. We also now have access to this image anywhere where this shared NFS server is located. So I have actually also prepared a script which will create a container using over the FS of any image in slash mount. It's just kind of a hack that I put together. And so if I use the container of default busybox and latest it will just drop me right into a busybox shell even though I don't have the busybox image on this host. So I can look at the release file. I'm not sure if busybox has one, but you see the point that it's a standard busybox shell and I can do anything in that shell that I can do it in the busybox container. It's a run-c container running with OCI utilities. So the hack does really clean up the folder so I have to clean it up manually, but I can just create containers instantaneously out of any image that I approach to this registry without having to pull them and I don't necessarily have to use Docker. So there's really no Docker involved in this except in the push. So on any of the clients I don't need Docker. I just need run-c. So I'm gonna pull Fedora and then tag Fedora and go on to this registry. And I have to put it in a default namespace so that it has a project that it lives in. And then I'm going to push it and it'll take a second to push. So while it's pushing, I'm gonna explain some of the technology that's going on here. I mentioned earlier that we go forward. Basically what we're doing is we're pushing these images and exporting them into an OS tree repository. So OS tree is specifically designed to store a number of file system trees. So it's kind of a natural fit for images that you would never want to change. You just want to create containers out of them. And the benefit to OS tree is that we can store many different images in the OS tree repo and it will be able to duplicate common files. So two of them just have the same sort of large JRE or whatever library it can detect. Those are the same object and it manages them with hard links. So they'll actually get linked against the same file system or resources that won't consume extra space. So if I created a small modification to an image and pushed it, it would not take that many extra space. So it looks like the push worked. And we're gonna go over to the host machine and see if we can start it. And it doesn't look like it's, it has pushed but it does take a little bit of time to explode. So let's check up on it by looking at the images directory and looking at the log here in the OpenShift console and we can see that it seems to have exploded but it can't take a little bit of time to create the tag that it uses to resolve names into image checksums. So I can look at the link file and I can make sure that the link has been created correctly by just inspecting its contents. And despite the fact that my terminal gets a little messed up here, it does seem like the link has been created and if I run this script again, it'll drop me right into a Fedora shell. And I didn't have to do a Docker pull Fedora or anything. So I'm gonna run the version of Docker and I can look at the release to see if this is effective with the Dora 24. Okay, obviously we gotta do a little cutting on the length of this but you get the basic idea. He sent me this last night but we basically have the ability at this point to have a Docker registry sitting out there that instantaneously as soon as any image gets pushed to it, the image would be available via NFS. And it's not, right now we haven't implemented it yet but there's nothing to prevent you from using Cess or Cluster or anything else. We'd actually like to get to the point where, we'd like to get to the point where we can share these things even using OX3. So we wanna be able to support all different protocols, network protocols for accessing your container content and not have to do a Docker pull on every single service and whenever I wanna get a container. So that's first point, system containers. Giuseppe in that corner here has been working on the concept of system containers. And what system containers are on, first of all on an atomic host. How many people are here playing with atomic host? Okay, a whole bunch of people play with atomic host. What's the most aggravating thing about atomic host? It can't have been. Yum, doesn't work, right? Yeah, yeah, you can't install Vim, you can't install Emacs. So it's real aggravating when you get on there to do it and really what we're trying to do with atomic host is we're trying to implement an operating system where all the software has to be installed in front of container images, okay? So what we wanna do is have software must be installed as container image but we have a catch 22. Kubernetes requires SCD and FlannelD to be running before Docker comes up. So we have to have a couple applications installed in the system in order to use Kubernetes and those applications are actually gonna be used by Docker. So how do I do that? They need to stop before the Docker so it can set up its network and these containers must, well, those containers can be run on when you only file images and Docker has problems with container priority. It doesn't have any form of container priority. Now Docker's talking about putting it in but up to this point they haven't had it but the dirty word that Docker doesn't like, system D, has priority, right? You can set up that says Docker relies on FlannelD running and FlannelD relies on SCD being up and running. So I can set up a priority list using unit files to set up proper starting of my applications but I can install SCD and FlannelD because DNF doesn't work. So I have to install them in the form of containers but the only thing I get to run containers is Docker I got catch 22. So we came up with a console we were calling system containers. We're using the atomic command. So the atomic command is sort of the atomic CLA command is a tool we're using to wrap all this functionality to build the atomic command is how you would install and now you can do an atomic pull and an atomic install of what we call the system container. There's a new tool called Scopio that's got showed up in Fedora over the last few times it's actually inside of rel and sent us at this point but you can use Scopio now it's been enhanced to pull images we're going to talk a little more about Scopio as we get through the presentation and we use OS tree since we want to have images stored but we don't want to use up time at this space we're going to again use OS tree and since atomic host is based on OS tree it kind of works well together and we want to create a system to unify because we want to set up this precedent so that we can set up a precedent a little bit of containers out and out and I'm going to use Run C everybody here Run C okay Run C is a new way if you were using Docker 1.11 Run C is a standardized pod so what's called the open container initiative it's a standardized default way that you run containers and Docker actually in 1.11 you're using Run C underneath the covers to run containers but we're going to use Run C directly inside of our system to unify us so atomic command has been implemented to handle this so you can do an atomic pull if the item is raw okay anyways you can do an atomic pull dash dash system at CD and atomic install at CD system at CD we put out at CD and slendled the containers that can handle this they're both available on Docker Hub right now so you can actually install these on top of atomic codes you can install them on REL system or Fedora system as well it's nothing preventing that yep wouldn't that atomic but doesn't atomic install do a pull automatically if it doesn't have it on the system do you really need to do the separate pull I don't think you need to you can just do an atomic to do an atomic install yeah I think you can do an atomic install dash dash system at CD so this is based on the document that I wrote six months ago so that probably should be that's why you guys are experimental so with system containers now we're using Run C we're using system D and we're using OS tree for the back of the back of the store but the container you can't update the container right when a new version of the CD comes out you have to use a tool like Docker some other tools to build a new image push it to a registry and then this tool would pull it down right so again it's in production think about that you're not developing it you're using it in production another thing that we're adding is what we're calling simple signing we've been wanting to have the concept of simple signing what we mean by simple signing signing that is similar to what we use with RPM where a user is able to go out and sign a document and say hey I trust this image Dan Walsh put this image out and Dan Walsh trusts it okay but now you get the image in your company and you're working for Acne.inc and you come out and say well we're not necessarily going to trust all images that are signed by Dan Walsh but we trust that image that was signed by Dan Walsh so that Acne.inc can come in and sign that image so we can actually end up with lots and lots of signatures based on the same image and the signature can be separate from the image all it has to be is you add it cryptographically prove that this image was signed by Dan Walsh's public key or by Red Hat's public key or anybody else's public key but we're adding the concepts to atomic and these tools for simple signing it allows you to use a PGP type signature very simple signature to sign it sign basically the checks on that is appropriate for the OCI or Docker image multiple people can sign the same image signatures can be stored on this new atomic open shift registry but also we want a lot of signatures to be stored anywhere if you want to put signatures onto a website and say you know there's 40,000 Docker images out here I trust these four okay how do you do that so what I would want to do is take those four images get their checksums sign their checksums and put it on a website somewhere so now I can do an atomic pull of that application set up a tonic so it goes out to the registry to my website and says okay I just pulled this checksum Dan does it matter what you sign yep Dan will always trust it so we can trust and install on a system or ACME trust that we can install on our system really simple stuff not building up a hierarchy but just basically allowing people to play with with signing based on this concept then we want to eventually build policy rules engines that say you only allow your applications to run certain applications so we can build a Docker and some of this is not quite built yet but we can build an authorization plugin that basically sets up a policy that says I'm only gonna allow my Docker engine to run applications that are signed by ACME.acme but now if your random user goes and pulls down an application when that application goes to run it'll check and see if it's properly signed if it's not properly signed then you get thrown out we want to build this tooling all the way up into Kubernetes so Kubernetes can start to have a policy around this type of thing, yes what about revocation, make our keys compromise yeah I mean you could we have that problem you could remove the signature file but yeah we always have that problem we've had that problem with RPMs for 30 years now so right now we're just trying to get the simple signatures we can talk about revocation in a later time but most ways that people have done revocation in the past we could build that into it but let's not get bogged down on that we've been waiting for some kind of simple signing for a long time okay I don't know how much time really good plenty left okay so we talked a little bit about signing and we talked a little bit about using the atomic command and the containers and being able to run containers on shit copy and write file systems on shit storage we want to look at a new tool that we're building now okay it's called OCID and basically it's it's components needed by OpenShift and Kubernetes in order to run containers in production but we want to look at a simplified demon but the four components that we're going to use in an OCID environment the first one being container image transport so we need a way to get that image to the local system especially if I want to run a copy on a write file system then I need a way to store it so once I've pulled an image to my machine I need a way to store that image on disk the third component we're going to need is we're going to need a way to run the container okay I need a way to actually start a container and then I need an API a wrapper around all this application so what we're looking at building is this OCID spec now let's go look a little bit deeper into each one of these so the first one the way we're going to pull the application container image transport we've already mentioned scopio in terms of the system decontainment so scopio has gone out and implemented basically the equivalent of docker pull and docker push okay it also has full capabilities to do the simple signing scopio was originally written as a tool for remote viewing so scopio is Greek for remote viewing just to give you a little background around we've been using scopio for the last few releases for viewing image containers on a registry so right now if you want to look at a particular application a particular image is JSON using just standard Docker you have to pull the entire application dot in your system and then look at the JSON what we wanted to do is basically look at the data at the registry and without having to pull it so that's why we originally built scopio but after we built the ability to look at the registry we actually figured out it was easy to implement the process of pulling and pushing using the Docker protocol we've added the ability to push and pull images from the registry and we can pull and verify simple signatures as well and we've worked with CoreOS to split this go library into a separate thing and we've decided to put it out inside of the containers so basically it's said to put into scopio which was a CLI we've broken scopio into two parts so now we have a vended go language library that we're calling container slash image and this gives you the ability to pull and push it the idea is to allow other applications to potentially use this library inside of their applications not just hire scopio but allow other applications to use and there's since CoreOS is working with us on it there's potential that they can use inside of Rocket so we basically wanted to standardize and rip out one part of pulling and pushing and put it into a separate application that could be used and just have people work on just that part enhancements so the second part, now we have the ability to pull and push images from the Docker registry or inside of registry now I need to store it so after I pull the image I need to be able to store it on the desk so anybody here play with the atomic CLI okay when you play with Docker images you always have this problem I want to look at the content of the image but I don't want to actually run the container on it so back several versions of the atomic ago we added the ability to mount an image so I can take a Docker image and store it in the content and actually mount it at a mount point where you can do atomic mount fedora slash mount and then go to cd to slash mount and you'll see the root of S of the fedora image that's installed in the system the really cool thing is if you're running containers you're running a container on the system you can do an atomic mount that's live fedora slash mount you can go into the containers mount point create a content then go into the container and actually see the content show up but really we're playing around underneath the covers and allowing you to actually work with the copy and write file system while it's running inside the container so we did that functionality a while ago it works with device mapper and overlay fs but we wanted to make that more standard the problem is that the Docker daemon has all its blocks all its handling of this copy and write file system all of the graph drive code is built into Docker and Docker is actually using a locking system and locking system is all I remember so there's no way for other applications to interact with the file system while Docker is working so we originally started to look at could we take the graph driver all the copy and write file systems moving into a separate tool that Docker and the separate tool could use simultaneously and we actually found that it was very very difficult when we started to play with that and we decided to stop doing that and just to pull the tools out and get two of our tools working together using locks on the local file system and once we have it perfected we want to go back to Docker at that point and say hey we've got all the storage drivers now working independently so different applications can use the same storage at the same time as long as they interact properly in the locking system so let's backport this into Docker so we decided rather than start with trying to manipulate Docker in it we wanted to prove that we could use it and it turned out that it's kind of neat that we can be able to use all these copy and write file systems so we call this tool a different integration it's graph C, graph tool, cowlman and store tool so I think right now it's called store tool so we want to create a Docker graph drive code that's dependent on library CLI and we've again worked with Core West and we're creating a thing called container storage and all this is doing is setting up the storage to run images, set up those root FS's on the system either using, right now we're just implementing all the copy and write file systems that are currently in Docker, independent of Docker but I want to add support for OS 3 network storage so I want that tool to learn if I have an image stored on an NFS scale if I have an image stored on OS 3 deal with an image stored on a copy and write file system then I deal with it and we can start to look at how we'll be able to launch containers on different file systems different back ends based on this tool and again, it's simple all this tool does is handle file systems totally independent, the library is totally independent it can be, people can work on this tool at a separate pace they don't have to be working with other tools and we can take patches independent of other tools and CoreOS potentially could use this as a storage back end right now CoreOS Rocket just supports overlay FS's in the back ends so if they switch to this they'd be able to get advantage of using all these different file systems back ends open container initiative runtime this is the further salon of all these tools right so we've got something in pushing and pulling we have something that actually stores the image on desk the next thing we won't need is ability to run a container so the open container initiative was started about a year and a half or two years ago now a year ago, I don't remember exactly when but basically what's happened is RunC got developed based on the open container initiative RunC in a specification for how you specify which container you wanna run what the applications are, what the labels things like that have all been standardized you mean used by Docker they're being adopted by Rocket at this point how the framework, how the tools is starting to build using the specification RunC is just an initial implementation of the specification and we're starting to see other people Clare Linux is about to release a mechanism for running containers on virtual machines and they're following the specifications so if they're ready to swap out RunC with Clare containers you'll be able to run these containers but they follow the open container open container initiative specification and as they go forward I think we'll see run other systems using it as with Docker 1.11 it uses RunC as a default back end so all the software's shared between Docker or anybody else that wants to use it and the OCI tool so that's all about RunC one of the problems with using RunC is you have to actually go edit the JSON files if you wanna configure your if you wanna configure a container to run you don't have a simple Docker tree command line or a Docker run command line to configure your JSON you have to actually go all VI really complex JSON files to create a specification that's what RunC uses to actually run a container so we implemented a tool called OCI tool which is actually sucked into the open containers onto OCI tools and what that allows you to do is it basically falls pretty closely to the Docker run command line but you say OCI tools Docker tree command line OCI tools generate and then dash V whatever you want in mountain volumes dash dash PIT equals host all those kind of commands and then it just goes down and generates the JSON file that you can use for your container so it makes using RunC a lot easier because you can specify on a CLI what you wanna run or you can continue to edit the JSON file OCI tool generates a specification and you can launch RunC if you look I believe that the system containers that we were using earlier we used OCI tools to generate that he was smart enough to generate it and so if you wanna play with RunC I would heavily advise you to use OCI tools to generate the specification so the last part of this is the OCI this is sort of a wrapper around this application and we're actually wrapping this all these tools together underneath an atomic command but we wanna be able to do it both in atomic command and as a demon so OCI-D is sort of the tool that wraps up all this stuff it's the open container initiative demon I prefer to call it OCI-D but I think it's funny, but anyways this is the least developed part of this but we decided to develop it and what this does is Kubernetes I guess one of those things Kubernetes has specified ways that Kubernetes can talk to container runtime environments so the rocket is now using it I think they still talk to Docker in a traditional way but Kubernetes has basically said if you create a demon that will follow this protocol and you can set up a kubla to talk directly to that demon and have it launch containers on it, we have so OCI-D is an implementation of that API as specified by Google site Kubernetes and it's a service for launching containers that Kubernetes can talk to so when Kubernetes wants to execute a container it's gonna send OCI-D a message that says I wanna run this container OCI-D then it's gonna go and pull an image using Scopio so it'll go out and somehow get the image down if Scopio slash the image tool knows that the thing's available by NFS it comes right back and says I already got that image or if it's pulled down to its image store but after if it has to pull the image or pull it then OCI-D stores the image on this using the storage tool that we talked about and then finally it will launch the application we want to see okay so that's OCI-D and it's a standards-based alternative to Docker and Rocking okay so it's a standards-based but the goal is to make this a fully open source project anybody can contribute to it and we want to event right now it's underneath containers we want to eventually get it into potentially get it to OCI as being a standards-based thing but again it's for running containers in production we're not trying to replace Docker okay we don't want to that Docker has a whole bunch of functionality the goal of this tool is not to implement Docker build, Docker commit in some of the other features of Docker all we want is a standard tool that is basically able to do what you want to do when you container is in production which is I wanna run this application on this machine so it needs to be able to pull the application from the service install it on some kind of file system and then run the container and that's it and we want to base it on right now it's based on Kubernetes but again we're implementing the same functionality inside of Atomic so Atomic will eventually be able to not only run system containers but you'll be able to do Atomic pull and do dash dash instead of OCI backend you could say I'm gonna use the device mapper backend and it'll pull it into your local storage device mapper not inside of Docker and you'll be able to launch containers based on that and eventually we want to get once we have this up and running we want to move open to OCI-D by default and then have a standardize another nice thing about this is where Docker is constantly changing you know this will give us a little more stable underpinning for running containers but we don't have to constantly swap out based on all the innovations of building containers and different things that Docker is trying to do and we'll get a little more stable back in so we've got questions at a second but containers in production will focus on system containers and system services and basically Kubernetes running clustered applications that's our goal and at this point questions yes so what is there anything conceptually preventing you from running application containers under the same system like what's the well we need it in this system we need to by application containers I guess you mean impact container no I'm saying so you're emphasizing the use of this just for the system containers you need to run as EDF allow Kubernetes right but let's say I just throw my WordPress container at it well I would say that's a container in production WordPress is neat right so yeah when you're out defining as a non-production container there's one that's under development so if you're building a container if you're experimenting with a container you're going to probably need a different tool than this this is more about a container image already exists at a registry and I am putting it into production okay anybody else yes so all that I mean probably it looks like that OCI-V has similar features with container-V container-V doesn't implement anything to do with storage if pulling and pushing all container-V does is execute Run-C so yeah it would be similar functionality there would be nothing to prevent eventually something like Swarm or take advantage of this tool as well all we're saying is we believe fundamentally from putting a container into production you need to be able to get the container installed onto a machine load it and then run it so we're adding those three components and then having a demon again the demon can talk to Kubernetes and we want to allow all three of those packages to develop all four of the components to be able to develop independently so we don't have to lock everything into one big monolithic application yes so beginning you said that Docker is focused on developers and it seems to me that it is focused on developers so much that it's actually not suitable to run it in production I'm not saying that don't put words in my mouth it seems to me that is what I'm saying you said it I didn't I'm not getting into trouble now I think in the future I want to support Docker I just want to look at what in my opinion is better ways to run containers in production than covering Docker workflows again I think Docker is great for building applications I think it's great for running containers I just need alternatives so if you envision the foreseeable future of workflow where you continue to do Docker build and Docker run during development and at some point you do a Docker push to promote it to some production indication at which point it goes into the containers and production framework that you described here correct I think that now the Docker becomes one way of building containers I think other all I really want is I want different components I want to get back to Legos of pandering the situation and not have everything be this one monolithic where we're constantly begging people to accept patches because I want some different workflow separate these out into different components I think the development will get easier at least for production anybody else? yes maybe a slightly provocative question we have people like Courtney and Landon working on minimizing our base image is it true that one of the things we're telling things about using remote storage is that any extraneous bits that aren't actually used during running don't have to get pulled whereas as you described you can so isn't it true that if we get to the platonic ideal of no extraneous bits in the image that you're running then there's not that advantage of shared storage where containers goes away you got back to the point and I think you want to have it configurable where the admin can make that decision if every time I boot my machine I have to pull down the 50 gig executable then it makes more sense to install it locally if every time I install my application I'm just launching a patchy and it loads in some CGI scripts I can take that overhead constantly rather than dragging down the big image and I get the advantages but we all make that stuff one of the nice things about using NFS is I believe in production almost everybody's going to have the data because if you want to have a large cluster you have to use shared storage so you already have NFS set up or Gluster Assets set up so why are we picking out the executables have to be pre-installed or forcing it to always be pre-installed so we need to take advantage of the tools that people have been using NFS for 30 years for a good reason and I showed NetApp with this patch are you going to show that app? one minute anybody else? okay any comments about the presentation I'd like to hear about them as I said we're probably expanding this we go forward and we're going to sit down on the amount of time that's shown hopefully the next time I do this I'll be able to show the entire OCR ID so thank you for coming


Hi, beautiful. Getting a perfectly snatched pony can be a daunting task sometimes. Getting all those little bumps out, making it super flat and tight to your head, it takes a lot of practice.  It can be a beautiful, glamorous, fabulous process that can be actually really easy to accomplish in under five minutes. Today I'll be showing you how to do three different types of ponies, a nice and low one down here, a mid-height one, and a very high Ariana Grande pony.  So I'll be showing you all the tips and tricks I've accumulated throughout the years of doing hair, and I'm just gonna say, stop wearing the saggy ponies, please. You never look effortless, you just look sloppy. I want no more saggy ponies in this world after this video comes out, okay? That's my goal here. Why don't we just get it started? Let's do it. No more saggy ponies. So you're gonna need some tools for today's video. These phalanges are not enough. You gotta have a lot of power on these things, but you also gotta have other things. So today I'll be using Wave Tech. You can always use hairspray or any mousse. You'll need a comb, preferably a rat tail comb. I'll be using a Mason Pearson brush to smooth the hair out today. You can use kind of any brush. Obviously, Mason Pearson is most optimal, but it's a very expensive brush. You don't need this, you can use pretty much anything. I'll be using a hair bungees today. It's these things with the hooks on the ends, okay? You can also use just regular elastics, but these are much easier when you're doing really slick back type ponytails. Also, you'll need a flat iron, curling iron slash crimper, anything you wanna do. However you're trying to style the actual pony, you're gonna need that tool for it, okay? I'll be using all different kinds of things for that today, but it's whatever you want, really. Other optional things you might need are a toothbrush for smoothing out edges and getting those edges really, really laid flat. I'll also be using a Viper smoothing oil. Don't exactly need it, but you wanna hydrate those ends as much as possible. You'll also need a broken tripod stand. That'll help you really do a good pony at the joke. Let's do a not a broken one. That'll really help all of us. Hi, Miss Manny Quinn. Haven't seen you in a long time. She's been retired for a little while. Haven't needed this long, luxurious hair for anything recently, but you're back. You're one of my favorite clients because you never talk back to me, and today I hope it's not gonna be the day. Okay, so to prep your hair, you're gonna want it nice and flat. I gave her a little blowout. It doesn't need to be as perfect as this. Just as long as these edges and sh** are all smoothed out, this is gonna be essential for making a nice, sleek ponytail. We're grabbing our calcone, and we are doing a center part, which is already there. I'm also just gonna part it going all the way back to apply the Wave Tech foam.   I definitely learned a lot from him, and a lot of these techniques come from him. Just wanna give credit where credit is deserved. Today I'll be using Wave Tech because Wave Tech has gentle holding properties along with shine benefits and a little bit of hydration. It's gonna be the perfect product to get that really slicked back ponytail. I'm gonna put it right on the comb like this and just rake it in there. Now, this will dry, so don't be alarmed if you get all wet hair, and we're gonna go through all of these sections and do the same thing, and just get that Wave Tech all on the root. I know it seems weird putting a Wave product when you want your hair to be pinned straight, but a lot of products can be used for different things besides what they are actually advertised for. I do that all the time. I will use products in the way I kind of think they should be used instead of how they were prescribed to be used, and make sure you're getting all the edges and the back of the head. Yes, Miss Manny Quinn is going on a Tinder date tonight actually, so we are trying to make her look as best as possible. She is in fact a lesbian and we love. All right, and just like that, we have one side complete. It took me no time at all. You gotta get that product in there. Just helps lay it down so much better in place. It's already staying back. Look at that. See, that's what we want. Now let me quickly do the other side. We have the beautiful Wave Tech Wave foam all applied to the head. As you can see, it's already so much flatter. Ooh, it's looking spicy. Ooh, this is my ponytail dance, baby. Let's do it. Okay, we're gonna use our Mason Pearson brush or whatever brush you have, and we are going to start brushing the hair down. We don't wanna brush it up. We wanna brush it down and around the ear. And as you can see, that foam is really helping smooth the hair out.    So we wanna put the head back when we do our point tail and I'm just gonna rake this all into my hand. Oh, she is gonna have a migraine tonight. So I'm gonna take my hair bungee and I am going to start on the bottom and wrap it around. Bungees are awesome because my hand doesn't have to move. And I get to get that really tight in there. You see that magic, dude? F*** with me. Are you kidding? And let me just tell you, Ms. Manny Quinn here has a lot of hair. So to get this literally on her head, the mousse really helped. Now you can also go in and blow dry the mousse down. However, she's almost dry at the moment. And if your hair is as thick as hers, I'd probably recommend putting an extra elastic. Please use an elastic that's the color of your hair. It really helps conceal it. Now I'm just gonna take some hairspray on my toothbrush. I mean, if she doesn't really have anything going on up here, but if she did, I would just smooth it out. Oh, do you see the way she just like looks so good now? Now I'm gonna take a hair from underneath, move it out, make sure it's nice and straight and I am going to begin wrapping this hair around as I start to spray it with hairspray. That way it sticks to itself and doesn't come undone. Because who knows what she's getting into later. I don't know. And when you get to the end, I'm just gonna stick this hairpin into her hair, close the hairpin and I stick it right through the base of the pony. And now it's sticking out here. And I just yank it through and we have a perfect closure on our ponytail now. Now I'm just gonna grab a little bit of Viper, put it all over my hand, run it through the ends and the top to add shine. And now you can leave the hair straight or you can do something else with it. I'm gonna use my Babe Waves crimper today to put some crimps in it. I'm just gonna start on the bottom of her hair. It's the easiest for me to lay the hair on top instead of moving the hair out of the way. And as I crimp the hair, I'm just gonna push the hair back up towards her. That way it reinforces that crimp. Oh, Sam, who is this? Yeah, you got kind of slayed. She's looking so good. I love the big, full waves. If your ponytail is not as full as hers, you can totally get ponytail extensions. I just love the drama of it being really sleek and straight up here and it being really big dead here. And also her hair is really soft, but you can tell it has hold and it's so flat. That is exactly what we want. That is the low ponytail. It's super easy, guys. Just a few steps and you're done. You gotta have the right techniques, obviously, if that isn't clear already. We're gonna move on to the second type of pony, which is the middle of the head pony. The standard pony, if you'd like to call it that, I would not like to call it that. But let's move on to that one. On to the next pony, baby.  Okay, it's about to defy some gravity, baby. We're gonna comb the foam in in the direction that the hair is going to be going. So it's gonna be going pretty much up like that. And then we take our next section. We apply more foam and we go this way. I already have foam in the hair, so I won't be doing it again in every single section. All right, we're getting rid of this part up here. We're saying goodbye. We are going to put our hand where we want the ponytail to sit. Right at about eye level back is a great place for a pony. And this should give you a slight facelift from the front. That would mean that you're doing this correctly.  Should definitely hurt a little bit. With the head back, we're gonna wrap our bungee from the bottom and we're wrapping it around. I said slick and defined. Now, because our head was directed backwards, you can see that there is absolutely no bump or anything going on underneath here. And we got it just so smooth from that foam. Like, oh! Sorry, I don't know what's up with ponies. It's really doing it for me today. All right, now that we got that secured, you can also put another ponytail over this, just a regular elastic one to make sure it's really secure, but I won't be doing that today because I just don't think it's necessary.  Because I know not everybody has this much hair. It's really important that you're getting the pony in the right place. Since we're any lower, it would make her look a little frumpy. So we're wrapping it around. We are spraying as we go. Once you have about that much left, we are just gonna put our hairpin on here, close the end and stick it right through the ponytail.  And that way, the ponytail actually stays wrapped by itself. You can put a little more hairspray on that to really smooth it out, but that looks fire. I'm sorry. This will look even better on brown hair because you won't be able to see any imperfections because it doesn't reflect light as much. I'm gonna style this with a little bit of bend and bevel and like flirtiness going on. I don't want it too styled, but more like elegant. We're gonna put a little more viper in her hair. Boom, boom, boom, boom, boom. I'm gonna grab my one inch Marcel iron and I am just going to curl the hair like so. And we're just gonna take it out. Just a quick little one, two. And I'm actually going to pin this into place. You do not need to do this part. I'm just gonna use a little clip and just pin it like that. You guys would like to shop any of the products you use in today's video. You can below. I'll try and link as many as I can. We're just gonna unpin these. Got beautiful bouncy waves going on. And now for the magic part, we're gonna take our Mason Pearson brush, smooth that right out. All right, now that I straightened the back, we're just gonna put a little bit of hairspray on my toothbrush here. Get all those hairs laid flat. Can also do it in the back as well. Boom, baby. Oh, she looks so good. Now when she walks, everybody's gonna be looking at her. Hey, hold on, wait for me. I'm coming. And she's running, you know that type of vibe. And from the front, she looks so sleek and gorge. I love it. And that is the mid ponytail. She's looking fabulous. But we saved the best for last. The smashed up eye mother-in-law ponytail. Let's get this out and let's do that. All right, we're on to the highest and craziest, sexiest ponytail of all. If you're wondering why my client's on the ground, it's because that's the best place for her right now. Okay, you can also flip your head over to do this. I am going to, obviously, make sure we have enough wave tech foam or hairspray in the hair. Now this is gonna be important to really snatch this thing up. You guys didn't see I applied the foam in the last two looks. Why don't you just rewind to that? We're just gonna section off half the hair right now. I am gonna make sure that we have foam all underneath. Just like that. I know, it looks messy. It is. Pretend you have eyeliner on, right? Like a winged eyeliner.  We're trying to go, whistling, boom, right up here, like bang. I'm just gonna start gathering the hair. The Mason Pearson brush really helps that because it breaks through all that hair and makes it so super smooth. That's it guys, I'm done. All right, honestly, I had to wrestle her a little bit off camera because this stand does not work. So I had to put her between my legs and just yank your hair out. And yes, I did get 10 hand cramps trying to put this together because she has so much damn hair, but we did it.  Cause I don't think so. So we got this pony and as you can see, she has absolutely no bump back here. It is so smooth and straight. And from the front, she is looking delicious already. And it is staying up really high. Again, you can put an extra ponytail holder in here if you want extra security. But today we're just gonna leave it like that. Now I'm gonna take my toothbrush, spray some hair spray on there. And I am going to lay all those hairs in the correct place. You can even get some of those tiny little gaps out of the hair and just really make everything extra smooth. Perfect.  Make sure it is nice and straight. And we are going to begin wrapping the hair around and using our hair spray as we go around the head. And once we get this little piece of hair over here, we're gonna grab our hair pin and stick it on the end and stick it right on through there. Right into that pony. And we're gonna find the end. This is always like the fun part for me. I don't know why. Oh, this is a tough one. Oh my God, I'm gonna break my teeth. I got it. And you should have something like this once that's done. Now just spray that. Ooh. Now let's just put the finishing touches on here. I always like a nice beach wave with these high ponies. I don't know why, but I do. We're just gonna take our sections from the underneath like we always do and start waving with our one inch Marcel iron. And I'm just gonna open the iron up and twist around it. And I'm gonna make all these waves going the same way because I want them to clump together just a little bit at the end. But if you don't want them to and you want them a little messier, you can do them all different directions. That's totally fine too. Now I'm just brushing them out real quick, doing a little bit of teasing underneath to bring some more height back to this hair. Oh, she's looking spicy. Ooh, girl, you look so cute. That is the waves. They're all complete. That is how I want it kind of relaxed but also kind of polished at the same time. Just like a good in between the last two. I think this looks stunning. We did that gorgeous super perky high pony. It literally took no time at all. You can also finish this off with a little bit of oil or some shine spray. But she looks so gorge and flirty and fun. And this is the ultimate bad pony tail. You want to look like the baddest in a block? This is what you usually go for. I'm loving it. I wish this was my hair right now. I would just be rocking it. Better than Miss Manny Quinn could. Just saying. So those are the three most magical pony tails you can do to your own head. I hope that helped you be able to do your own pony tails. I don't want to see any more saggy pony tails. Like I said before, please make it stop. This should be quite simple for you with just a little bit of practice. So if you end up doing a beautiful pony tail, I would love to see it tag me on Instagram. Follow me everywhere else. Here is where you can find me on every other platform. Go for it. You guys can also text me. This is my number. Yes, it's really connected to my phone. And yes, I really do see all your messages. And yes, I will be texting back a few of you guys who text me right now. And if you text me, you'll be notified every time I upload a new video. If you guys want to check out Viper or any of the other products I use in today's video, they're linked below. You can also head to X-Mondo Hair on Instagram. You can also check out my color line. X-Mondo color, which is what my hair is colored with today. You can check out X-Mondo color on Instagram. And you can head to the website linked below. If you guys aren't already subscribed, it'd be so dope if you would subscribe. It is right down below. You just click the button. It's so easy. You can also click the bell icon to be notified every time I upload a new video. And the like button to be nice. Because it's the giving season. Today's Instagram shout out goes to Adrienne. She says, hey Brad, absolutely love you. I love you too.  And it was the first time dying it. And her hair was black previous to this. She used to have tight curls. And now her hair is so straight and damaged. Can I give her any advice on how to fix this and how to get the curls back? Unfortunately, your curls are gonna be probably gone for a while. You're gonna have to do plenty of hair masks and make sure you do bond building masks, things like oliplex. You can also color your hair with my color line, X-Mondo color. You can do it purple. And it will actually be beneficial for your hair. It will not damage it. It will actually make it a lot better than what it is right now. Other than that, you can use curl enhancing products like wave tech wave foam, like what I used today. And that will help bring your wave back. However, it's just gonna take time to get your wave pattern fully back when your new hair grows out. So I hope that helps you. And I hope you get your waves back. Please, I'm praying to the hair gods. That's all for today, guys. Thank you so much for watching. Don't forget to leave your extra lines. And I'll see you next time. Bye, guys.


I know I've been talking about it for like 18 months now, but I know at the start of this year I said it was coming soon and I really did think it was coming soon. I was waiting for a few panels, those panels arrived, then companies said they had new versions coming out. There's just been so many things chopping and changing, there's been shipping delays and chips shortages, all that sort of nonsense that's happening at the moment. I'm almost there, but I'm waiting on one more panel now and it's the Platinum LEDs Biomex and they've had a lot of delays and a lot of last minute changes, which is why I've often said, all right, it's about to start, it's about to happen. And then yeah, I give another email saying, oh no, actually it's going to be pushed back further. Pretty much I've got all but one, I've got all these panels here and I could sit down and do a big comparison with them, but the thing is the Biomex won the 2019 panel and it's a big panel. I mean it's what I use and have been using and have been recommending, it's a big player in the space. So if I went out and do a big comparison, it takes a lot of work, there's been a lot of time already invested into this and then like a whole week or two of filming and testing. If I do all that, package it up, publish it and then Platinum can now come out with a brand new panel like that day or as I'm working through that process, it's just, yeah, it's going to make everything redundant pretty much. I mean, I hope you know what I'm saying, like it's, yeah, so pretty much we're just waiting. We're waiting for Platinum to announce and release their new Biomex French. When will that happen? Yeah, I don't know. The dates I have been given previously have come and gone and yeah, they often have been pushed back or they've been pushed back more times than not now. So I really don't know. I expect this year and hey, you know, Murphy's Law, chances are I'll publish this and I'll give them a hand. Here we go. Because I do have all these brand new generation panels sitting here, I was like, well, we might as well do something with them. So what I've started doing is reviewing them one by one. I test them for a couple of weeks. I get all my tools out, my spectrometer, EMF meter, a radiance meter and all that sort of stuff and I am testing them and I'm putting together some pretty neat reviews. Some of these reviews are quite long, you know, 30, 40 minutes. So I'm doing both the short version and the full and depth version. I've done that for a couple of panels now. So if you haven't seen them, I'll put links to them below. So that's the new Lightpart LED pulse panel. It's the Relight Rising Advantage 1500, there's a minor red light review and I've got more coming down the pipeline as well. So again, be sure to subscribe for those and if you haven't seen those videos, check them out. Now, the idea behind those videos is that if you are looking at one particular product, you can watch that video and get a lot of data from it, right? But also it will allow you to compare on your own. Before I do this big comparison series, you'll be able to look at the review of one and the metrics I'm putting out and compare it with another one. But to make it even easier so you don't have to sit through hours upon hours of videos, what I'm doing is I'm putting all the data in one big Excel document and I'm going to... It's going to be nice and easy to use so you'll be able to go in straight away and see, hey, like what's the power output of the Midaread and compare it to say the red light rising. And I'm going to put everything in there, the EMF data, shipping, warranty, return policies, price of course, and I've also got some of my own calculations, especially around value and power output. So you'll be able to see some really interesting data and that will help you make an informed decision and know where you should or shouldn't spend your money. So if you want to get access to that, it's going to be available over at alexfigus.com and a blog article called My Red Light Therapy Buyer's Guide. Now I'm going to put a link to that below. At the time of recording, I've only tested a few panels so the spreadsheet is looking a little bit bare, but every week or every two weeks I'm going to be testing more and more panels. I'm even going to go back and test all my old ones.  I'm going to put it all in there. So if you do have a Gen 1 Juve, for instance, and you're wondering if you should upgrade to the Juve version three or the, I don't know, Midaread, Midaread Pro, for instance, you're about to compare the specs and say, oh, it's a big jump in power or it's, you know, got these features. So that will all be over at alexfigus.com. Again, I'll put a link below. It will be, or you can just Google, alexfigus, redlighttherapybuyersguy. Now, in that blog article, not only will you be able to access this spreadsheet that's going to be consistently updated, I'm also going to break everything down to help you figure out what's redlight therapy panel or device is best for you. So I'm going to help you with things such as wavelength, treatment area, power output, price, budget, support, value, all these sort of metrics. They're all going to be in there and to help you get your hit around, you know, all things redlight therapy. And for those of you who don't want to go into all the data and do much thinking, I'm going to break down all my recommendations. So best panel under $1,000, best panel for power, best panel for multi-wavelength, all those sort of things. So you can just go in and say, all right, this is my goal. This is my objective. This is what I want. OK, Alex recommends this panel. So that will all be over there at alexfigus.com. And like I said, I'll be updating that every few weeks when I review more, more panels and devices. So pretty much I highly recommend you head over to alexfigus.com. Jump on my email list because I'll be announcing that blog to my email list. But also subscribe here on my YouTube channel because on my YouTube channel, I'm going to be publishing more and more reviews. And then every few months or so, I'll do an updated video on my buyer's guide as well, because there's going to be so much data, so many new panels coming in. And I've got literally like dozens. I think it's like 30 or 40 odd panels and devices to test. So that Excel document is going to be full of data, which is great if you're into that sort of thing. If not, you just want to know, hey, what what product should I buy? Again, head over to that buyer's guide and you should be able to find pretty quickly what is best for your goals and budget. Otherwise, you can always leave a comment below and ask me in the comments section, I'll do my best. And for those of you who are waiting for my 2021 comparison series, like I said, it will be coming soon. I hope anyway, I really hope will be asking. But as soon as I got there by Max, I'm going to start recording because I've prepped everything. Everything's good to go. So we'll get it out as soon as we can. In the meantime, if there's anything that does confuse you about red light therapy or it does confuse you about or you get confused when it comes to buying a panel, you don't know what to look for or something. If there's sticky points, please leave them in the comments below because I'll be sure to incorporate all of those ideas and issues into my buyer's guide. Because like I said, I will be updating that all the time as more information comes in and as I learn more and as I test more panels. But anyway, that's all for now. And again, I apologize for the delays, but I hope in the meantime, you're finding some value in the content I am putting out and the dedicated reviews.


What's up everybody, Jason here for jasperyscasebrakes.com. We just sold out a little two-box mixture of 1920 panini mosaic fast break plus team all pick your team number five number two box break as well And again, we're giving away the pelicans in this break if you buy two or more teams So how to get entered just by two teams you buy two teams you get one entry by four teams You get two entries by six teams you get three entries remember if you only buy one team you will not be entered to win the pelicans, but Since we sold this out tonight guys I was gonna give away a hundred fifty dollars in break credit, and we did so bam So at the end of the break we'll take all the customer names that bought in Aranimize them and customer number one just one winner of a hundred fifty bucks just like that winner take all Remember no veteran commons are gonna ship except for Luca LeBron and Giannis So there you go guys. So here's the customer name. Just give away the pelicans now guys So let's out besides the customer names and we'll be about two or more teams you have a chance to win And eight so that's four entries for Andy. I'm also gonna post up another one of these if you guys want to run it back Branching just brought the Grizzlies, but good luck man. You still have a chance to win the break credit Brian Owens with two There's one Daniel to Eric with two Jonathan Matt L to to Mike just with the risen last one module and you got Nancy with two That's one entry and then Nick Iguelas and then Oliver with two. So that's one entry. So I had a couple customers just get one So out of the possible 14 we're getting 13 entries So there you go, boom Let's roll it and we got a three and a six for nine times number customer number one gets the pelicans One two three four five six seven eight nine Andy had the odds Six and a three three to six nine times and boom Andy at the top. Congratulations. All right, yes So I got box a fast break box of team all That's actually let's go with team all first actually Actually one set guys. I took up on the Instagram site the fast break boxes We don't have any more actually I see a couple goals. All right. Good luck everybody You got Zach Levine and an Eric Pashall. How about that rookie gold color match for the Golden State Wars Got Keldon Johnson You got Ricky Rubio Kendrick Nunn LeBron James and a gold James Harden Rockets Lucas Amonich how about a rookie Kendrick Nunn red NBA debut Miami Heat with that one. That's Daniel Steven Adams Kevin Porter Jr You got Mason Plumlee. How about a Lucas Amonich red? spurs When an Eric and Keel Alexander Walker Clay Thompson quandari weather spoon for the spurs Jimmy Butler. You got Rui Hachimora. You got Grayson Allen Rosdaigus Zach Collins and Brandon Clark Tom Brady superfan So my next box guys fast break. So after this we're gonna do some prison team all Then after that guys It's open for about at least another hour at least until 10 guys. I remember I'm here always at least till 10 o'clock Of course, you guys want to try to break some more stuff. We can stay a little bit later But I said One break at a time guys. All right. Good luck guys Garland Bruno Fernando Luca Wow, how about an international man of mastery Luka Donchich? disco Looks nice Matt L with the Dallas Mavericks Was I on base not too bad to get one of those Monterey's Herald for the Clippers to 85 Andy with that one Jalen Noel Little disco Kelly Arbor Jr. Taco Disco Richard Holmes Rui Hachimora little Zion NBA debut Ricky Rubio Brandon Clark and Nicola Melly PJ Washington Jr. We got a disco rookie of Bruno Fernando We got bull bull Tyler hero NBA debut RJ Derek White Kendrick Nunn and we got John Wall. Nice That last Bob Mojo Fast break signatures. Yes, Mike. Look at that Wizards John Wall last Bob Mojo autograph very very nice That's wrong my teeth devil Jackson Hayes Carson Edwards. All right, we still some color guys Luka Patrick Ewing MVP's LeBron Water's PJ Washington Jr. Deris Garland We got a Korean color for the Lakers Blue disco at a 85 Cameron Johnson is a Roby the Andre Hunter Grant Williams We go Andrew Wiggins, Jared Colver, Talon Horn Tucker.  Disco very nice two box mixer. I must say Jordan pool Myers Leonard Romeo Langford Eric Pashall Luka Donchich Jackson Hayes Ricky and A Bobby Port is purple to 50 John Moran base and a little R. J. Barrett NBA debut. Very nice guys. Very very nice Well, do a quick little recap Remember we'll give away some break here right now as well but Very nice little mixer here two boxes. I'm gonna post another one. We can run it back at least one more time Let me quickly just penny sleeve all this just so we can get that out of the way But right now we'll randomize the customer names that bought in customer number one gets 150 bucks straight up Yeah, that was a pretty good box. I'm not sure Daniel.  But you know If you want to make sure it gets a good grade with surface wise I always want to clean it with microfiber cloth regardless even if someone breaks with gloves, you know, I mean Because it might not even be my fingerprints. It might be the people packing it We're touching it. Yeah, I saw some rookies here. The a rookies in some colors About a couple goals including that pass all color match There you go. There's that Kendrick none John Morant No, we don't do any cleaning Andy. Maybe that's something in the future. We can potentially offer but for the most part No, so that's what we tell people if you want to send it from straight from here to PSA That is no problem. But just know that, you know, we're not gonna clean it for you The partners not gonna clean it. He doesn't offer any service like that So at your own risk, of course, I think for most people, you know, it's better to get the card first do what you got to do Send it back to us so But yeah, I mean, I don't really know how to tell from the centering of cards like this Always know that there's sometimes like a little surface scratches on the back just from the printing And I want to say it looks good But there you go, man. So that's what we always tell people just remember that, you know That's something maybe down the line If we hire someone just to strictly clean the cards and we can offer that service, but I know March doesn't offer that so don't we so Always better to just get the card in your hands or give it to somebody that might clean it first and send it to us and do what you Gotta do All right, you guys Because you know surface is a big thing, you know, the card might be centered perfect everything edges But then the surface might have a little fingerprint on it or something like that Like I said regardless if it's me or it's another breaker or it's the packer whatever the case may be There's always something that the I can't see So you got to try to clean it as good as possible like a microphone Microfiber cloth and it says someone that likes to some of that knows how to do the cleaning pretty strong So there you go All righty, so let's get a little dice roller now and Let's copy and paste the names that are in this break Remember customer number one gets a hundred and fifty dollars 150 bucks. Yep roll it One and a five six times. Good luck one two three four five and six Six times Matt. Oh, there you go. Congratulations 150 dollars coming your way I'll send that code to right now at the end of the video. Appreciate you guys. Jasper's case breaks.com