Thank you all guys for coming. I wasn't expecting this many people, but that's a good sign, I guess. First, let me start a little bit by introducing myself. My name is Gonzalo Raffuls, and I do quality engineering for Red Hat in Czech Republic for its virtualization solution, which is over on the AppStream. So during this talk, I will try to go fast and make it light, it's been a long week, so to begin with, I'm going to be talking a little bit about I'm going to actually show you a simple test scenario for running PyTest on Jenkins, and we're going to have a little bit of a higher review on PyTest custom plugins and how PyTest loads custom plugins, and then we will look a little bit at the challenges faced at the time of trying to replicate these Jenkins job on your local environment. With Docker, and we will look also at Docker as a solution for doing this, and then if we have enough time, I'll make a little small live demo if it doesn't fail miserably, but yeah, okay. Now I'm not going to be talking, I'm not going to be, I don't want to intend to sweet-talk you and to start using containers. You should be aware of the security implications that comes with containerizing your applications, and we're not going to talk about containerizing Jenkins that's already been done, and it's ideal if you already have some some high overview on Jenkins, PyTest, and Docker, and also most people would think that this is about the Docker PyLibrary, which is for managing your containers through Python code, but that's not the case in in this talk. Okay. Now, so why did I decided to talk about this and how did we came up with this solution for for containerizing Jenkins? Now when I first came to Red Hat, I was given the task of replicating my Jenkins job, our testing framework on my local environment, and for doing so I was given a six-page document that it actually took me approximately two weeks to complete, and then when I realized how big of a fool's errand it was to to try and do so, plus like I also realized that most of the people on my team was was having issues when trying to replicate their environments as well, but at the same time those environments that were successfully working at some point with the constant evolution of our testing framework those setups were easily lost, so there has to be a way to to keep up with this and how it's centralized and make sure that no one had to waste two weeks of their time to set it up. Now at the time there were, we were not using virtual environments, therefore you had to install the all the requirements from the testing framework on your local machine and that could have potentially caused some dependencies, some dependency issues on the libraries that you already had installed on your machine. Now also like the biggest benefit of these was to be able to put Docker image on an internal registry that you can run on your organization so that whenever a newcomer started working just by pulling the image would be able to start debugging straight away. Now, how many of you have been using PyTest on Jenkins already? Okay, that's a good number. So there's a couple of things that you need to take into consideration at the time of trying to replicate this Jenkins environment or this job run. So first thing if the job that you're trying to replicate is PyTest, you should look at the PyTest options on your job, on the configuration of your job and here you can have all sorts of things and these parameters can be used by Jenkins in many other places during the job execution. In the case that I will show you in this sample scenario, this small sample scenario, we only have a parameter that will be used on one of the build steps that I will show you further on. Now, you might also have distributed tests and you could also have parameters for determining to which hosts you want or on which hosts you want the job to run. So that's one thing to take into consideration. Then if the job itself has any git integration and if you're actually this is the most common practice, so you probably do so. So it's important to check that all those repositories are going to be used for our local environment. So this is really important and we will actually link all those repositories to volumes on our containers. Now, finally, I would look at the build steps on the configuration of the Jenkins job and it will depend on the complexity of your testing framework, but you might have, in this case, I just have one build step where it's actually a dynamic bar script that takes one of the parameters that I defined before for choosing which tests to run and you might have additional build steps that could, for example, in our case, generate configuration files that are used by custom plugins and you need to make sure that all those steps are covered before you decide to run PyTest on your container. Now, with PyTest virtually, oh, I see, I got it. Okay. With PyTest, virtually any Python module can be registered as a plugin. So how PyTest does the plugin discovery, how it checks all the plugins that are on your testing framework, it first starts by loading all the built-in plugins that were installed with PyTest. Secondly, it looks for, it actually loads all the plugins that are registered through setup tools entry point. I will show you a small sample on the next slide where I am actually passing one plugin that is outside our testing framework through the setup tools. Third thing that it does, it pre-scans the command line for the minus p name option and it loads the specified plugin before doing the actual command line parsing of PyTest. In this case, you can also choose to deactivate already registered plugins by prefixing the minus p option. Actually, I didn't expect this to show like that, but it's minus p, no semicolon and the name of the plugin. So that's how you avoid loading plugins that perhaps are causing issues on your testing framework. Next thing that PyTest loads, it looks for all the comftest.py files as inferred by the command line indication and if you don't have any test paths specified, it uses the current directory as a path. Now, you need to note that PyTest does not find comftest.py files in deeper nested sub-directories at tool start-up. So it is usually a good idea to keep your comftest.py files in the top level of your root project. Next, it does also load recursively all the plugins specified by the PyTest underscore plugins variable that you can specify inside your comftest. So that is actually the last place where PyTest looks for custom plugins. So now if you want to make your plugin externally available, you may define what is a so-called entry point for your distribution so that PyTest finds your plugin module. Entry points are a feature that it's provided by setup tools and PyTest looks specifically for the PyTest 11. It looks for this entry point and it actually looks for this entry point and it loads all the plugins that you have defined under this entry point. Right. So now when I first started to trying to containerize these environments, I started by creating a base image with Fedora and a couple of instructions that I already knew that had to be there. For example, one of the first things that I had to do was at the time of linking my local repositories to volumes inside my container, I realized that I had to include all these repositories into my Python path. So I managed to do that by including the Mv instruction on the Docker file for creating the image and later on I started trying to install all the requirements that were defined on our testing framework requirements.txt file inside the root directory of the testing framework and it was then when I realized that some of the requirements that I was trying to install with PEEP had had header files from RPMs that were missing or were not included on any place on my testing framework because all these dependencies were either already included in the operating system that was being used for running these tests or being provisioned by Foreman or other services that were invisible to my testing framework. Also, how I managed to see which plugins were loaded by PyTest at every point while I was trying to containerize these was by using the trace config option from PyTest which actually shows you all the plugins that are being recognized by PyTest at the time. Now, as you can see there, I have this run statement that I included on my Docker file. Now, it is recommended that you minimize the number of run commands because in that way it makes for less overhead at the time of actually running your container. So, now, when you're leaking a volume on your container to local repo directory on your machine, if you have the SE Linux policy enforced, then the container won't be able to write to this directory. And in the case that it might be possible that your tests are generating log files that are saved on that same repository. And if you have SE Linux enforced, the container will fail with a permission deny or an IBC message in the host sys log. So, to avoid that, you need to change the context of the directory, in this case the repository directory, to that tag in particular, SV sandbox file T. Now, this has become much more easier since Docker 1.7 because as you can see on the second piece of code here, I am preparing the set at the end and since Docker 1.7, this statement automatically changes the context of the directory that you are trying to link to your container. Therefore, the container will be able to write to that repository. Now, further on, at the very end of my Docker file, I define these two instructions, the entry point and the CMD. Now, the entry point, it virtually makes your container executable. How so? Because when you run your container image, the first command that the container will run will be the entry point and the CMD, it's any additional options that you would like to run with your entry point. But why is it separated like this? It's because the CMD is overwriteable. You can overwrite that by adding options to your run statement for when you want to run your container and therefore it makes it more flexible. Now, at the time of debugging your PyTests, you might want to, I'm not sure, I use IPDB for example, but unless you use PDB or PyTests on PDB, you won't be able to debug because PyTests is capturing the standard output, so you need to run PyTests with the minus S option so that you are able to use any other debuggers apart from PDB or PyTests PDB. So, I went through that real fast.  You can find the testing framework that we're going to be using on this live demo on this repository and the second repository contains the Docker files and the Docker compose file that I am using for instantiating these Jenkins environment. So, I'm not sure if I'm able to see here, but yeah, cool. So, this is a Jenkins instance that I have running on a container, of course, in my local machine. And this job in particular is a parameterized job that takes only one parameter and based on the input that we put there, it will run one test or the other as we saw on the bash script before. So, I'll run a bad test and a good test. One should fail, the other one shouldn't. Most likely both will fail, but... Okay, so, we got a failing test here and the other one, it's success. So, let's say I want to debug this failing test that is running on my Jenkins instance. Now, I already created a container image that will replicate my Jenkins instance and a container that I can use for debugging my byte test locally. Now, I have some scripts here. So, if I run this script for running this bad test, I can show you what does this file contain. And it's basically a run instruction for Docker to run with the FC24 container that I created with this Docker file. And basically I am telling straight away to byte test which test to run. So, now in theory, if I... This is the testing framework. So, in theory, if I edit my bad test to include this breakpoint, then I should be able now to run my container and I should be able to hit that breakpoint. And, yeah, there you see. So, and you can... Because that actually doesn't... It actually... It actually hits the breakpoint when an exception is raised. So, it actually stops the execution whenever a trace bug is raised. Now, you might have no failing tests that you will still want to debug. So, okay. So, let me show you... I got another script here as well. And let me show you... I have defined on my comf test, I have defined a small plugin, a small custom plugin that is in the form of a mark. And I am using that on my good file. So, basically this mark is for including an option name on the byte test run that will look for any tests that are tagged with whatever I introduce on my byte test run. So, if I go here... Now, so if I run the good file now, this file in fact has... As you can see, I am passing the minus E option and the good tab that was defined on my marker for that test in particular. So, if I try to run this container, and I change the label to whatever, it will actually skip the test because it's not the label for good that we had defined on that test. Now, additionally, we can see with the trace config, this is also just running byte test with the trace config. On the trace config, you will see all the planes that have been registered by byte test, as well as the comf tests that have been loaded by... So, if I do this, you can see... In this case, it's showing me that it has loaded the comf test from the root directory, which is on the container itself because I am routing my volumes... I am linking my volumes to the main root folder on my container. So, that's pretty much it. So, yeah, that's pretty much it. Now, if you have any questions, I'm afraid we don't have any more time. Do we? Okay, yeah, sure. Sorry, man. For this container in particular? Uh-huh, okay. Yeah, sure. So, this is the one for the Jenkins instance that it's actually picking up the base image from Docker Hub, this is the Jenkins latest, and I include some other instructions for what I needed at the time. Then you get the Jenkins slave Docker file, which is just the Java container, base container, and there I am downloading the Java agent from Jenkins to Jenkins for executing the file then. So, you can find this on GitHub. There, you can find the Docker files there. You had entry points and stuff. Where are they? Ah, well, for this talk in particular, I didn't use any entry points or CMD, but it's like that, it's just like you can only use CMD once, and the entry point should be your default command to be executed with Docker. Hello, thank you for the presentation, it was very nice setup. Sorry, I cannot hear you. Thank you for the presentation, that was a nice setup. I have two questions. First of all, this looks like one of the benefits of this might be able to run multiple tests from Jenkins at the same time. Is this something that you've tried and it works properly with the setup? Yeah, you can run any test that you have on your testing framework if you already have it set up like that on your Jenkins instance. So this Jenkins instance was just queuing them because that's how Jenkins was set up? Yeah, this Jenkins instance was just to show the two different places like the Jenkins job that is running the byte test and my local byte test run that will be able to debug the test. Okay, cool. My next question, can you tell me what that shell you had, it showed whether the git was dirty or clean? It looked very nice, I liked it. It's all my set SH with some tweaks, yeah. Okay, thank you. Hello, thank you for the talk, it's very important. Well, my question is like this, when it comes in my understanding personally, when it comes to deploying your application, I can understand the benefits of using Docker. There are a lot of advantages like encapsulating all of the businesses inside your containers so you can ship it as is or make your application cloud native ready, something like stuff like this. But talking about testing only, what is the real advantage of using the containerized solution with respect to other available solutions that in my opinion might be more lightweight solutions like virtual environment? So have you considered this other solution? What is your opinion on this? As I said before, given the complexity of our testing framework, this was the best solution at the time. And as I said also, I don't want to preach or sweet-talk you into using Docker or containers for replicating your Jenkins instance, but this is actually saving us a lot of time for newcomers and also it's maintaining some sort of uniformity on our testing framework. Do we have any more questions? No. Thank you very much Gonzalo once more.


Good morning folks. Steve Rhodes coming to live from the shores of a very pleasant Delray Beach, Florida. It's your 11 a.m. update. We've got a sea of green out there. All the U. S indices that we track. They're trading to the upside dollars up 88 points about a quarter of a cent. Six cents for the S and P or 32 points. One and a quarter percent for the one on Nasdaq 100 210 points there for tense for the Russell. That's an eight point move. Semis are up two and a quarter percent. 99 points there. Trendy's up 33. Gold's off $10. What's up with that? You've got the U. S dollar index back 562 pips as we speak. Silver's up 39 cents. Lights recruit up a buck 78 natural gas up 18 cents. Third of your treasury is flat trade out at one nineteen twenty seven. Let's figure out what all that means after I get rid of this leaf or something to my throat. Not a leaf. Must have been a little dust particle or something. Let's go take a look at nine panel. Market update chart. We begin with the E. S. many got an A to B equal CDT upside. Initial price projection forty nine forty four. That retracement about a fifty seven percent retracement in price does not generate a bearish reversal candle soon. We'll see fifty ten as the price objective price right now trading below its fifty day expense moving average that gives fires the edge out there. Nasdaq it's already past its one to one A to B equal CDT level that was at seventeen five seventy nine its next price projection level seventeen nine seventeen. We take a look at U. S. dollar index yesterday we thought it was breaking out but that descending trend line. Are those descending trend lines held. But now what price is also done is pull back this morning. It's found support. So we know we got resistance that is descending trend line. We also know we have support at the top of that daily profile. That means watch one two sixty one. As long as that gold is going to a gold if the U. S. dollar now so this is suggesting that the U. S. dollar index will pull back right. Well we're going to go take a look at the U. S. dollar index during the traders edge show. So we'll get a better feel for that. Goldy locks has made it be equal CDT to the downside. Initial price projection. Nineteen eighty six and change. Silver's got made to be equal CDT to the downside. Initial price projection twenty one thirty four. Right now it's over just consolidating with inside its profile. That's between twenty two thirty seven where the sport is at twenty three twenty one. Where resistance is that in the case of light speed crude. Nice rally running right into profile resistance sets at the top of a new profile. That resistance levels at seventy five forty six a price can close above that. Then we are headed higher. If we take a look at natural gas it did test and reject its swing point. This was a teeny nine count bottom swing point from back here in December that was December the 13th out here. We should rally further. Here's the caution sign with regard to natural gas. There's a new daily profile that is forming above price. That's a bearish message out there. That says natural gas probably hasn't given us the ultimate bottom. Folks they do for the trader's end show. But if you have to start your Wednesday have a wonderful one thanks for joining us. We'll look forward to speaking with you again soon. Take care now.


Welcome to the Executive Office of the Director. The mission of the California Governor's Office of Emergency Services is to protect lives and property, build capabilities, and support our communities for a resilient California. Accomplishing this mission requires a large and diverse team of individuals with a wide array of experience and expertise. The Executive Office of the Director includes offices that contribute toward and support the California Governor's Office of Emergency Services and our vision to be the leader in emergency management and homeland security through dedicated service to all. We begin with the Office of Civil Rights and Equal Employment Opportunity, otherwise known as EEO. The EEO office provides proactive training and consultation to Cal OES employees to ensure a workplace free of discrimination, harassment, and retaliation. EEO also oversees the EEO compliant process and is committed to the appreciation of diversity and inclusion. The EEO office also supports the grants department by ensuring grants sub-recipients are compliant with EEO regulations. The Office of Legal Affairs advises and advocates for Cal OES programs and administration to navigate through legalities associated with emergency services, homeland security, public safety communications, and all phases of disaster management. The Office of Legislative and External Affairs represents Cal OES before departments, the legislature, Congress, and various community groups on legislative issues, track state and federal legislation, and prepares briefing documents for the director's speaking engagements or major events. The tribal coordinator improves and maintains communication and collaboration and assists with informed decision-making between Cal OES and all Native American tribes in California. They also educate internal and external partners on the cultural settings of California Native Americans, their emergency management and homeland security issues, and cultural sensitivities. The Office of Private Sector Non-Governmental Organization Coordination designs, coordinates, and implements statewide outreach programs to foster relationships with businesses, associations, companies, and universities, as well as nonprofit, nongovernmental, and philanthropic organizations. It also works with Cal OES to maximize the inclusion and effective use of private sector, philanthropic, and NGO staff and resources in all phases of emergency management. The Office of Access and Functional Needs identifies the needs of people with disabilities before, during, and after a disaster, and integrates disability needs and resources into all aspects of emergency management systems and planning. It also provides support and technical assistance to other state agencies, local governments, and public and private partners in planning and response, and serves as a liaison for the disability communities and Cal OES. The Office of Crisis Communication and Media Relations provides the media and public with information about Cal OES, its programs, policies, and projects. It also conducts Cal OES's disaster preparedness campaign and other public events, as well as providing comprehensive crisis communication training. It also manages the Joint Information Center, or JIC, during large-scale emergencies. Internal Audits does exactly what you would expect. It performs audits on Cal OES's internal operations to ensure effective and efficient operations, compliance with state and federal rules, laws, and regulations, and the reliability of Cal OES's financial reports. Audits also provides a central point of contact for all audits and reviews conducted by federal and other state agencies, coordinated through the Audit Chief. The Office also investigates complaints of improper governmental activities against Cal OES employees or any entity entrusted with grant funds administered by Cal OES, including fraud, waste, abuse, or misappropriation of state or federal funds. Finally, the Office of State Military Department Liaison provides expertise in the full spectrum of military support capabilities to Cal OES's senior leadership, in order to effectively coordinate and integrate defense civil support capabilities across all emergency management functions. Want more information on the Office of Executive and how each branch contributes to our mission? Visit caloes.ca.gov and click on the Cal OES Divisions link.