My name is Rachel Perry and I am the founder and owner of Strategic Data Analytics. I've been performing data analysis work for close to 20 years and seven of which are the most recent seven have been for legal aid organizations. I created the content for and will therefore be introducing you to the data analysis framework along with Scott Friday, who is our web designer. Just a little bit of background, the data analysis framework was an idea generated during a TIG project a couple years ago that was a partnership between the Legal Aid Society of Cleveland, Montana Legal Services Association, Strategic Data Analytics, the Northeast Ohio Data Collaborative, and Cleveland State University. During that project we worked on significant analyses of vulnerable populations in the greater Cleveland area and in the state of Montana for the two legal aids that serve those areas. Our analyses revealed patterns, some of which were expected and others which piqued interest, raised questions, and generated discussion regarding what the data revealed and how that new information could be used to improve client service effectiveness and efficiency. The analyses were fruitful for those two legal aids but we wanted to make data analysis more accessible to a broader audience. So we developed an outline of the data analysis framework. Then starting last year under a new TIG project, that's a partnership between NJP Strategic Data Analytics and Scott Friday Designs. We have actually created the data analysis framework site and all of its contents. Scott's going to talk to us a little bit about some of the technical details and then we'll get back to the content of the site. Thanks Rachel. So hi, my name is Scott Friday. I'm the developer on this project and I wanted to just quickly go over some technical details about the site. So for the back end we are using Drupal 8 and we're utilizing the new built-in REST API features that are available. And this is basically a way for external sources to interact and grab content from your Drupal site. So for example with this kind of setup you have the freedom to build the front end of your site with pretty much anything you'd like. As long as it can interact with a REST API which pretty much everything can. So for our front end we are using HTML5 with the AngularJS JavaScript framework. And with this kind of setup it's a little bit different than the usual Drupal site, but I think it has a lot of benefits. And if you are thinking of upgrading or moving to Drupal 8 soon, I would definitely recommend looking into the REST API features. So Drupal.org has some great documentation that can get you started, but I have some tips and advice that can save you probably a lot of time and headaches. So feel free to get in touch and I'd be happy to share those. And thanks and back over to Rachel. Alright, thanks Scott. So I'm going to go ahead and take us to the site. Alright, so this is the home page. And for anyone who wants to follow along actually on the site, I hope you can see the site up there. But I'll read it out. It's HTTPS forward slash forward slash daf.lsntap.org. And that will get you to the home page here. Alright, so just sort of some high level stuff about what the data analysis framework is. So it provides analysis instructions to help organizations answer five high level questions and or 118 more detailed data sub questions about eligible people, their legal needs and services provided. So hopefully when we when you go to the start site and start looking through, you'll see data questions that other legal aids have posed that might be similar to questions that you have. And it'll help walk you through how to gather data and then do the analysis to answer those questions. It's simple to use so organizations that are just beginning to look at their data will find it helpful. But it also offers a level of detail that will make it helpful to more data savvy organizations that are more interested in more sophisticated analysis. The instructions include guidance on what internal and or external data are required in order to be able to answer the data questions and undertake the analysis. And every example shown uses actual legal aid data. So scrolling down still here on the home page, there's a section called things to think about. And there are, I think, five sort of things that we think are important enough that we highlighted them here on the home page. And those are watching for data patterns, running every data, finding by staff, how to deal with difficult data, data integrity, and factors that can skew your data. So in each one of these categories, we have examples that are actual examples from legal aid organizations. So we'll just look at one to give you a sense of what we're talking about, the kinds of examples that might be helpful to you. So this is a dealing with difficult data section. And there's this first example here refers to uncollected data. So in this particular example, it was data about formerly incarcerated persons. Legal aid was curious about those individuals, but for policy reasons had decided not to ask clients about their previous incarceration. So that legal aid had to rely on external data sources with information about the reentry population. So that's just one of the types of examples that you can find in here. And I would encourage anyone who's starting out on data analysis project to scroll through these to read them to see if they provide any meaningful guidance for you as you get started. All right. So when you're ready to start actually beginning using the framework, you can get to the framework in a couple of different ways. But we're going to go ahead and start by clicking on this Begin Exploring button. And this takes you to sort of the heart of the site. It describes the four main sections, the first being the data questions up here at the top, the next being the analysis types. And there are data resources and partners. And then finally the data analysis framework itself. We'll quickly go through each of the sections so you know what's in there. Let me scroll back up. All right. So at the top are links to the five main data questions, which will take you to then those more detailed sub-questions and then ultimately to the analyses. So the questions are who is eligible. This refers to the poverty population in your service area. And generally you'll be looking at external data to get information about who is eligible. The next is who requests assistance. This generally refers to the folks who come to you requesting assistance. So intakes is sort of our main source of data there, internal intake information. Then who do we help? That refers to the clients who you are actually able to serve. How do we help? Refers to the level of service you were able to provide. So did you provide brief service or were you able to give more extended service? And then what resources are required? Generally there are of course lots of resources that are required in the work that you all do. But generally the data that we're looking at here has to do with the amount of hours that are worked on particular types of cases and for particular types of clients. So just to get you a sense of where these buttons take you, I'm just going to click on the who is eligible question. And you'll see then that there is a list of sub-questions that we've come up with again with input from various legal aid organizations. I'm not going to go through these in detail because we'll look back at them again shortly. So I'm just going to click back on the data analysis framework and get us back to sort of our home base here. Then the next section goes through the types of analyses, the examples of which are included in the site. So for example, a snapshot measures counts or percentages for a given period, usually the most recently completed year. If any of the counts or percentages are unexpected, a comparison, trend, or spatial analysis might be necessary to better understand the reasons for those unexpected results. And the other types of analyses that we are incorporating here are comparisons, trends, geographic distribution, and geographic concentration. And I won't go through all of them in detail. I think in general most folks probably understand that a comparison is going to be comparing to or more variables. Trend is going to be looking at changes over time. And then just to clarify the difference between the geographic distribution and the geographic concentration, I will read over those. So the geographic distribution analyses show how people or problems or anything else of interest is distributed across service areas, which can be divided into smaller areas to reveal spatial patterns. So we're looking at sort of where do people with particular legal issues live? How are they distributed across our region? Then the geographic concentration compares the concentrations of multiple variables to determine how the variables and location impact each other. So in other words, if in a particular geographic area the poverty population is concentrated in one area, we would probably expect that most clients would come from that area. So then if you look at your data and see that the majority of clients don't come from that area, that will reveal a difference in those concentrations that might be worth investigating in more detail. And then scrolling down just a bit more to talk to you about some of the other resources that are here on this home base page, there's information about data resources. So we'll just click there. And there are two links here. One is related to internal data, and it provides a list of sort of the basic case data fields and client data fields that you might want to start pretty much any analyses with. And then if we look at the external data sources, there's a lot of information here, and it's a lot of detail, and that's purposeful. Because we know sometimes it can be intimidating to figure out where to go when you're looking at a site like the US Census American Community Survey. And so there's information, well, there's a link here to the site that will get you the data, recommendations about which tables in terms of the five-year, three-year, or one-year estimates. And then if you scroll down, there are vulnerable population examples and recommended tables from the ACS. So again, it can be intimidating to figure out which tables to look at. And so here's a list based on various populations. And then there's also in each section a link to a site within the Census that talks about that particular population and various issues that they've identified in the broader national data. So hopefully that helps with figuring out where to go in that site. And then there are, down here, these links I think are worth highlighting. There are some other entities that already aggregate and analyze data, and it might be worth looking at some of the national examples. And then also some state and local examples. We have just a few examples listed here, but there probably are similar sites around wherever you all are stationed. So let's go back. And the final, oh, sorry, the next resource is partnerships. And this is a table that was actually put together for us by a social scientist that worked with us on the original project. And so he put together a list of 628 potential university social science research partners from around the country, and it's sorted by state. It'll probably be a little bit difficult to view here, but you can download it and it's sorted by state. So you can scroll down and find a potential social service partner in your area. Okay, so finally let's scroll down to the actual framework itself. This is the visual depiction of the structure of the tool, and it can be a point of entry. You can click, if you're interested in a snapshot of who is eligible, you can click here and be taken right to the instructions and the analysis. So the way that it's set up is that the types of analyses are listed in the colorful boxes on the left, the snapshot, comparison, trend, geographic distribution, and geographic concentration. And then the high-level questions are listed across the top, who is eligible, who requests assistance, who do we help, how do we help, and what resources are required. So again, you can click on any example, and if you don't know exactly which type of analysis would be most effective, I would urge you to click on multiple examples to see if they shed some light on which analysis. This might be relevant for the questions that you have. So we're going to actually, though, scroll back up to the top and enter via these questions. My assumption is that once you've used the framework a few times, you would probably go straight down to this colorful matrix. But I think when you're first starting using this tool, it's helpful to start up here with the questions. All right, so let's start with who is eligible, and we'll start looking at some examples. So we saw this a few minutes ago when I clicked over here to show the sub-questions, and I will just highlight a few of them. So some of the sub-questions that you might have, how many people are eligible? What are the demographics of the people who are eligible? Then let's see, how is the number of eligible people changed? How have the demographics of eligible people changed? Where do eligible people live? Those sorts of sub-questions. And so let's go ahead, and let me just highlight, though, that there are many sub-questions listed in each section, but of course this is really just the beginning and designed to prompt you to identify the detailed data questions that you might have. So of course there are many more questions that we could include, and the many more that you will have, I'm sure. So let's just go ahead and say we're just getting started, and we're interested in having a snapshot of who is eligible. So we would click here, and for every example you'll be taken to a page that looks like this. You will see the data questions across the top, so that allows you to switch easily between the different questions if you decide that your data question maybe is in one of these other categories. Likewise, you'll see the different types of analyses on the left, so you could easily switch between different types of analyses. And just to remind you, the sub-questions for that particular section appear over here as well. You'll see a description of the type of analysis up here at the top to remind you about what it is that you're looking at. And then every example is set up the same. It starts with an example data question. In this instance, the question is how many people with specific demographic characteristics in this example, seniors who are 65 or older, are eligible for our services. Then the next section is multiple analyses are possible, such as what proportion of our services, of our service areas poverty population includes seniors, what percentage of our areas seniors are in poverty. These numbers can be compared to internal data regarding the percentage of our clients who are seniors, and should that match the portion of the poverty population who are seniors. So these are just some ideas to get the juices flowing about what you might be able to ask and answer. Then you scroll down and there are data sources. And so in this particular example, we're just looking at external data and it identifies the specific tables that we would recommend using. But in some of the examples, you'll see both external and internal sources recommended. Then the next section is the instructions on the steps to do the analysis. This particular one shows some helpful screenshots for downloading data from the American Community Survey site. And it tells you in detailed instructions how to get at the data. Then after explaining how to get at that external data, there are tips about what to do once you download the data to Excel. Now we don't provide detailed instructions about how to use Excel. We hope that most folks know how to use Excel, but it didn't feel like the right venue for describing detailed steps about using Excel. But none of the examples here are terribly complicated. So this provides you just an example that was created with that external census data taken into Excel with a table and some pie charts created. And every example that we have on the site will show a sample visual like this. So in this particular example, the analysis looks at the share of the poverty population in Montana that is made up of seniors and also the percentage of seniors who are in poverty. So those are two different views of poverty for seniors. On the left, this shows that among everyone in poverty in Montana, 14% were seniors. Then on the right, this shows that amongst all of the seniors in Montana, 31% are below poverty. So thinking back about some of the questions that we looked at, a follow-up question might be that does the fact that one-third of seniors are in poverty make that particular population a high priority for this organization? That would be obviously something for them to answer internally. So now let's just say you're interested in understanding your eligible population, but you want to look at a spatial analysis. You can scroll back up to the top and click on this different type of analysis here, the geographic distribution. But I actually want, again, because we're just starting looking at the site, I'm going to actually go back here and enter it from the question again just to walk you through this. So if we started out saying we're interested in who is eligible and wanted to look at the geographic distribution, this is the route we would do, which would show us some of the sub-questions. But again, the other way is to get to the types of analysis is to click on these boxes on the left. Okay. So in this example, it's the same exact structure that we saw in the last one, but our example data question and analysis are different. So in this case, we're asking what is the distribution of our eligible population throughout our service area? And there are multiple analyses possible, such as where are the areas with high poverty and low poverty rates? Where are the areas with high and low numbers of people in poverty? Keeping in mind that looking at numbers and rates can result in very different patterns of poverty. You can look at broad trends by county or municipality or trends closer to the neighborhood level. Are the patterns of poverty as you expected? Or are there any surprises? So were there any pockets of poverty in your service area that popped up that you didn't expect? And then again, this shows a recommended American Community Survey site, I'm sorry, table. And it doesn't go into the screenshots of the last one did, but you can look back at that last example to look at those screenshots. So we have a couple different example analyses here. The first one, example one, does the distribution of eligible population vary by community? And in this example, one of our social service partners from a local university used GIS mapping to look at the data for Northeast Ohio. This is actually one of the analyses from that original TIG project. He used ArcGIS. It's a mapping software that most legal aids don't have, so the instructions here are very high level. The idea being that an example like this could be shown to a social service partner in your area with ArcGIS capabilities and that person would know how to replicate this type of analysis. There are other examples that we show where you don't have to have that kind of software and you can still do some spatial analysis. So we'll get to those. The map that you see here shows where poverty rates are highest. So the dark brown is where poverty rates are the highest and lowest. The more yellow areas are the areas of lowest poverty. The dark area in the center, go right around here, is the city of Cleveland. And then the dark areas to the far east are more rural. So those were pretty expected concentrations of poverty. So then the second example does the distribution of eligible population vary by community. And then this one is using the built-in mapping feature that's in the American Fact Finder site. So again, this is an example where you don't have to have fancy spatial software. You can just do this right from their site and their instructions here on how to create a map like the one that you see here, which, similar to what we just saw, but with obviously slightly less detail, this is a map showing the poverty rates by county in the state of Wyoming. So it's clearly not nearly as powerful and doesn't have the same kind of flexibility as GIS mapping, but this is something that's accessible to everyone. All right, so now let's say you're satisfied with what you know about the eligible population. And I will just remind you, though, that there are multiple types of analyses you can look at for who is eligible. But let's say now you're interested in knowing more about who requests assistance. So obviously, compared to the who is eligible section, we have many more sub-questions here for each of the different types of analyses. And they ask many of the same kinds of things in terms of the volume of people and their demographics. But in this instance, we're honing in not on a different population. We're honing in on the folks who actually request assistance from your legal aid. Whereas the who is eligible, we were looking at who in your community is eligible for your services. We're going to hone in on the trend section here. And so we'll just look at some of the sub-questions. How has the number of people requesting assistance changed? How the demographics of people requesting assistance changed? How have the number of people requesting assistance from your defined groups? So that might be a vulnerable population that you've identified as a high priority for the organization. How has the number of people requesting assistance within certain categories of legal problems changed? So that might be housing cases or family cases, for example. So let's look at an example trend analysis set up the same way. And a reminder that the description of the type of analysis is right here at the top of the screen. In this instance, the example data question is how has the number of people requesting assistance from our organization changed over time? Multiple analyses are possible. How do the trends in intake numbers compare to the trends in eligible people over time? What have been the annual changes in intakes compared to the annual changes in the eligible population over time? What percentage of all eligible people have requested assistance in each of the last five years? So in this instance, this is one where we talk about using internal data and external data. This S-17-01 poverty table is one that we use in a lot of analyses. But then it also spells out for you what type of intake data from your case management system would be helpful. And so it's some pretty basic stuff, but the demographics about what you are curious, the open and close date, other case information that might be useful, including problem code, other demographics, et cetera. And a note to exclude cases that were identified as errors or duplicates, but to make sure to keep cases that ended up not being served. And that's so that you can do a comparison of those who you're able to serve and those who you're not able to serve. So for this example, we'll go down to the example analysis steps, which again are the instructions. And so there are some basic Excel instructions and the instructions regarding downloading the ACS data. And then there is a specific recommendation here to create a combination chart, which is a type of chart in Excel. And for this particular image, we're looking at the changes in intakes and changes in the eligible population over the course of five years. The intakes being the green bars and the eligible population being the blue line. And so you can see that there was an increase in intakes and a bit of a dip. This particular legal aid knew that that had to do with some resource and staffing issues. And then as is the case in most areas, the eligible population was on the rise. At the bottom of the page is another example of an intakes dashboard and again intakes being who requests assistance. The example uses an open source program called Microsoft Power BI that can easily use data from Excel, among other sources, to create really nice visuals, including multiple types of analyses. I actually presented a tutorial on this program at the, at last year's, the January 2017 TIG conference. And would be happy to send that to anyone if they're interested. I really recommend that as a, this tool as a resource for conducting analyses. It's very easy to use and create some nice visuals. So let's scroll back up and take a look at another question. And that's who do we help? Again, multiple sub questions. And for this example, we're going to scroll down to the geographic concentration area. And that's the example we're going to look at. And so the questions, the sub questions in that particular area are things like, where do the people that we helped or were able to serve, where do they live? Where do the people serve with different demographics live? Where do the people served from your different defined groups live or different legal problems? So let's go ahead and click, I'm sorry, I'm going to scroll back. The, actually the example that I, but I really wanted to show you was the geographic concentration. And so let's actually look at those sub questions. Does the geographic concentration of intakes match the concentration of serve people? Does the geographic concentration of intakes match the concentration of serve people with specific demographics? Does the geographic concentration of intakes match the concentration of serve people from your defined groups with specific legal problems? So again, we're sort of always looking for what do we expect to be true? And then what does the data actually tell us is true? So now we'll click on the geographic concentration. And so we're going to, again, compare the concentrations of different variables to determine how, you know, determine if there are things that are, that require some further understanding so that we can focus in on where potentially things such as unmet, there are peaks of unmet need geographically and those sorts of things. There is a link that provides more information about location quotients. And so if you wanted to get more details about that, you could link here. But in a nutshell, what we want to do is to compare the share of the poverty population by county to the share of the clients we serve. So for example, what would it tell us if in one county, if one county had 10% of our service area's poverty population but only 5% of the clients we serve? So the location quotients which are calculated in this example compare those percentages and determine whether the service provided is lower than expected, close to expected, or higher than expected. So for this particular example, the question that we're looking at is how does the proportion of people we serve from different parts of our service area compared to the proportion of eligible people in those same areas? And this lays out that we're looking at closed case data and provides some recommendations of the kinds of fields that you would want to include. And then the recommendation about the external ACS table to use. So this example actually uses, there's lots of steps here, but we will end up with this map. And this example actually uses Microsoft Power BI. And there are lots of detailed steps here because I know most folks are not familiar with that tool compared to Excel. And so, and then likewise because location quotients are new to folks, there are pretty detailed descriptions here. So even though it looks like a lot of steps, it's actually pretty easy to do. I'm not going to walk you through word for word, but basically you gather external data about the share of the total poverty population that resides in each county. And then you gather data about the share of all the closed and served cases by county. You divide the percentage of served cases by the percentage share of the poverty population for each county. And you end up with ranges of numbers that range from, in this instance, 0.1 to 3. And let me just scroll down so you can see the image. That's hard to get on the whole screen. So this is again comparing the concentration of served cases in the state of New Mexico to the concentration of the poverty population. So the green areas are the ones that ended up with numbers of 1.26 to 3. That number itself is not as important again. It's dividing those proportions. But because it's above 1, it's telling us that we are serving more people in the green areas than would be expected if the only thing we were looking at is the share of those counties share of the poverty population. So for example, if in this county over here, if this county over here has 5% of the poverty population but 10% of the cases that we serve, that's the reason for it being green. We're serving at a higher rate than would be expected. Of course there are going to be things that come into play, policy decisions and priorities that may skew these things. And so when you see something that's more than expected or less than expected where you'll really want to dig in and try to understand is if you don't have an explanation already. There isn't a priority for the organization that leads to that more than expected or less than expected category. So again the yellow is where the level of service or the volume of service is almost equal to the rate of poverty in that area. And the red is where the poverty population is a higher share than the share of the folks that we're serving. And so sometimes that might indicate, you know, you can do this at all different geographic levels. You could do this at smaller geographic levels as well. And some folks have used this kind of understanding to say, all right, well in those red areas we might need to do a little bit of additional outreach or something along those lines.  And so they were omitted from the analysis because those numbers were too low to it might be skewed data. All right, so let's look and I should highlight again that this is again a spatial analysis that does not require fancy geographic software like ArcGIS. It's that free open source Microsoft Power BI that was used. So let's scroll back up and go to look at another question, how do we help? So how do we help is really getting at again the levels of service? Are we providing brief or limited service? Or are we able to provide more extended service? And this time we'll look at a type of analysis we haven't looked at yet a comparison and look at the sub questions that are things like how does the level of service differ by legal problem among different demographics. So there are some legal problems that we know are that require more extensive service and some that can be resolved with with more brief service. But there might be some fluctuations in that and some patterns that weren't expected that are worth looking at. And so basically that's the focus of this set of questions and this example is looking at comparing the levels of service to see if expected results arise. And so the comparison analysis review linkages between two or more variables and then cover information about client conditions and data relationships. When unexpected data relationships are discovered investigation is warranted to better understand those linkages and determine whether they indicate the need for client service or advocacy work that simultaneously targets multiple conditions at once. And just a reminder again about the structure of the site we've got our sub questions here and our links to the other types of analyses and links to the other types of data questions across the top. The example data question here is how do levels of service differ by legal problem among different racial categories? And there's just a note there that because many legal aids count Hispanic as a race rather than ethnicity it is considered a racial category in this example analysis. There are multiple analyses possible and some examples listed there. The first one which legal problem codes have the highest percentage of extended service cases. And the example here is do you know why and this is based on actual legal data? Do you know why 55% of clients with problem code 73 food stamps receive extended service while just 6% of clients with 63 private landlord tenant cases receive extended service? So that was a question for that legal aid to answer. And then another type of analysis and examples within problem code groupings to clients with different races have similar percentages of extended service. So for example, do you know why 60% of Hispanic clients with food stamp cases received extended service while 38% of white non-Hispanic clients with food stamp cases received extended service? These analyses are based on intake and close case data and in this case don't include external data. So again there are these example analysis steps which are like the instructions to conduct an analysis like this. And this table shows how to review the levels of service. So this column here is brief and this column is extended. These are the problem codes and then these are the racial categories. So the results that we felt warranted additional review are highlighted in either yellow or red circles. And so similar to the examples I mentioned above, in this instance when we're looking at the food stamps, the food stamps cases here in the sort of the second category, we can see that, and I called attention to this earlier, 60% of Hispanic clients received extended service and the 38% of the white clients. And so again we would want to understand the difference, you know, were there language issues or something else involved that meant that some folks needed additional time or what was the issue there. And then if we look at the red circles, we can see that 55% of all food stamp clients received extended service and again this example I called out before, the 6% of the landlord tenant cases. So even just a simple table like this can, when you start comparing the results across problem codes or across racial categories or across levels of service, you can uncover some interesting relationships that might warrant further investigation. So we'll look at one last example with the last question, which are what resources are required. And so I think I mentioned at the beginning that there are of course lots of resources required to serve clients, but we focus here on data around hours spent. And we'll look at the one type of analysis we haven't looked at yet, which is the geographic distribution and the sub questions there are for cases in which geographic, for cases in which, that isn't worded correctly. We'll have to edit that wording. So we want to know where hours are spent and another example of questions and what areas are the most staff and pro bono hours worked. All right, so the data question here is, do the average hours per case that we spend on cases vary by client zip codes? We consider multiple analyses are certain types of cases that require more time or less time coming from specific geographic areas. Do we tend to spend more time on cases for clients who live near our offices? Those sorts of things. And we used internal closed case data. Now this example, which also has lots of detailed steps, is using another free open source software called Cardo. And again, because it's one that I'm assuming most folks aren't familiar with, that's the reason for the length of the instructions. But just like with the Microsoft Power BI tool, I think it's pretty easy to use and this provides you with real step-by-step instructions on how to use it. So let's look at the visual. So what we're showing here is the average hours per closed case in 2016 by zip code. And that's for zip codes with 10 or more cases. So the darker the blue, the more hours spent per case. And then all the way over to the yellow, those are the fewer number of hours spent. So this organization would want to dig in further to understand how other things might be impacting these results, such as the distribution of particular types of legal problems. So our legal problems that require more time are they focused also down there where we see more hours, down in the southeast corner of the state. And then likewise, our legal problem codes that we would expect to require fewer hours are they skewed up in the northwest section. And if not, then what might be causing that difference? So we have reached the end of the examples. I want to just pop this up. So we've walked through today just six example analyses from the data analysis framework, but there are actually 25 different analyses combining various data questions and various types of analyses. Every example is different and provides unique instructions and unique visuals. And as you saw, there are different types of tools used from Excel, Microsoft Power BI, Carto, and then ArcGIS. It is our hope that the guide will help you figure out ways to analyze your data and data about your community and in ways that will allow you to serve your clients as effectively as possible. Thank you, and I'm happy to answer any questions. And of course, our contact information is there if you want to reach out to us individually. Yeah, definitely. If you have questions over that, please feel free to reach out to us or to ask questions generally on the LSM TAP listserv. I know there was a lot of information that was covered here today. Rachel, out of each of the tools that you talked about and the examples that you gave, if someone is doing this for the first time, should they dive in with one of the more complicated tools or how would you recommend someone get started? We've got a few people here that this is going to be their first project. Yeah, and that's a great question because the intention here is to try to make this kind of analysis less daunting for folks who have not done it before, but then obviously we saw some examples that did include some heavy lifting in terms of analyses. And so for folks who have more experience, they can find what they need here too. But I would say if you're just starting out, and I know a lot of folks are, and it can be overwhelming to figure out where to start, I would start noodling around just clicking on these different types of data questions and looking at those sub-questions to see if any of them, when you read through, are any of these things that you've wondered for your organization. And to just check out the different types of analyses, examples, and what you might do is scroll down first to the visual and then see if you can tell what it's showing you. And then if it's something you're interested in, you can go back up and look at what data you need and how to create something like this. And let me show you how easy it is to go between the different questions and the different types of analyses. So you can scroll across the top here to get at all of the questions. And then if you click on one of the analyses, you can scroll through, and again there's all these different examples here, you can scroll through. And so I think what I would suggest is just spending some time clicking through and reading things to see what sparks your interest. And I do think that these questions are in probably the order that I would recommend, which is to look first at who is eligible, although if you're not comfortable accessing external data, which is what this question is getting at because you're looking at community data, maybe jump right over to who requests assistance, intakes, who do we help, cases served, and how do we help, levels of service. Those three might be the first place to start with internal data. But more than anything, I think poking around, again, all of these questions were generated by legal aid organizations, the high level questions, and then those sub-questions. And then all of the examples that you see are using real legal aid data. So next question here is, can you tell us a little bit more about MOBI as a tool? Oh, is it MOBI? I wonder if they were referring to the Microsoft Power BI tool that I mentioned? It could be very likely because the Microsoft Power BI is free at this point, right? Yes, it is. And I think it's a really fantastic tool, especially for folks who are not as comfortable with creating their own types of charts and graphs and getting them to look lovely. So let me get to an example that shows that tool. So it's wonderful because you can download data to Excel, either internal case management data or external census data, or both. And you can link Microsoft Power BI to those data. And then with just a few clicks, you can... Here, let me get to this one. With just a few clicks, you can create these different types of charts and graphs. And it might be kind of awkward and clunky if I tried to pull up the program right now just for a live demonstration. But if whoever asked that question wants to reach out to me later, I could certainly do that with them. But it's really fun to play around with, too, because you can say, I think I want to look at this in a pie chart. And just with a click of a button, you click on the pie chart option. It creates it for you.  That isn't giving me the kind of data. I actually really need trend data over time. Then you just simply click on the trend data button or bar chart data, and it'll just change it for you. And so it's really, really easy to use. And again, I'm happy to share that presentation that I did back at the January TIG on that particular software. I'm also happy to talk to folks offline about it. That would be great. And there was also a request here to possibly do an online training over that. And we will definitely look at that for next year's training. Yeah, I'd be happy to do that. Yeah. And there's a question, is it better than Excel charts? Well, it probably depends on your level of experience with Excel. If you're really advanced and know how to create pivot tables and pivot charts on your own, and you'd like to have a little bit more control, then go for it. But if you're just starting out or just aren't as comfortable, this is, I think, really easy to use and really a lot more efficient. Because I think those of us who have used Excel can attest to the fact that sometimes you get sort of buried in the minutiae of trying to make it look exactly how you'd like it to look and what's the best way to depict different information. And I feel like Microsoft Power BI kind of does some of that for you. So I would say if you're starting, well, even if you're really comfortable with Excel, I would say dip your toe in the water and test it and see what you think. But then, again, for beginners, I think it's great, easy to use. That looks like all we've got at this point. I will also remind people that we have done some past trainings in Excel. They're archived on our YouTube channel. They don't specifically cover this, although when we do trainings for next year, we will definitely consider that. Any parting words? I guess the parting word would just be that I know that it can be intimidating sometimes to try to figure out how to tackle the amount of data that you all have. But I would urge you to just jump in and start playing around with some of these things because it can provide you with such important insight about your clients and their needs and how well you're serving them. And it really doesn't have to be hard. I hope that you'll find all of these examples and instructions easy to follow and replicate with your own data. And I would just say if there's any technical questions, feel free to email me.


Welcome traders to our technical charity with me Patrick and we want to draw your attention to an opportunity here that I'm tracking the Kiwi Swiss from the day-to-day time frame in training this channel. We're looking for an 11 swing corrected pattern so we have 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 so we're looking for an 11 swing loader so I'm looking to bearish reversal patterns against the trend line resistance here, 64-70 to set short positions targeting a move down into the 62-37 monthly range support and projected descending trend line support. From there watch for bullish reversal patterns to re-engage on the long side targeting a break of the trend channel and move up into monthly range resistance at 65-40 or so, weekly range resistance at 65-10. As always, trade us, plan the trade, trade the plan the most important and manage your risk. Until next time, thanks very much.


Good afternoon, everyone, and welcome to the Stock Swoosh Show Market Review. This is the QQQs. Market is playing out the way that I thought it would when I looked at it yesterday, except for it has not done the buy set up yet on the daily chart. It looks like it's going to, though. It's only $12.30. It's three and a half hours left on the day, and the market is going to trigger a buy set up on the daily chart over $9,506. It's $0.30 away. Market's doing it. Market's power-treading all day here since it decided to lift. The interesting thing is in the pre-market this morning I thought, well, this market's going to hold. It's going to be bullish. And then it took a little while to get going. I thought, hmm, this is interesting, because the spine looked like it was going to break down again and it was weak. And then it faked like it was going to go lower, but flipped. And then I was wondering if it was going to fake to go higher, but then it held, and the Qs always held this morning. So the spine was looking weaker than the Qs, and they were going to, and markets, two markets needed to come together. And they did. They came together in the direction of the trend of the market, of the overall market, which is bullish. So could the market have come in today? Yes. The spine was weaker than the Qs. Did they know? The market is playing out today, right now, today, intraday, the way that I saw it to play out today last night, and the way I saw it to play out today in the pre-market. Despite the sluggishness, the sluggish activity that happened in the morning, which happened for a good 30, 45 minutes, the market did hold today and went going. Stop going in here today, because I'm sure a lot of people made an attempt to short this market, but you cannot do it. You can't do it. And I've been saying this a couple of videos ago, I said, don't short this market no matter what you do. And I'm sure people shorted this market two days ago, shorted people shorted this market here. I'm sure people shorted this market this morning. Look at this. Look at this. Look! Market over 95 today, wherever it goes. Follow through here into the close of the week. Follow through into the close of the week. The gap down to me was a gap down. Was it a gap down? I would have shorted? No. Did I short this market? No. Can you short this market? No. You cannot. There is no such thing as tops. There is no such thing as bottoms. There is no such thing as anything like that, and stocks don't have to do corrections. Neither does the market. I talked about this in the video I did last night. This day here that you wanted to come in, if you wanted to correction the market, that was it. That's all you're going to get. Go long. Could have gone long here. You could have gone long here this morning. You could have gone long here today. People that want to buy this market and understand what's going on are seeing this as an opportunity to buy and to support, and that's what's happening. So the market is getting bought into support, and that's what happened today. Market gap down into support and got bought, and that's what's happening. This is a huge bar to be happening today in a gap down. Massive, massive retracement back up here, and a great, great hold for this bullish market. It will continue higher, and the market will get over the high, if not tomorrow in the next week. It is a great, great area for this market to be doing what it's doing, and the bottom line is that if you want to go long and core long, if you're into going long, if you want to go on the market, this is an opportunity. The market hasn't even gone anywhere yet. The market hasn't gotten going anywhere, anywhere, anywhere at all yet. The market is going to have a huge massive rally between now and the end of the year, and if people don't get in, they're going to miss it. So there will be no correction. You want to call the bar that happened two days ago, correction, fine, that's it. That's all you're going to get, and a gap down today, basically. And this is really, really something here. Look at the nice lift this is getting. Just what a huge tower of strength, huge tower of strength. So I saw the market would do this yesterday, and I saw it this morning in the pre-market. It took a while to get going. It took a while to get going. It was tricky in here, because I know people were shorting this, and the market was rallying over the high, came down, didn't break the low, looked like it was breaking lower, then came over the high again, and finally, finally got going. After 10 o'clock, but here it goes, market right up, right up to the target of 95. Right up there, it's just going to go pooh, it's just going to go right up there, and it's 12.30 in the market still, so much time left in the day. And I'm just going to throw this out there now, and I'm seeing it now by the time of the day. Market could get over the high today. This is nuts. Market could get over the high today. I see this number. I'm just reading this live right now, 95, 70. Market could get over the high today. The law in the market, any pullback here today, if you want to go long, if you want to trade, market could actually get over the high today. I'm not saying this is definitely going to happen. It's definitely going to get to the target of 95. But I'm throwing it out there now. It's a potential market to actually get over the high today. It's another dollar from here. And when I did the video yesterday, actually, when I did the video last night, I said, no matter what this market does tomorrow, no matter how it gaps, if it gaps up, if it gaps neutral, if it gaps down a little, it gaps up a little, it gaps up a lot, it gaps down a lot, it gaps down whatever it does. I said the video yesterday, no matter what this market, however it gaps tomorrow, the market's along. I did say that in the video last night. I said it in the video last night. And that's what happened. Market gap down this morning and was along. So this is Melissa with the stockswush.com. Have a great day, everybody. Buy the market, any pullback if you want to go long today. First target's 95. We'll get there. That's not a dream target. It's just regular. And the market could get up over the high today, which is 95.70. And if it doesn't today, it's probably going to tomorrow. So beautiful calls I've been making here in the market. Go long, no reason to short. Anything remotely in this QQQs or the spies, you find weak stocks with good bearish golden gaps to short. And that's what you do. Email me at Melissa at the stockswush.com if you'd like more information on how to read gaps and learn how to trade and learn how to read the market. The next gap class is this week in July 12th and 13th. Email me at Melissa at the stockswush.com if you want to sign up. Thanks, everybody. Have a great day.


This war wasn't just provoked. It was provoked deliberately. In an interesting speech about the way U.S. imperial aggression provokes violence around the world, anti-war commentator Scott Horton made reference to an April 2022 article from Yahoo News that had previously escaped my attention. The article is titled, In closer ties to Ukraine, U.S. officials long saw promise and peril, and it features named and unnamed veterans of the U.S. intelligence cartels saying that long before the February 2022 invasion, they were fully aware that the U.S. had provoked, that's in quotes, Russia in Ukraine and created a powder keg situation that would likely lead to war. By last summer, meaning the summer of 2021, the baseline view of most U.S. intelligence community analysts was that Russia felt sufficiently provoked over Ukraine that some unknown trigger could set off an attack by Moscow. A former CIA official told Yahoo News' Zach Dorfman, who adds, the CIA and the Office of the Director of National Intelligence declined to comment. Dorfman writes that initial support provided to Ukraine during the Obama administration had been calibrated to avoid aggravating Moscow, but that, partially spurred by Congress, as well as the Trump administration, which was more willing to be aggressive on weapons transfers to Kiev, overt U.S. military support for Ukraine grew over time, and with it the risk of a deadly Russian response, some CIA officials believed at the time. Policymakers would always say, if we do X thing, if we give the Ukrainians X system, how are the Russians going to react? And our answer would always be, you can't look at any one thing in isolation, the unnamed former CIA official told Yahoo News. And we might look and say, well, it's just a few hundred man pads, or a few Humvees, but it's missing the point that the Russians are taking all of this stuff in the aggregate, and they're drawing this picture of this ever increasing relationship between the U.S. and Ukraine. I understand the moral argument, says former CIA official Jeffrey Edmonds regarding the weapons transfers into Ukraine, but I also understand the argument that, well, why would you want to give these things if it's just going to increase the chances that Russia does something? So while we members of the public were blindly speculating about whether or not Russia would attack Ukraine, the U.S. intelligence cartel was fully aware that the U.S. was taking actions ensuring that that would happen. That's the environment the U.S. security state knew it was operating under when it continued to taunt the idea of adding Ukraine and Georgia to NATO right up until the final moments before the invasion. This war wasn't just provoked, it was knowingly provoked. Off-ramp after off-ramp was sped past by the U.S. war machine at 100 miles an hour on its beeline toward a horrific proxy war, because Empire managers had calculated that such a war would serve U.S. interests. And now we routinely see U.S. officials like Mitch McConnell openly saying that this war serves U.S. interests. They really couldn't be more obvious about it if they tried. It's been funny to watch the response of Empire apologists to NATO Secretary General Jen Stoltenberg's surprising refutation of a year and a half of Empire propaganda by openly admitting that NATO expansion provoked the invasion of Ukraine and acknowledging that NATO powers rejected Moscow's proposed compromises which could have averted the war. Basically the only argument they now have after this admission is to say that Russia should not have viewed NATO expansion as an existential threat. Their only remaining trick is to argue with reality, to basically say that yes, it's reality that NATO expansion provoked this war because Moscow saw it as a threat, but reality shouldn't have been what reality was. They argue that Russia should have felt completely different feelings about a military threat on its borders than nations like the United States would feel. Since as we've discussed previously the last time there was a credible military threat near the U.S. border, the U.S. responded so aggressively that the world almost ended. That's really all they've got. Yes it's true that all the people who've died and lost their homes in this war did so because we were amassing a hostile military alliance near Russia's border, but in our defense the Russians should have thought different thoughts in their heads than the ones we ourselves would think about a hostile military threat on our border. If all Westerners deeply understood all the suffering and danger that has been unleashed on our world by this war, and deeply understood the fact that their own governments played a role in starting it, the political status quo of the Western world would be impossible to maintain. Which is why such unprecedented levels of propaganda and internet censorship have gone into preventing Westerners from coming to such an understanding. Westerners were deceived into supporting yet another evil war, which once again is showing every sign of dragging on for the foreseeable future with no exit strategy in sight. The only difference between this war and all those other wars is that this one is laden with the risk of nuclear annihilation, a risk which the U.S. empire has been treading less and less carefully around as the bloodshed continues. The more you think about it, the more horrifying it gets.


Hi, everybody. Thank you for joining. I hope everyone ate. They're not hungry. They're ready to hear about some cool Awesome automating stuff. My name is Lauren Santiago. I'm an assistant administrator for the identity and access management team at Red Hat. I work in IT there Today, I'm going to talk to you guys about how I got started with Ansible Ansible tower infrastructure setup at Red Hat IT The on-call process automation that we set up with Ansible and Ansible tower We're gonna talk a little bit about Nagios event handlers and Ansible tower and at the end have some time for Q&A So how I got started with Ansible, I started as an intern at Red Hat and when they converted me to a full-time employee, I Didn't have very much Linux experience So I wanted to prove myself and I was volunteering to do everything that would come up And one of those opportunities was release engineer and they already were using Ansible for their releases So that's how I first started using it and then over time I started training as Release lead which allowed me to start editing playbooks, troubleshooting issues, helping people create their own. I worked with upgrading Ansible, upgrading the code So I just started getting more hands on with it and that's how I got really in Ansible I've been using it about three years now and then my favorite thing to do with Ansible is Help train somebody who's never used it and automate their first task or something that they've always done manually and show Them how to get it automated schedule it and Bring them into using Ansible as well So for the Ansible tower infrastructure setup at Red Hat We we have more than two data centers there, but With Ansible tower we have a staging environment, which is testing But we have our main one in Phoenix 2 and then we have our DR in RDU 2 Both of them are behind an F5 there There's three tower nodes and in our primary data center for Ansible tower I have a postgres database that's clustered So there's a active and a passive which we could Use a playbook to cut over and switch it anytime we want if we need to which also helps when The DBAs are upgrading and doing different things so we can prevent outage Because we use Ansible tower for something like self-hilling with the on-call process We created a DR environment because we didn't want to ever have any downtime So that one has a secondary Postgres node that is passive until we cut over to DR, which we use a playbook to make that one active the Cut over to DR right now with a lot of the applications there is manual not all of them But with Ansible tower it is I mean we use playbooks But if someone has to run each playbook to get it set up that way But it's just so we don't have any outages so it's always running and available for us So some of the steps that were automated with the on-call process Well, this is all of them. So a Service alerts on a host that's broken Nagio says our monitoring system. It's monitoring it So when a host is broken in or the service is alerting it will then call the Ansible tower API and Then so initially what that playbook will do well It'll call a job set run a playbook initially. It'll set five minutes of downtime. It does this to let Nagios Actually run the playbook or make the call the tower to run the playbook and give it time to fix the host Then once the playbook finishes an alert clears we have a blog that's creative. It's a production system in our Document documentation space in jive This is so the service owner as well as the on-call team knows that a Playbook was ran against a host and that it was broken It's also Nelson IRC to the on-call team and the service owner about what her hosts alerted and what job was ran and what fixed it and then an email is sent and IRC message is sent if it fails which then Nagios then pages the on-call person as well So we have it set up like that just where it used to be the on-call person would have to Acknowledge the alert themselves Then go open the documentation space find the playbook that the service owner Provides and then manually follow the steps to fix the host then they'd manually have to create the blog and So if this just makes it where none of them have to do it So I just want to include some pictures so you guys could see this was Some jobs where you could see Where I mean it's running at three or four in the morning where there is someone on call 24-7 but they didn't actually have to get paged or get online to fix it or anything like that it all was fixed so The on-call team and the service owners can look at the jobs that were ran They also have the mojo blogs and then they're also implementing code now where Attracts with the CMDB where if a host keeps alerting or it alerts multiple times within a certain time period It'll open a ticket and service now which then it'll go into service owners queue because They don't want hosts to keep You don't want a production host to keep restarting services every day Something obviously more is wrong and they want it to be looked into fixed all that so that's how everyone's more aware of what's going on Here's an example of the blog posts Right now This is just like one of the first basic ones where you can see it announces a playbook was ran by tower on the host the date the time It gives a link to the tower job status They're adding right now in the inventory files Variables to the service owners so they can also be tagged so it's not where they have to go look at it They'll get an email or notification that their host was broken So some configuration infrastructure management about red hat just the touch base of some context of what I'm talking about So puppet is our main Configuration tool that we use there Nagios is our monitoring tool. How many of you guys in here use Nagios as your monitoring tool? And so what engine they use for releases ad hoc repair commands They use it for building hosts on VMs Ansible towers used for the cell filling DB configuration And then they're also moving release process and then a lot of teams and IT have scheduled jobs in tower of daily tasks that they want completed Before this integration between Nagios and Ansible tower did not exist So we developed our own and open sourced it so I can I'll talk about that more later For those of you who don't use Nagios or don't know Nagios is an open source Monitoring tool for computer systems. It's built for Linux. It can be used on other operating systems those periodic checks of services applications networking It can be agentless and it uses the agent it uses in our PE agent It's just a little bit of background Just so you guys can kind of understand I'm sure everyone has at least a monitoring service it may be different, but they're all pretty similar just with different benefits This is also an example from Nagios. So so you can see what a host monitor by now This looks like the local host over there would be the host there's default checks that come that Nagios will monitor and then service owners can configure Custom checks for certain services either to make sure it's running if they want to have a script ran and make sure it returns A certain result they can do that And then they can determine what gives back an okay status and what gives the critical status Which would have determined what's paged or what caused the Ansible tower API to run the playbook Sanary monitoring workflow for Nagios so right now if a service is okay It will check every five minutes against that if it comes back critical it would Check again and but a minute later Then it checks again a minute later and it checks again once it comes back critical the fourth time It would alert the on-call person Then what they would have to do is what I've seen before They would have to acknowledge the alert to silence it and then that person would have to go look for documentation Which is in a separate place in the documentation space, and then they'd have to perform the necessary actions That's provided by the service owners And then once it fixes it it goes back to okay and Nagios continues to check five minutes So with the workflow with Ansible tower Nagios, it's a little different So we still does the five-minute check every time when it comes back critical it triggers the event handler But it doesn't run the script to call the Ansible tower API because we don't want to we were trying to prevent fossil alarms Which before it would check two or three times before it would page the person on call We don't want it to actually page the person on call So on the third check is when the API handler makes the call to Ansible tower that triggers the job Sets the downtime and again the downtime is set so Nagios doesn't keep triggering the event handler or page the person on call and it gives Ansible time to Run the playbook hopefully fix the host clear it and if it doesn't then that's when the on-call person still paged It'll take the note out of rotation take the corrective out actions It'll put the note back into the rotation But one of the things that they were forcing the service owners was to create a check in their playbook That has to return a certain value before Ansible even put that host back into the load balancing because we don't want to put the broken host back We'd rather get the on-call person to look or get the service owner on to look at it If it's normal hours and see if we can get it fixed Then it creates the blog post about the event sends a notification to IT on call person And then Nagios is back to green and happy and checking every five minutes Um, so right now the developers deploy services they define their Nagios checks using puppet It's put in puppet modules developed by IT. It's very little code like I was saying some sorry pre-defining came it comes with Nagios they can get as Detailed as they want the service owners know their systems and what their applications need to do the most so that's why It heavily depends on them at least working with us or them coding what they wanted to do Ansible tower generates a generic host inventory From the Alyssa host minor by Nagios. This is also like a dynamic inventory that's created from them This is helpful because what the on-call team ended up doing is things like SSH or NRP something that's monitored already by Nagios They can create a check to restart the service just a generic template that if that alert So this way every single host that's mar by Nagios if one of those alert Tower will know. Oh run this playbook to fix that service So and then IT built sort of standard system repair playbooks a lot of services just need a restart that Is a really simple playbook We also created one that had like extra vars where if it's not something you could put if they have us a Host that has multiple services that sometimes a needs to be restarted sometimes be needs restarted So they created a generic playbook where or a Generic job template where it could have extra vars provided by Nagios and by the inventory from the service owner Developers are welcome to build their own repair playbooks host inventories. However, they want to do it. They could depend on Nagios There's plenty of documentation that is one of the things I will say is key to is if this is something that you do implement document everything so Service owners can also jump in and do their own and don't have to depend heavily on you, but you know like Myself and others make themselves available to help with whatever they need So to break down the Nagios event handler definition So usually it's a this is kind of like in puppet and that's how they Code that in there. So it's a command name where for example We're using the tower handler one that gets called and then it's a command line Which is usually a script which is the code Where it will now get us will call the interval tower API and then there's the state of how many attempts because By default with answer will tower we have it set to the fourth to the third But service owners can change it like hey We don't want to any action taken unless our host alerts five times unless or some sometimes they want it Action taken immediately first time in alerts. We don't care. We want it fixed There's the downtime where they set the downtime on the service the host name, which is the host that it's alerting inventory They can hard-code it in puppet if it's for sure always going to be that they can use the dynamic inventory Which comes from Nagios and then there's a limit which limits it to that host because sometimes, you know, they have Ten hosts in an inventory. They don't want to restart take out all of them So they limit it to they can pick limits where You know if host a alerts, but they want to fix a c and e they can change it to where All of those come out at once one at a time. There's different limits. They can set This is an example with the handler so the event handler is at the end and that's just where it would Call the restart service there. They're telling it to use the generic inventory in the service name, which It's it since it's not included in the playbook. That's provided in the inventory It's just like an extra bar. So Nagios is will tell Ansible tower and ansible tower played run the playbook with Ansible and I'll know to restart a ct PD for the host that's alerting and It's based off the generic inventory. So it's going to be pulled automatically from what with alerting So The code that was created for the handler to call insible tower If this is just like a simplified version So you guys get to see because I wanted it to be more of like tech tech deep dive You guys can see the code the code is not that long. It's only probably like a hundred and eighty lines But I wanted to put a more simplified version for you guys just to see like what it's doing You can see the extra bars the jobs that call in the job and the if l statement Another way they do it so we have Splunk which tracks The insible tower jobs and what's ran on what hosts we also have the the jobs that are tracked in ansible tower and Then we have the logs of ansible tower being called in agios So that's another way for the on-call team to troubleshoot tracks see if anything's happening or not happening like it's supposed to and Then it's also it's helpful when We're using the cmdb and the logs for tracking how many times a host alerts if it's alerting every other hour every other day multiple times a week for then them to trigger the job to make the Service now ticket for the team to fix their host or to at least make them aware that it's alerting because That was the one thing when we first did this we didn't want teams to not think that there anything was happening with their hosts or Not reading emails not reading blog posts because it's easy. I am from believers not everybody reads email So like it's easy to look past it and this is just a way You know now you have a ticket in your service now Q You have to look at it fix it figure out the problem And then one other step that it was taking is if they don't fix it then they're gonna stop offering the The monitoring service and then they'll be responsible for themselves So the Nagios and tower handler script like I said before it didn't exist no integration and I was a little painful but Worked with other people in IT. We got some code and that's how we got this working Worked with Ansible. It's open source now. It's on GitHub Contributions are welcome. It's there for anyone who has Nagios and is interested in this It makes it way easier. I will make sure my slides are up to to help with anything and Always available for email too if people have questions Some of the success that we had with it was over a hundred pager playbooks were automated since February of last year I think that number is even more now 50 of them are converted into a single generic service handler template Which was like restart services a lot of them were more Customized it just depending on how detailed service owners would be and work with us to get the playbooks coded 15% of production alerts are automatically handled by tower and never paged on call person now They're moving more of the application release process of Ansible engine into tower they're planning to migrate Jenkins Ansible tower for base OS image build and They're using Ansible engine for VM deployment and they're supposed to move that to tower DBAs storage team networking They all have a lot of their configuration set up in tower where they just provide inventory Push that and then they have they just go kick off a job and press a button And then they have the stack of whatever they need built Multiple teams at it began implementing their own automation needs A lot of people once they saw demos of this and how it worked a lot of people wanted to jump on board to have their stuff Included with that, but then they also were like can we automate this schedule this so there's a lot of people who didn't know what all Ansible and Ansible tower could do and then working with them to get everything set up So it's been very useful and helpful, and then a lot of people are interested in it And so now I have some time for Q&A Does anybody have questions? No, so they they change shifts so the way it works is the on-call team I'm not on that team anymore, but it's a global team. So there's three shift changes and So it's usually The u.s. Time and then after that I'll show you takes over and then after that India and Burno Those guys do but they all have they all are supposed to follow up on the blog post to see what was alerting What wasn't they all also have a handoff where they talk to each other about what what had been going on And they update each other on that and then plus they have all the automation where if something keeps alerting They had the blog post to see their communications with the service owner and the service now ticket that gets created to them from their queue I don't think I understand what you're asking No, oh, they have so they have the way they have their their duty set for on-call It's they actually have an in puppet where they have the on-call rotation. So they're getting paged. They have Email aliases that page them and then they also have IRC They have an IRC bot that pages them to that they have to always have up and open and then I Wouldn't say that's the only two ways that they get paged and then Tower will announce announce an IRC emails and do the blog post for production systems Yeah, that's why they did this to prevent from so like of course if they still get paged they have to get on but They're not getting paged as often now because it's Self-healing it's automated and then everything's still being blogged or tickets are being opened and so they still have the Responsibility of looking over it all that but it's changed instead of them getting paged at 2 o'clock in the morning and having to hop on and fix things They were fixed and They don't have to do that You're welcome. Well, so with When Nagios is calling ansible tower, it's the same Nagios user every time and so that helps with the tracking and But like a service owner can hop on if they need to use tower to restart their service and it's fine the tracking only comes from the Nagios user and So that's what would get flagged if Nagios is kicking off tower jobs for you and restarting a service every hour or every day and I mean, that's usually not expected or wanted and especially in production or any environment, but Production is where they get really flagged Well, it'll recheck it, but it won't re kick off the job It'll page the on-call person and they can look in there and see that the job still running if it is So that was one of our issues before and so that's why we said the five minutes of downtime We didn't want to set too much in case Something major is going on and we want the person to be paged if it's not fixed within five minutes And so far with all the ones that we've automated They are fixed and validated within five minutes I mean there could be the use case where it doesn't but usually if tower doesn't fix it It's a bigger issue more than just the host. It's usually More like infrastructure size something bigger is going on and then more things start going down and then We also don't want them taking all the nodes out of rotation Like if you have four we don't want you to go and complete outage because of it And so there's preventions from that too. So like if you have one or two already out It's not going to take out all four. It'll start paging on-call I mean the service owners give us that so So some service owners have complete scripts that have to They provide it so it's whatever they need like I don't understand all the applications and what they do So it's whatever they want like if they need to go in it like I don't know say they're just as full and it causes Some other issue they can create the playbook to go RM-f Some files and then run some other command and then restart services It's whatever they want to do, but they provide that to us Yes Yes, I mean so they get they get access to everything but the inventory Because we don't want them going in and messing with not their stuff Yeah, but they get access to everything else if they want to edit a playbook The only ones that they're not supposed to touch are the generic ones like service restart if they want something special outside of what's provided They can request it or they can code it themselves and then just say they need it pointed to their inventory or they could use the generic one Well, that's why They do where they create the well they do the blog post, but then they also do the service now ticket where Let's the service owner know. Hey, this is happening You need to look into your host and figure it out and fix it and then that's where if they don't fix it Then the on-call team the operations team and say we're not going to monitor your hosts anymore Kind of a way of forcing them to fix it so that's where what's in puppet is where you configure the check for Nagios and You tell it what to check against like if the service is down if it's returning a 500 or 404 Like you configure what you want it to let you know so it's all it's working with the service owner You have that in puppet Nagios knows what to look for in that aspect and then if it alerts a certain way for Nagios it knows what to call in the tower API and then You have your playbooks written in Ansible as a job templates and then it calls it there based off the inventory Could you repeat that it was really painful for on call, but they were getting a lot of alerts and I mean I used to be on that team and on-call life wasn't for me, but It was a lot of pages in the middle of the night a lot of noise they still have they have on-call rotations They still have work They have to do and it was blocking them putting them behind them not being able to complete their work so now this lesson focused on other things for ITU and operations and Infrastructure and not so much having to stop every two minutes to go look at alerts. That was the biggest issue But it also I will say some hosts or some Service owners didn't realize that their hosts were so bad until It was being tracked by Nagios and then the tower API kept kicking off playbook runs Which then brought attention to them to look deeper in the things because we were like hey This is happening every day Look into that and then it helped them realize they were bigger issues that they had to address So that also helped in different ways too Not many but it is possible like a little network blip or something where Nagios doesn't have that connection at that second and Then it checks again five minutes later, and it's clear then that's usually what it what it's there for The Nagios checks set up like that were before I was even there But that's what when I was talking to the SME for Nagios He said it was set up that way just in case it was something minor like that I've never seen where it's been a false positive. It was usually always a legit thing, but that's why So they can easily open it when I was doing this project I was working with them hands-on all the way because we were trying as many people automated it in this and Out of where on call was having to manually do it Now that I'm not that person anymore they can still open a ticket for consultation for the SMEs and they can help them They can if they have someone who knows Ansible Yeah, they can do it themselves But if they don't they will work with them because we want it automated and we want it to be easier for them Yeah, it's kind of like but I mean it's worth helping them for an hour If then you don't have to do it anymore. Well, thank you


So I'll have my coffee this morning. I realized there was a tweet from Tavis Ormandy and Whenever he tweets it's usually something pretty interesting, especially when it's in regards to last pass He has long been poking away at last pass, which is one of the reasons I like the product Yes, a flaw was found. Yes, the flaw was fixed So don't worry you're safe as long as you're not bypassing or stopping updates from coming through to your system Including your standard Chrome updates your Chrome browser updates your Mozilla updates and your plug-in updates as long as you're not stopping them You're fine. So we'll at least clear that up right now second Why do I keep using last pass if there keeps being flaws found in it? And this is where people have a big misunderstanding of security all the time They assume because they flaw was on their product the product therefore should not be used I Prefer products not that have flaws in them, but that have security researchers like Tavis Ormandy Constantly poking at the code last pass has also gone through pretty extensive code reviews These are important aspects to any product whether it be hardware or software if it has any software on it at all That runs that hardware You want a strong security research team to do code review to constantly be checking it for bugs And you can't just assume because some new product is on the market and has not been hacked that it is secure It is not secure until it has gone through a lot of testing and even then it's whether or not those testers were able to find it So that is a really subjective thing of just how good these testers are Tavis is one of the best in the business and works for Google's project zero Now this particular bypass was a little bit interesting So last pass could leak the last use credentials due to a cache not being updated This is because when you can bypass the tab credential being Cache being populated by including the login form in an unexpected way and the code is actually a little bit interesting So the concept is that if you go to accounts like Google comm and prompted to click the password dot dot dot and fill it in And then go to example comm I know what they're doing is saying that if you have it filled in but sitting there is now cash because it's in there But you haven't logged in yet to clear the cash Then you enter the council over on another one another site for example example comm And it would be able to see the credentials that were pulled for the previous site So it's kind of there is a high severity level to this But it's also a little bit tricky to do and he didn't have at the time of recording right now a Proof of concept put up but he just figured out how to wait automatically compare these until it succeeds and Basically, what this is where it gets a little bit trickier is to take another URL like any Subdomain of the URL you were logged into to help hold those credentials forwards And then I frame them into another piece and like I said leave links to all this you can read through and This will get updated if Travis proofs post this proof of concept down here But what I really wanted to talk about you directly related to this is that whole concept of Because there's a flaw found you should stop using the product now that depends on how those companies respond There's a lot more to security than just a flaw found stop using it a flaw found company responded properly and they did Travis went through the proper disclosure method of Finding the bug creating a proof of concept, you know, basically you have to prove that this is real Submitted it to the teams over at last pass last pass looked at it go that's valid and we understand the flaw We understand how to reproduce it which is an important concept whenever you're doing bug bounties or bug hunting is being able to Reproduce the bug not having a one-off of not sure how it happened because that becomes really hard for the Programmers to fix but that proof of concept was put together to have us found it Last pass took care of it and update was issued now. It's all public disclosure and out open This is the cycle of security. This is an important aspect of it and like I said just because There's people poking at these products and then finding the flaws It's all about how these companies respond to the flaws will be the determining factor of whether or not that product stays secure And any product that doesn't have people poking at it does not mean it's secure So you could release some software today and make that bold statements never been hacked Never been a flaw found in a product But until these companies go through solid code review, they're really hard to trust This also makes it very difficult here to start new companies and Gain the user stress now There's always some people who go new product and jump on it, but I'm always very cautious on these especially when they're products that maybe Just don't have much market saturation. So there's not a lot of use There's not a lot of time that I have to poke at them that I'm really not a great pen tester It's not what I do you want the really high-end pen testers to go through do code reviews and really poke at it You want them to have a whole process over on somewhere like hacker one where they offer bug bounties to encourage Larger groups of people to poke at it and by the way when you're starting up a company, especially anything in the software realm This is part of the process now is having your code reviewed I talked about this before with keybase and they were very public about their code review took a while before they got to it But it also cost them a whole a hundred thousand dollars to do It is not arbitrary to do an extensive level of pen testing and code review on any particular product This is why I do like last pass. I I'm shocked that log me and has not destroyed them as a company So if any you were going but log me and own seven didn't they do bad things log me in is a um Not a company that has a reputation of doing wonderful products, but last pass has been one that I think they've left alone They realize it's a popular product. It has Excellent market share and with people like to have us poking at it They seem to keep their security up to date and they're still as responsive as they've ever been now For those of you don't know or why you use any password manager The password manager has to be a zero trust system last pass is a zero trust system For those of you wondering or trying to research that there's a few other ones out there like Bit wardens another one people have asked me to review and I just really haven't had the time to dig into it But all these are based on zero trust where companies like last pass do not have your password That's why all of these attacks are always occurring at the browser level because it is filling in passwords within the browser So anytime You have something filling in passwords within the browser itself because the decryption is happening on your side at your computer Because it's not happening at the last pass level That's why these attacks happen here. It's the only place they can happen They're not attacking last pass directly last pass by not having your passwords removes that piece of attack surface By going we don't have the password You got to check the browser and that's where tavis is really poking at it is within the browser And of course, this is a credential caching problem where you have filled into credentials, but haven't submitted them It's an interesting bug. Um, it's so exciting that last pass fixed this right away That makes me very happy because I see this could be exploited And we don't have any proof that this was or was not exploited in a while So we don't know no websites were found. This is just tavis working on his own research and going I wonder if and that's how a lot of security researchers work. They kind of poke at something and go I wonder if it would do this and when they find it, you know, there's a whole aha moment And then you turn into a proof of concept and a bug report So my conclusion is i'm going to still continue using last pass. I still trust it as a product I trust it more when I see people in security researchers poking really difficult at it And it's partly because of our market share.  Thanks And thank you for making it to the end of the video If you like this video, please give it a thumbs up If you'd like to see more content from the channel, hit the subscribe button and hit the bell icon If you like youtube to notify you when new videos come out If you'd like to hire us head over to laurancesystems.com fill out our contact page And let us know what we can help you with and what projects you'd like us to work together on If you want to carry on the discussion head over to forums.laurancesystems.com Where we can carry on the discussion about this video other videos or other tech topics in general Even suggestions for new videos. They're accepted right there on our forums, which are free Also, if you like to help the channel in other ways head over to our affiliate page We have a lot of great tech offers for you And once again, thanks for watching and see you next time