Rebound the stagmas are another localizing the stagmas. Note how when the patient gazes to the right, small amplitude jerk the stagmas to the right can be seen. When gaze is shifted to forward gaze, the amplitude and the stagmas stays the same, but the direction shifts to left beating. Now the patient's in left gaze, left beating the stagmas are observed. When the patient shifts gaze to forward gaze, note how the direction of the stagmas will then change to right beating. This cycle can repeat itself indefinitely as long as gaze is shifted. The localizing value of rebound the stagmas concerns its relationship to disease and disorders of the cerebellum. It can be seen in degenerations of the cerebellum, infarction, demyelination. And what we're observing now is just a continuation of the previously mentioned cycle. Again, note how the direction of the stagmas changes.


Hello and welcome to the session. In this session we are going to discuss the following question and the question says that discuss the center and spread for the following graph. Now let us start with the solution of this question. Now here we are given a graph showing curves for two different data sets. Here pink curve shows one data set and green curve shows another data set. And here we can see that the two curves are symmetric. Here the center of the pink curve will lie at the highest point and here we have seen that this is the peak point of the pink curve and its center lies at this point. And we can also see that the curve is symmetric about its mean and let us denote its mean by x1 bar. Similarly the center of the data set represented by green curve will lie at the highest point. So here we can see that the center of this curve lies at the highest point and the curve is symmetric about its mean. Let us denote its mean by x2 bar. So here we see that centers of the two data sets are different. Now we shall discuss its spread and we know that the standard deviation is the measure of how spread out the data is. It indicates the average distance from the mean for the set of values greater standard deviation shows more distance from the mean for the set of values. When we compare the two curves we see that the spread of the two curves is not so wide and here we can see that it is almost clustered about their means also we see that the spread of the two curves is same. So we can say that the two curves show same standard deviation thus we can say that the two data sets have different centers but same spread which is the required answer. This completes our session. Hope you enjoyed this session.


Okay, I think we'll go ahead and get started. Just so you guys know, I don't expect this to run the whole hour, so I might be able to get out of here early, and I know it's late, so it's not too heavy. We're not gonna be learning like statistics here, just to show you some light concepts that hopefully you'll find useful. I'm Mark Sonebaum. I'm a performance engineer at Acquia. I'm not a statistician, and I am not a data scientist. I'm just somebody who deals with data sets a lot, and the things that I'm gonna show you are generally pretty safe techniques for looking at exploring data sets without getting into statistical methods that can be much more error prone and best left to experts. Was somebody did this as if they couldn't hear me? Is that the case? Okay, good. So just a quick example of why this stuff is important. Has anyone ever heard of this data set before? Okay, great. So it's just four sets of X, Y pairs.  So the first thing somebody might do is do some summary statistics on them. Each Y and X axis for those four data sets have the exact same, X has the exact same mean, the Y axis has a very, very similar mean, standard deviation is almost the same. Those two summary statistics would suggest that these four data sets are almost identical. But when you plot them, that's what they look like. So it's a really, really great example of how if you don't look at your data and you're just using summary statistics, you may not actually know what's going on. You may be asking the wrong questions. And so that brings up the topic of exploratory data analysis. I was coined by John Tukey, I think he was 1977. He wrote a book on this. His main idea was that at the time data was being used to confirm hypotheses. But there was an issue with confirmation bias because you basically start with your hypotheses and then use the data to see if it passes or fails. And his point was more that you need to explore the data set to figure out if you're even asking the right question. Because once you explore the data set you might realize that there's questions, there's interesting questions you can ask about the data that you wouldn't have known otherwise.  If you've used something like a new plot or if you've used D3 or any of the JavaScript libraries, it's very likely that you were just treating it basically as a way to present the data. But data visualization is much more than that. It should be part of your analysis process. So the first thing that I usually look at is I figure out just if the data set is correct. I've seen results from people multiple times who didn't do this check and just showed me the results and say this doesn't make sense. And then when I go look at it, it's like, oh, well, this data is garbage. And so just having the data, like looking at the first couple rows, looking at the number of rows and make sure that it's exactly what you expect is a huge help in finding bugs. I always do this and I end up, it's pretty rare that like the first time I write out the data set is I did it the right way. I usually find something wrong with it. And so the basic process is you formulate a question that you want to ask about the data. Make a very fast, like basic plot just to make sure that everything looks right. And then hopefully that will tell you something about the data just by looking at it and then you start to refine your question and then you just iterate through this process. You also need to ask yourself can the question be answered by data? Sometimes it's unfortunate that you, you have the one data set and you may find that this is not the case. You can't actually answer it. But at that point it's a good idea to stop, to stop and either get more data and not report the results for like the limited data set you have. And because this is part of the iterative process, it needs to be very fast. The process of say like getting a CSV and then like pumping it into D3 and getting D3 exactly the way you like it that is not fast enough to support like an exploratory analysis workflow. So this quote, it's a really good one. Hadley Wickham I'll introduce later, but basic idea is that the bottleneck in this process is cognitive. And so we need to optimize the process for that, not for say like the presentation layer. And that brings us to R. R is designed with this process in mind. It's interactive. It, well actually I'll go into that in a second. You might ask yourself why I would want to learn another language because a lot of these things can be done in your existing program language. The interactivity of it is a huge help. It also has some very robust two dimensional data structures. It's very common in like the two general languages for this are R and Python, but not actually Python. It's like Python with SciPy and all the other things. And they have data tables, which I'll introduce later. It's basically like two dimensional like matrix like structures. It's like an in memory structure of a CSV type of thing. And also there's like built in graphics output. Almost, you can almost think of it as like a, like a third stream outside of like standard error and standard out. And the community is amazing. The base R language is a little bit limited, but we have what has been called like the Hadleyverse. It's like this guy Hadley Wickham, he's a data scientist that I think at Rice University and works for RStudio. He's wrote tons of packages that make the language like infinitely better to use. And usually for most tasks I'm actually just using functions exported by his packages. And so it's a, and it's vibrant too. Like when I started four years ago, half the stuff wasn't out. And so it's constantly getting better. So just quickly to cover this, my favorite way to install this is just brew install or brew cask install. I recently had somebody complain about how difficult it was to get compiled on brew, but don't try to compile it, just install the package, they're good. And this is what you see out of the box. You get a console, and then you get this other window that's the plot viewer. That can be useful, but you're probably gonna get to the point quickly where you want to save what you have and work with actual files. And so as much as I'm not generally like an IDE guy, I can say RStudio, it's a free IDE, it is excellent. It's absolutely worth using. You can, you just type your files in there like a normal editor, and it has a built-in. You can see this, here's where the code is. Here's the console. Here's some different ways to explore the data. And then all your plots get pushed over here. That's a very nice workflow, it works fine. I don't want to go through and just like tour like the whole R language, because there's a lot to show and it's kind of boring. But I'm just gonna like as we go, show you little bits so that you can understand the examples, but just some like very, very basics. The vector is like the most primitive type. It's a vector if you know a vector from any other languages, but the main idea is it's multi-value and every value in it has to be the same type. So this example of like a vector of numbers and a vector of strings. Data frame is the structure that you're going to be using most of the time. It's a two-dimensional data structure like a matrix, but every column has to be of the same type and the of equal length. So it essentially, you can like internally visualize it as like a CSV. And it's a real simple example of how to make one and then what it ends up looking like. Most of the built-in functions in R are vectorized, meaning so if I have the floor function and I give it one number, it gives me one result. And also that C function, that's just how you create vectors. If I give it essentially an array of numbers, it does that for each array. And so most of R assumes one or many. Like actually if you make like a character, say like something equals the letter A, that's actually a vector of length one. It also has functions as values, which we'll probably recognize like from JavaScript. You define functions by just like creating a function and then assigning that. And you might have noticed it has a weird assignment operator. I'm told that back in like Bell Labs, because it started based on the S language that was in Bell Labs in 1976, I think. That was apparently one key, but they just kept it. And so reading data in, you wanna get your data into a two-dimensional format. CSV or TSV works really well. I would avoid JSON because JSON is like infinitely nested and the data structure, there's a bunch of different libraries in R to read JSON, but you end up with like a pretty unwieldy object when you get done. You always have to flatten it. I'll put these slides online and I don't expect you to actually like copy down that gist, but I have a script that I use that flattens any JSON file. It essentially just takes all the nested keys and concatenates them with dots. And so I use that when I have JSON that I need to pull in. So here's just a quick example of how to do this. Here's a CSV that I got just from Varnish.dat, which obviously didn't come straight from Varnish.dat. It doesn't come out as CSV. So one thing you can do when you're starting is don't focus too hard on making R read the data format you have. Write a script in whatever language you have to create a clean CSV file and your life will be a lot easier. And so there's just like the super, super basic reading the CSV file and making a plot. This is using the base plotting system in R and that's the last time I'm all show that. It's not very interesting and there are better options now. And just to quickly show it, it also has a pretty good package for reading logs. Like basically the log format is delimited by spaces and then also respects quotes. And so I have a sample access log like this. I can use this web reader package. Just you read CLF, same for common log format and then I get a data frame back, parses everything into the right columns and then assigns the right type to it. And just some quick terminology before we go forward. In this world, you'll hear, you'll hear talk about variables and a data set. A variable is essentially a column and rows or observations. And there's essentially two types of variables. One is continuous. You'll most likely like, I think essentially the only two are numeric and date. So continuous means on an axis it could go on forever and there's no like discrete steps in between. Versus categorical or discrete. I think D3 maybe calls it like ordinal or something. That's just individual strings essentially. And I'll show examples of that. But it's good to recognize which one you have because that will affect what you're able to do with the data in terms of visualization. And so just some basic plots with those two ideas in mind. Working with this data set here. This came from a monitoring system that just checks a website from different locations. So say you just wanna look at one continuous variable. A continuous variable in this example would be like the time that it took in milliseconds. The most basic, like simple visualization for that is just a histogram. It's one of the most useful plots that you can make because it gives you a very clear vision of what the distribution of data looks like. And then I also like to add that bottom piece which is called a rug, which just shows, it also shows the distribution of data. And it can be useful if your bins and the histogram are wide. Ben you'll notice I added a bin width. By default it just divides it into 30 bins. But this is something you always wanna play with because like so the bin is basically how wide the bar is. And so you can get different insights out of the data by messing with the bin width because it's very domain specific. So you have two continuous variables. So say in this case we're dealing with response time and dates. The simplest way to do that is just a line graph. This may look a little boring. And I know like now everyone uses stuff like Grafana or Grafana or Grafana and the other one that it's based on. What is it? Cabana, yes. And so everyone immediately just wants like what's called an area graph where it's like it's dark. You have like the outline and everything under it's colored. But that conveys actually no more visual information. So I would encourage you to just like live with the boringness of these because it doesn't matter. The other way we do that is actually a simple scatterplot. And you'll see here I added alpha. Adding alpha to scatterplots is very useful because you can get a better idea for the density.  So I can know that's only actually a couple points whereas I hear it's essentially black. Meaning there's tons of points there. It's also an example like an R you can render to, I think it renders by PNG by default but you can also render to SVG. This is a plot you don't wanna render in SVG because you can end up with a million little SVG objects in your browser and you don't want that. So if you have one continuous and one categorical that's when you end up with, I don't know how to describe it. I think it's pretty obvious when you're looking at it. So you're saying I'm saying the categorical variable is the agent. And so it splits it up by agent. If I didn't add that, then it would just basically be one plot in the middle. And this is a box plot. I'll go into like what that is a little bit later. And I also didn't introduce at all this plotting system. It's GG plot. This is like the one that I like to use. It's one of the most popular. And this AES function stands for aesthetics. And so the idea there is that you're doing aesthetic mappings. And so you're saying I'm mapping the agent to the x-axis, time to the y-axis, and then I'm mapping the color aesthetic to agent. And then so it knows how to essentially just make the plot once you've given it that like semantic information. So for two continuous, one categorical, you go back to essentially the line graph but now you have a line per categorical variable that you told it to make. And this is actually very hard to read because it's all just on top of each other. And so another way, right, another way to do that or if you also have two categorical variables and you want to split it up two different ways, you can use what's called a facet. And so you can say do that same thing but actually, yeah, my slide is wrong. This is not two categoricals, it's still one categorical. It's just clearer because I'm using agent for the facet and then also for the color. But it's a nice way to just quickly break it out and get a graph per item in the list. I use this one very, very often. And so there's sort of a joke in the data science community that you spend most of your time actually like getting your data in the right format to work with and it's very, very true. I think this is one of the things that trips people up a lot because you will never get clean data sets. You will always have to do something with it before you can actually like start plotting or just start exploring it. And before when I started, I would do most of this just like in a Ruby script and just get it like pristine and import it which if you're new, that still works great. But the tools to do some of these more common things are now in R, like this tidier package which is another Hadley-Wickham package. This example of the most, one of the most common things you have to do. So if you, like that initials of Varnish as CSP we looked at before, you have time, client connections, client requests, cash hits. The value of all of those columns are essentially the same, actually with the exception of cash hit, but it's a numeric value. And if you only ever wanted to say, so you wanted to graph time and client connections, you can do that pretty easily. But if you want something like the graph I showed before where there's a line per client connection request or cash hits, then you have to rethink what the data set looks like in terms of what is a, what is a value and what is a variable. And in this case, you could argue that client connections and client requests are not actually variables, they're values. And so you can use this gather function to make wide data long. And so you turn that into a metric and value. And then whichever the value was is in the value column and then that assigns to the right metric. This is like the most, it took me a while to figure this out when I first started in R, but it's like the most common thing you'll have to do is you'll have to always make wide data longer because everyone makes wide data because it's easier to think about it. It's like it looks that way in Excel. But then as soon as you do that, you can do a really simple aesthetic mapping. So you can map color to the metric. And now I have lines per metric that I had. You can also end up just another simple example. You can also end up with stuff like, like especially performance tools might, they might output their value with say like milliseconds attached to it.  And that'll actually essentially end up as a categorical variable not continuous when milliseconds is obviously continuous. Then I can use like this extract function, give it a regex value and then it splits that out for me. And then I also retain the milliseconds unit. So if there's anything that's different, I can treat them differently based on that.  Manipulation is another huge thing you'll have to do. The dplyr package is the best option there. It actually works with more than just the data frame type. It works with databases as well. It has a very SQL like set of verbs. And so you can actually translate it to SQL and use it on a database. It is also, I'm gonna take the time to introduce this pipe operator. It's not unique to this package. It's supplied by another package, but it's used a lot in modern R, and especially the packages I'm gonna show you. I'll show you why. So say you have this line of code.  That can get a little unwieldy. So this pipe operator lets you chain them together. And so it's much easier in terms of how you think about the data flow. It's much easier to edit. You can just comment out lines instead of messing with the parens all over the place. And so all that means is take what's before me in the pipe and use that as the first argument in the next function. If you're familiar, like Lisp and pressure closure has an operator for that. It's a common thing in functional programming. So filter, the absolute most common thing you'll have to do so that dataset I had before with the different requests coming from different places, I can easily say, okay, filter where agent equals this and then also where the date time is within this range. And then just continue to pipe that to ggplot and plot it. It's also a really great way to narrow down the dataset. Like I was recently looking at a set of log data for one day and it was, I think it was like two million hits, hit the log. And that was a very big, if I did the wrong thing in the plot, my machine would just sit there and hang. And so I first do a bunch of filtering to make sure that all of my visualization is right before I plug the entire dataset in. Group by and summarizing. And you'll notice that R was created in University of Auckland, New Zealand. And so the language itself uses British spellings. That's not a typo. Yeah, you may have noticed before I think Keller, how do you? So group by, I mean, you're familiar with the concept from SQL, but here's an example, I'm not plotting here, but I'm just doing a basic summary. And I'm saying that filters, basically filter out requests that don't have a time, because some of them didn't have a time. Group by the agent, and then do a summary. And then say, in this median and quantile, these are just base R functions. And so I'm saying, do the median quantile for the 95th percentile, 99th percentile on this column. And then because I called group by before, it knows to do this only for the rows that they're in each group. And then the result is I have those four metrics for each agent. Mutate is the way to create new variables. And so it's an extremely simple example. Say I have the time was in, I think, seconds, and I wanna see in milliseconds, I can just say, okay, mutate, total time in milliseconds equals the total time column times 1,000. And now I have that new column that I can plot. There's a bunch of examples of how you could use that, but it's a little boring. And so summary statistics, this is something that I think everyone is pretty familiar with, but I think we all need to think about it a bit more because it gets misused a lot. Mean and standard deviation. Mean is average, for those of you who don't know. Standard deviation, like you may have heard people talk about having an average and then not having a standard deviation is not a good idea because it gives you no idea of how distributed the data set is. But even having the standard deviation is dangerous. So here's just a quick example of what we're usually talking about when we're talking about these metrics. So there's a normal distribution and each of these columns is a standard deviation apart from the mean. And the general assumption when you're talking about standard deviation is I think it's like the first two standard deviations before and after are 66% of the data set. The two standard deviation I think is 95% of the data set and then three, 99% of the data set should fit in there. And this example shows that that's the case in a normal distribution. Here's some latency data. This was recorded from a web service and so it's response times. The red dotted line in the middle is the average and then those blue lines represent that many standard deviations away from the mean. And in this example I think it's pretty clear that the numbers I talked about before about how many or what percent of the data fits within those standard deviations is just complete bullshit here. It does not work at all. And so the reason is this is a multimodal data set. This is not normally distributed. So normal distributed means bell curve. If you're dealing with response time data or response time data is never normal. It's often multimodal. It's often or not only not normal, it's often multimodal. This is an example of a multimodal data set. There's one giant mode here. There's also a little mode here. Which is why the mean is actually in the middle. And so if you just have that number it doesn't really tell you a whole lot about the data set. So don't use these metrics unless you absolutely know for sure that the data you're working with is normally distributed. And I can assure you if you're dealing with response time data it will never ever will be. So one better option to use is called the five number summary. This is also from John Tukey. It gives you the minimum and the maximum. The medium which is the 50th percentile. And the lower and upper quartile which is the 25th percentile and the 75th percentile. It's also much easier to reason about because you can say so for the 50th percentile of the medium 50% of this data set falls below that value. But the 75th, 75% of the data falls below that. And if I do, so the summary function in R actually gives me the five number summary plus the mean. And you can see for the mean here is 25 and the median is 12. You can see just sort of how worthless the mean is in that situation. But there's also what the box plot is. He's also the inventor of the box plot. In a box plot, the line in the middle of the box is the median. The bottom of the box is the 25th percentile. The top of the box is the 75th percentile. And the lines usually extend. There's different algorithms for how to do it. But the lines usually extend to whatever it considers. I think it's actually like the 95th percentile. And then it still plots little dots for the outliers. So you can see what the long tail looks like. I really, really like box plots for really any data set because it gives me a very quick way to see what's going on and what the distribution looks like. I can get it, I can do it with histogram, but I can't, I can't, not to fast at a histogram. I can't divide a histogram up by different categorical variables. Joel, I'll show you in a second. So if you've ever used AB, which I'm sure lots of people here have used AB, here's a screenshot of its output.  The mean, that's totally useful. The maximum, that's totally useful. It gives you mean and standard deviation. Don't ever look at these. These are, just ignore them. Pretend they're not there. The bottom part, where it shows you the percentiles. So it's showing you the 50th percentile, 66, all the way up to 100. That is the absolute, that is the only data you need. It is the most useful data. It's also much, much easier to read into it. You can say 99% of these requests were under 239 milliseconds. But 50% were under 35. These numbers do not give you a useful way to describe the data set. And it's really unfortunate. Most load testing tools will give you the average. Hopefully among other things, but the treatment of this data is pretty lacking in most tooling. So, and just to show another example of how to look at, like other good ways to look at latency data. So I showed the box plot before, but a lot of times we want to see it going over time. You can also just divide it up into bins. And so I think for this one, I'd say like every 30 seconds, give me a box plot. And it's telling me quite a bit. I can see that the distribution is like very, very tight. And then all of a sudden it goes way up. And so something there, something is happening there that I would have missed otherwise. Another way to view distribution data over time is a heat map. This one's a little hard to see. But you'll actually, you'll see like the sysdig is doing something with this now. Like detrays had a bunch of heat map stuff. It's nice because you can see that there's a time going forward on the x-axis and the response time on the y-axis. And it's sort of like a moving histogram. The intensity of the white here is how many data points fall within that range. And you can see there's also like, so right now or on the very beginning, it's really tight, it's all just white there. But then all of a sudden, there's these like little tiny gray splotches. And you can see like some requests were falling in that range, but not all of the requests. Still more requests were down here. Which again shows that this data is multi-modal. And one of the reasons for that, for latency, is there's usually always a fast path and a slow path.  And so you'll see one, like a second mode because every time garbage collection runs, it's however many seconds that was added to the response time. For something like a Drupal site, it could be however, like whatever that interval is where you missed your cache. And so every cache worm, that's another mode. And so in terms of the actual presentation side, like once you go through this process and then you get a really good understanding of the data and you want to show someone, there's also some really great tools for that. Our Markdown is a very popular format supported well in our studio. Here's an example of it. It looks essentially like GitHub flavored Markdown, but it doesn't just embed the code. Any time that code outputs something, whether that's to the console or an image, it creates, or for the image, it actually creates the image and then links that into the Markdown document. So this presentation is actually all written in our Markdown. All of the plots you saw before, I didn't place any of that there by hand. I just put the R code in, rendered it, and then they all just show up. And so it's nice because I can just write a report, put the plots in, I can choose whether or not it's going to show my code in the plot. It's usually a good idea to keep the code in there and if you're trying to create reusable research or reproducible research, you want someone else to run that and get the same results, but if I'm giving it to my manager, they don't want to see my code. And there's also a one click published to a site called RPUBS, which is maintained by the company RStudio. This makes it even nicer because I can just get it to where I want it, say publish to RPUBS, and then just give somebody a link to that and share it instantly. If you want to, you can go to RPUBS slash msonobomb and look at all of my stuff there. There's lots of R examples about how to do things. And just to quickly show, all the plotting before was ggplot and one thing you might be thinking when you're looking at it is that it's all static and we now live in a world where we're web developers, we want everything to be dynamic. We want to be able to interact with it. That's valid. It's not that conducive to the exploratory process. Actually, being able to interact with it in the exploratory process can be important. But there's this new package called ggviz, which I think will essentially, will eventually replace ggplot, but it's not quite there yet. But just to show a quick example. Is that bigger? So this looks similar to the example before. I add a histogram layer, but then I can add an input slider. Instead of giving the bin width a value, I can say input slider. And so if I run this code, right, so in the viewer here, that's actually interactive. You can see what the histogram looks like when you change the bin width. Things change quite a bit. And this is all HTML and CSS. It essentially renders to a library that uses D3 on the front end. And I can pop that out and actually view it in my browser. It is pretty mature. The only thing it doesn't have is faceting. It's the only reason I haven't switched to it. Yes, Daniel? Very, very good question. So on the back end, it uses a project called Shiny, which is a R web server technology. So it spins up a web server in the background, and then it handles the interaction between the front end and the web server. Which is very important, because if you're using, say I was working with a data set that I mentioned before, there's like a couple million lines. If you put that in the browser, your browser will crash. There's no possible way to do that. But if I do this, I can actually just have that in memory sitting on the R server and just make requests to it. And it'll still be very fast. And there's also like similar to R Pub, there's like a shinyapps.io. Once I make that, I can just publish it there and then people can interact with it. And so if you want to learn a bit more about this, there's a bunch of books on the subject, but I actually suggest too that very recently came out. Roger Pang works at Johns Hopkins University in Biosatts, and he runs like the R Coursera courses and the data science specialization. And he put two books recently on Lean Pub, which is a site where you can just like pick what you want to pay for it. And they're small, they're really to the point and they're very easy to understand. They're not written from a perspective of like knowing a bunch of statistics background. And so I'd suggest that was very good. And those are the packages. This will be online later if you want to look at the packages that I showed. And that's it. I would ask that if you can find this session node and do an evaluation, that would be awesome. And I'll take questions if anyone has any. Yes, please use the mic if you have any questions. So, don't think the mic's on. Oh, there it is. So the, are there any resources that you could point us to for that basis in statistics? I was a lowly art major. And I taught myself engineering and there's a huge hole. Oh yeah, I was a major. I'm right there with you. So, yeah, so that's a big topic. None of the things I showed really requires much statistic knowledge. When you get into things like, I basically showed, I mean, ARR has an incredible support for modeling, like forecasting, like doing linear regression, that type of thing. I didn't show any of that. Because those were, I mean, some of those were simple, like doing a linear regression is sort of simple. But that quickly gets into the realm of if you don't have like a pretty solid statistics background, you can come up with incorrect results and you won't know that they're incorrect. And it's really best left to people who know that stuff well. I would say if you want to know it better, the data science specialization on Coursera is very good. Specifically, the statistical inference course. It's also very hard. I got halfway through and I had to take a break. And it's also, I found for me, it was interesting to know and I wanted to brush up on those skills. But it's not that useful for what I do. I'm not, say, looking at a population of people and trying to find clusters and trends and doing like K-means clustering, that kind of thing. I'm only dealing with these relatively small data sets that I can look at and then make very simple inferences. They're not such a large data set that I need to create a model for. So it may not, it may be the case for you that you don't need to go that far. But I would encourage you to support and see. Yes, go ahead. I have a follow-up if nobody else does. Could you speak to your data collection process? I know you talked a fair bit about the analyzing after the fact, but presuming you're not logging and sampling all of the metrics all of the time, I'm sort of wondering what tools you use for that collection initially, to get it into the thing that you turn into the CSV. And then also, is this sort of usually something that you do mostly to diagnose a problem once you've actually observed some thing? And so are you going back to the snapshot of the outage where you already have to have that data in advance or is it usually? I'm not doing that. It's really difficult to get the kind of data. I'm showing everything as raw data here. There's no aggregation. It's very difficult to get that kind of data from production with zero aggregation. It's hard to store. It's hard to get out of the machine in an efficient way. So typically you're not gonna have that unless you're Netflix and you have one second resolution. Very uncommon though. Most of the data that I'm looking at are, I mean it would be great if the server level tools did this stuff by default, like they would output a format that I can just easily ingest. Unfortunately it's not the case. This is usually me setting up an environment, trying to recreate a production event, or just say like just doing, testing something that just needs to be benchmarked or whatever.  And then my favorite tool for monitoring and getting good data out of is DSTAT.  Like it'll read PROC VMSTAT. It'll basically read PROC for all the things that you want. And the output, you can combine essentially the output of like VMSTAT MPSTAT, IOSTAT, NETSTAT, all those things that are very useful but you can't ever see them all together. You can run DSTAT with all the flags you want and then give it an output file. And the output file will be a CSV.  The header is not like valid CSV like it actually has, it doesn't mean rows. So like the odd numbers are essentially one CSV and the even are another. I have a function that fixes it. I have like a local R package. I'll probably publish it eventually that reads in DSTAT files. But usually I'm doing that. I'm like before I would like run MPSTAT or run IOSTAT, get the most parsable output I could from it, write a script to convert that to CSV and import it. But usually I'm doing that at one second resolution because in production you'll probably have something like a 10 second resolution because you don't want that like constantly running on the machine and taking up CPU cycles. But the problem is there's a lot of issues in production that you will miss. They happen like very short CPU bursts. You'll miss them every time if you're only sampling every 10 seconds. So this process in general, I mean like I find performance work is this way but especially like this analysis process. As developers we want to automate everything. You really, really need to resist automation because if you automate things you don't let the data surprise you. It's really like antithetical to the whole exploratory data analysis process. And I find people doing this all the time where like they automate this process and they get the number back and they're like oh, like here's the number. I trust it but it looks really fishy. And I want you to actually dig into the data. It's like okay, this automation is making an assumption that it's not appropriate for this particular data set. So I do, I write a lot of throwaway code. Most of my arcs like I write that script. I don't even bother putting in source control because I'm never gonna use that again. Which is why our and those libraries I showed were so great because I don't build up a body of code that I need to save. It's like it takes me just as long to rewrite a lot of that stuff as it does for me to go find where I did it last. Does that answer your question? Yeah, absolutely. If nobody has another one, I had one last one and I promise I'm done. We can talk all day. We've done it many times. Does Grafana have any part in this process or for anybody that doesn't know, it's a tool for building graphs a lot of the time out of kind of more real-timey, constant stuff like graphite or influx DB backends where you'd feed those metrics in all the time and then you can graph them. I'm kind of wondering if R is complementary to having that. If R would point you at the things you might try to create samples for and automate. If it reveals automation that would be useful or if it's just sort of like... Antibiotical as well. They're pretty complementary.  Everything's a line graph and everything's... The x-axis is always times. Like everyone wants to see the line going over time. Problem is for a lot of data points that actually doesn't give you the data that you need. Like having heat maps, having histograms is much more useful. I'm sure you've probably seen in Grafana, it's like, okay, this line is constantly going up and down. It's very difficult to recognize that that's actually like a multimodal dataset in some cases, right? I know like Brandon Gregg at Netflix is like really pushing performance companies to have more of those types of visualizations that are like richer in tools. They'll end up in hopefully tools like Grafana. But right now things like Grafana are very much tied to Grafite back ends that are time series databases. And so I like to treat those as just two completely separate things. I mean the original, Grafana, is from Elasticsearch. Elasticsearch can actually do a lot of these aggregations. So you could have really interesting visualizations come from it, but everyone loves line graphs, so it doesn't happen. Hey thanks for the talk, that was really cool. Thank you. So here's the question. This is great for you to explore the data on your own and getting some insights about what you see and perhaps give some feedback to your people, your team. How about when you want to explore, sorry to expose this data to other people so they might be looking for other aspects or they might be looking for different aspects. I know that there is a sleuth of attempts in the Drupal community to create some modules that would play with data, charts and graphs, graphs, D3, others, what is your favorite, if you have done such a thing, what is your favorite stack? Also I know that there's a decan guy over there who has also a nice visualization module, right? So for things like that, really all you need is a CSV file at a URL. And so if I've ever done, I can't remember, but if I ever were to do that, I think I probably have, as long as that CSV file is public and then say I'll do something in R Markdown and read CSV can actually take a URL instead of a local file and so it'll pull that data set down, do the analysis, I'll publish that and say okay you can see it's pulling from the URL, here's all the code I used to generate that, you could copy this code, fork it and make your own analysis of it or you could just read that data set in directly and use whatever library or processes you want. Python has IPython notebooks, I think which is called Jupyter now, which is not exactly like R Markdown, but there are lots and lots of overlap that I hear really great things about that project.  Because R does everything in memory, so if you have data set that is maybe close to like your memory size and you might be hitting the limits or it's just big enough to where it'll be very slow in another programming language, you really want to be using R or Python, something that can bind to C and Fortran and it does like matrix math in an efficient way. If you're doing this and then, if you're getting this data and then doing the analysis in the browser and then pushing it out to D3, it's likely that like the data set is pretty trivial to begin with. So a tool like this may be Overkill. I personally find using D3 from scratch to be like incredibly painful and not conducive to a iterative process at all. But if you really like that, you can totally do it. Anything else? All right, thank you.


The topic that I'm supposed to be speaking on, that I am speaking on is code as poetry and the subtitle is speaking, sorry, speaking to non-devs like a normal human being. Alright, so I think let me start off with finding out. I want to make this a conversation. How many ministers I have? About 30. But much, but not 10. Alright, cool. Alright, cool. Thanks. So I would like to find out, like I'm saying, I want to make this a conversation more of a discussion, like a peer-reviewed discussion. How many people in here can write code? Can. Andi tega. How many people cannot? Zero, zero. Alright, alright, good. I would like to find out something. This topic, Yaga, we are from a discussion. I was just talking to Tapio the other day. She wants to upgrade her website, et cetera, et cetera.  I'll do it myself. Sort of like what I was saying about some of these other clients. But I know it. I can do these things myself anyway. And I then ask Buti, but why would you want to go that direction?  That's what you understand pretty much. She has no problem. They're good. She knows how to cheat. I don't know. This is how my mama develop as I am. And I realized that the problem is simply that our clients do not speak code, right? And Jesus is in a problem. Chirung, chachu, guji, cibude in just normal, normal English guys. It's a bit of a problem. So I realized that it's definitely a communication problem. And I was just talking to our tutawed. They don't say guji. I'm sort of building up on what he was speaking about. So the idea here is I'd just like to find out from a few people how they interact with their clients pretty much the first time, how they collect the data, how they collect the content, how they collect the information and how they convey that message to the client to make sure QT, to know Fambasai. So I would like just a few volunteers. Seems there's already some hands in the corner there. Ten dekai, I'll start with you. Then I'll come through to you. Yes. Yeah, I guess. Yeah, please do so everyone can hear you. Yeah. I'm going to use the resources right here. I won't even speak much today. Not too few. Bumpai round of applause, guys. All right. Hello, my name is Ten dekai and yeah, I'm a developer person. So for us normally when we meet a client before we set up a meeting unless they want to meet first which we prefer to do the other way round is if someone calls and said would like a website and describes a little bit of what they want. In most instances what we've seen is sometimes they won't really describe exactly what they want, they'll be like I got your contact details from so and so I'd like a website, how much does it cost? Then we're always trying to explain we're like okay for us to give you the cost of a website we need to know exactly what you want. So then what we then do is we ask please send us your email and we will send you a web design questionnaire. And if you have any questions then let us know but basically on our web design questionnaire we can see the different things that you need and then we can quote you accordingly. So that's worked but it also hasn't worked so we're also trying to see how we can tweak it because some of the feedback that we've got from some of the people is that okay we see a questionnaire but some of the questions we don't understand so for us it looks straightforward we're like no do you want social media do you want this so we've seen when we look back on some of the questionnaires that people do fill we've seen that some of them though if someone goes through it though answer most of the questions but in some of them they were like actually don't know what this means so we're just trying trying to actually restructure that to see what we can do and then the other thing we've seen some people they're like I'm so busy I don't have time for a questionnaire but I just learned from Tawed that today what to do with those clients so it's okay. Thanks. Thank you very much. Please. Can I also have a lady please to come through and just let us know how they deal with that first time interaction with the client. Do I have any lady volunteers for after this guy? Hey. Wow. Hey Thelma. Trudong was the first time that you meet with the client how do you collect the data how do you collect the content and information from them how do you speak to them that's the idea. Don't worry we'll get to you once he's done. Yeah thanks.  So what I do I used to I studied this course called Systems Analysis Support so we used to learn how to collect a problem then convert it into a solution. So for me it's easier if a client calls me and then like I need a website and stuff it's easier for me to just meet up in person first. I try by all means to meet up in person get to know the person understand it okay it's a probably it's a stressful person more be it's an easy relaxed person or it's an individual and then we discuss on some of those things and then from that discussion I can see and even in my head I can calculate the cost I can even calculate on the complex stuff and now I'm easy it's easy for me to just tell them okay this is not going to work and this is going to work and stuff so first thing is meeting up in person and then talking later on. Thanks eh so I don't see Thomas he disappeared but in the interest of time I think I'll just move ahead so the reason why I just asked these guys to come through is I think it's about like I was saying it's about peer review it's about understanding how other people deal with certain things and what I've realized is the whole issue that I was saying about devs and non devs that I realized but a lot of times there is a what they call an information asymmetry at DTG where the developer is saying whatever he's saying one two three we shall do this for you we shall do that for you and the client is there and here has got their own understanding and then but also needs on a bye-bye when the product has been created then you're like but no you say you'll do this so the whole idea here I think they also share what I do anyways I I use the the questionnaire that the guys at Tall Springs use so I do that's my first probably after the meeting though after the face-to-face meeting because the whole idea is I want to learn whatever the trade is I think it was mentioned earlier on Kuti do you know what the people who you're working for are doing do you understand their brand because you need to be able to learn that trade understand that trade then figure out how you're going to market it because like it or not guys we are all marketers here you might think of yourself as a as a programmer etc etc but the moment that you then are using this software here for content management and pushing people's brands online you're a marketer so you need to figure out start learning how to market stuff how to market brands etc etc because so many times when you look at Zimbabwean websites I felt like they are just basically a magazine right it's just they bah you see it's a logo they you know it's a logo they it's a it's a slider it's a welcome slider it's always welcome to every time you know then you go down about us then there's you know our team and you can just so but what just basically happened was a person just went to elegant themes or they went to they went to theme forest and they just took something then they just basically replace the content but it's now about understanding which you know but guys what are your targets what are your goals etc etc then you start figuring out what the exact approach to marketing them online is so I say before I do social media management too and there's been times when I had a client for web design or end up referring which you know but you're not you don't need to go with this first because I think the data is also the issue of the traffic how you're going to then redirect people to come through to your website and you're even looking at their goals their marketing goals etc etc and you realize no no no no guys what you people need right now is to be on social media you need to humanize your brand right and you need to really push your brand there with us you know the majority of the people then the website can then come through yeah I didn't even really need 30 minutes to be honest but um yeah so this is just basically I did not need I did not need those 30 minutes but what I've just been learning over the last two years of freelancing is this really the face-to-face meeting is really about then finding out in depth and you need to be able to speak to them in English please don't speak to them in Python or Java or C++ or HTML or CSS don't need to mention what you're using a plug in just say it's an application just say it's an application it's really at times it's really about the the terminology the jogging that you use that's stuff that is that's simple can really make the difference between them understanding what you're talking about Kutikan jilkan Zimaus Aosung Aosure Mbeba because the that's really the miscommunication that then takes place but anyways there's how to ga-ga-ga I don't know if there's really any sort of comments feedback et cetera et cetera like I say it's a peer review discussion or not really doing much other than that so I don't know if anybody has anything to say that they think can benefit all of us here


Hello and welcome to NewsClick. The Ministry of Defence in a new notification has announced setting up of Defence Planning Committee. And to discuss the issue, we are joined by D. Raghunathan. Welcome to NewsClick. So, let's start with what is this DPC and what will be its role and basically what are its compositions. See, the Defence Planning Committee is a new creation. The idea, I think, is to create a new institutional structure to make up for various deficiencies in the past. It's chaired by the National Security Advisor. It consists of the three Defence Chiefs, the Chief of the Integrated Command Office at the Defence Ministry, and three Secretaries from the Defence Ministry in charge of finance, etc. The brief given to this committee is to draw up a national security strategy, a national defence strategy derived from the first to also supervise the defence production policies and programmes in the country, which is of course, intimately linked with the defence procurement system and some other tasks. It asks that this committee will prepare reports, which would be submitted to the Defence Minister, leaving open the question of where does the decision making lie? If the defence committee makes a recommendation, is the minister bound by it? If not, what are the criteria brought about to change decisions or is the political leadership going to override this committee? However, given the composition of this committee headed by the National Security Advisor, who is known to be very close to the Prime Minister and who answers essentially to the Prime Minister's office in essence to the Prime Minister himself, it's extremely doubtful if Raksha Mantri could overrule the recommendations of this, which means that you have essentially a committee which is outside the purview of political accountability, not answerable to parliament, quite capable of saying all these discussions are security related and therefore cannot be revealed, but at the same time give a allowance to the political leadership to say we have taken this decision ourselves. So, this also I think the recommendations talk about that the committee needs to follow up on the points that they have mentioned, if they are talking about production, they need to follow up with the ministers and since you are pointing out the current National Security Advisor deals directly with the PMO, then the defence minister's role will just be like in stamping authority, right? I find three very serious anomalies with the brief and the mandate given to this committee. The first is that the National Security Advisor who is chairing this committee is strictly speaking an officer with advisory role rather than an executive role, but this committee combines an advisory role with executive responsibilities which is highly anomalous. If there are executive anomalies which lie within the purview of the defence ministry, it should be run by defence ministry and be accountable to Raksha Mantri and through that to parliament. That does not seem to be the case here. The other major anomaly I find in this is till now the national security strategy, if you like, of the country was supposed to have been laid down by the National Security Council. Where does this leave now the National Security Council? The National Security Council was broad based, had strategic experts, had diplomatic experts, experts drawn from different fields, who could then have given inputs into forming the national security doctrine or strategy from which your military doctrine and your defence doctrine could follow. This combines all of them and puts them all into one basket. Not only does it I think marginalize the defence minister, it also marginalizes the Ministry of External Affairs and it also marginalizes the National Security Council and the various dimensions to that and essentially makes this a creature of the Prime Minister's office but with the three service chiefs added in a sense as decorations to say we have taken the military into account whereas actually decisions will be taken by the political leadership. So as you have pointed out and this way I have been witnessing when we see the NDA regime that consolidation of authorities or the power is happening basically at the Prime Minister's office. The way this government has been pushing defence production and bringing in private players especially from US and its allies, do you think this is also a move to fasten up that? Well it may play that role in the sense that since this committee has been given responsibility to oversee defence production and currently the defence production policy is heavily weighted in favour of private sector players within India with whom foreign equipment manufacturers from whom India is going to buy equipment are in a sense forcibly linked. On paper there is nothing, there is no compulsion but with a nudge and a wink you can always indicate to the foreign equipment manufacturer as to who he should tie up with in this country and with elections around the corner next year and a large number of defence purchases going to take place in the next year or so if this committee under direct leadership of the Prime Minister's office is given freedom you are likely to see crony capitalism showing up in a very big way. This is basically what's happening when you see Boeing tying up with about those 110 fighter jets and all basically then make in India and indigenous production goes into backbench because these when these private players come they're not coming with they're not giving us technology they're just producing for us we are buying them and money is going outside right. It depends on how you define make in India if by make in India you mean sub-assemblies, subcontracting, screwdriver technology then you have make in India on paper you can show that 30 percent 40 percent of the expenditure is taking place here but no technology would have been transferred, no serious capability would have been built which is the crucial function of a national defence or security policy which is that your industrial base should develop to the extent that it should be independently capable of supporting the military especially in times of conflict. So also because we've been witnessing there has been a massive protest by the defence employees the the industries which are there inside India they have been protesting for the fund crunch for the because there has been fund cuts and all what could have been an alternative that could have ensured that we grow we become self-sufficient on our own and we don't have to rely on the private players. The most important of these is of course to have a robust and self-reliant industrial base in this country not only for manufacture but also for research and development in that a very crucial role has to be played by the military I think till now this entire procurement process as well as the indigenous R&D process treats the military as giving inputs but then decision-making, financial outlays, procurement processes are run by the civilian bureaucracy which has very little idea of the technical requirements of these weapons platforms or of the armaments requirements of the military. On the face of it the only positive of this committee was the presence of the military chiefs because the one thing that this country has long lacked is an integrated planning which will involve top levels of the military leadership along with the civilian leadership of the country under the overall guidance of the political leadership that has not happened. Several commentators have made this the basis of a positive evaluation of this committee by saying finally we have brought the military into the process I don't think so because just by bringing in the three service chiefs does not mean there is an integrated planning process of the military which is built from the base up. The real answer to that would have been an integrated defence ministry which integrates the military presence along with the civilian leadership build that up have a technology forecasting and weapons platform forecasting system based on which the Indian industry of course in the public sector because they have the maximum experience in manufacture as well as research along with a set of feeder industries which could be from the private sector as well needs to have been developed. That's all the time we have for today Raghu and as the committee starts functioning and what they do we'll follow up these issues once again. Thanks a lot. Thank you for watching NewsClick.


This was the first time U.S. Air Force aircraft got to be involved in the Republic of Korea Air Force formation." I think the celebratory flight of the Rock U.S. Air Force was a great opportunity to showcase to the public the solidarity of the Rock U.S. Alliance.


All right gentlemen C3 in the Bucks here in California Jake. What did you think first off? How did you think the event went? I mean I knew the vision when I was talking with Cliff for months about this and I could feel that his energy and what he was Already doing and I saw what he was doing through the social media and stuff that he was putting out I was like, oh, this is brilliant. This is a brilliant idea. And obviously Cliff is He's down for Georgia. He's trying to get Georgia blasted out He's trying to get Georgia seen he's trying to you know work on little Georgia's you know how he's thinking of it You know, I got my kids seen I got to get my environment seen and I'm always been thinking that I've been thinking that since day one about here and I'm hearing like a kindred spirit in that same sense of like We got to get our kids in front of the best eyes and the best opportunities possible And so the dude like he made it out of thin air right there in Georgia at first And then you know, Iowa and New Jersey and this and that and I'm sitting there going Why doesn't California not have something like this? Like you got to be kidding me Wait, this is the state of Dave Schultz. This is the state of you know I mean, this is an amazing state that we have produced tons and tons of amazing Amazing names in the panorama of wrestling and we don't have a combine like this Like it just didn't make any sense to me and so I kind of was hey, man, let's let's do something like this out there He's like really okay. All right. Well, all right And then we just kind of started tiptoeing to put it together and making some phone calls and this and that and Man what we just experienced here like for me. It was it was such a gift, you know It was such a gift seeing Clovis, you know Working with the Clovis West dude, you know or you know guys from all those the hotbed of central, California Where they're just smashing each other all the time and now they're drilling with each other and helping each other up off The mat and okay now you get your five in and this and that whatever and it was just awesome and all the different Dads that I know and the coaches that I know like not sitting apart like sitting together Hanging out watching the kids and just having a good time. I mean I could go on and on obviously you can tell But I'm just vibing off of what what what we just were able to be a part of cliff Did you and dreamy p8s ever envision this? Becoming what it has become and it's coast-to-coast now. Did you and dreamy p8s envision that? I Don't want to say no because I just I'm kind of a big thinker. I don't I don't know I just do I just do my dad was just a worker and I just I don't I don't worry about Growing up watching him. I don't I don't worry about kind of what what what might not happen We just we just made it happen. I came out here two weeks ago and Registration wasn't going great. We're planning this thing and we're kind of looking at each other like, you know We're gonna put a plug on this thing and I'm just and I just told Jake I said we're gonna make it happen, you know and and like you said we just hustled a little bit more and ended up being It ended up being great. So short answer. I just I try not to think about it You know and we make it happen and do this was a this was a true testament to the wrestling community Just coming together and kind of what I told the kids at the end of camp was just you know I want those guys being like us. We were out there wrestling and moon yosas wrestling and we got Brett Sanchez and guys coming in that they're they're brought ten kids in a van and slept wherever You know to get here and they just they did it because I love it You know and they're on the mat and wrestling it was just this is a good experience You know so I think once we do that one place and people see it and they kind of experience To social media and and things like that and they want to be a part of it And I think people kind of looked over the fence of what we're doing and then they came and I think we got I think we got snowball effect going so is California better for it is Georgia better for as Iowa better for this New Jersey gonna be better for it as it made the places actually better places. Is it help kids? Absolutely, I think it gives them a platform. I mean there's people that don't come for for Political or internal or personal reasons and you know, that's opportunity they missed but It's an opportunity nonetheless and there were some kids and I look at it this way if there was it was probably 6870 kids here if if two kids get an opportunity that never would have got an opportunity We're doing our job the vent serves a purpose, you know So that's that's kind of the way I look at it and that's you know That's why Jake's so fired up because I I don't know some of these kids that Almost hate each other because they're from different schools or wear a different singlet So like ignorance was bliss for me I'm just expecting everybody to show up and work and he's like looking and going oh my god And I just I'm just like oh it's supposed to be this way, right? So like ignorance was bliss for me and kind of Pete in this sense So we just we do what we always do we just grind and promote and people show up and if they don't they don't You think this is something we'll see in the future. I mean this states huge there could be a Bay Area one I mean easily the metro areas over 10 million people up there metro area over it down here is over 15 million people They're so it spreads.  Where do you want to see it? How can we make it better? Give me some feedback and and I Without the coaches, we don't have the kids without the kids We don't have the coaches and and I got to rely on guys like him and and and mark and Daniel and and and Terri Pelle and and and say hey, you know what should we do is this too much is this too close and And we'll figure it out, you know, we'll figure it out, but I guarantee you one thing we ain't scared We ain't scared to pull the trigger on something. So yeah And a nutshell. Yeah, we're gonna do more but we got to make sure it's right. Well, and also, you know There's great kids all through the state We got some farm kids out in the middle of nowhere that'll come and wrestle you just give them a chance I mean I wrestled something.  I don't think I've been taken down in like five years Seriously good took you down. I couldn't believe it Reyes. Couldn't yeah Reyes could you I think Reyes could he just did it But yeah, that was a sophomore K. He took me down.  Can you show a couple things? Of course his face was like you're asking me. Let's go. Let's do it You know and like he just walks up to the circle, you know, what do you guys want to learn? Some kids like throws.  Yeah, and then he's like, but it was a slick one It was like a single kid bites and yeah, and it was a whole series You just pulled it out of his little wrestling in psychopedia. I'm on oaky state overload. You guys know that, right?  Okay, but but but looking at it looking at it. We're probably gonna see more of this What needs to improve? Just I know that's how your mind works. Yeah, I mean Improvement's great, but just just convincing people that it's it has value, you know Just just that I want everybody to benefit from it Not everybody's going to but convincing the coaches and convincing the kids kids that there is value And I think that should we just keep creating an experience we keep we keep guys like you involved that are gonna cover it We continue to when we had a lot of rolling live stream the entire time You know of just live stream up with with five different cameras So like just continue to let people see the experience and then they they stick their toe in the water And then they then they commit fully well hang on a second. I want to here's what needs to improve I want to say this because you don't know this part. Here's what needs to improve, California Here's what needs to improve coaches. I understand being competitive I understand there might be some water under the bridge. Oh Calvary Chapel We recruit this school recruit blah blah blah blah blah blah blah blah right now today There was none of that. It was all about getting these kids seen by Jamil Kelly and Kevin Ward And I mean that was great and by the way, man, I got to see Jamil teach He is an amazing teacher an amazing communicator Meetin Kevin Ward for the first time. I'm I like that guy. I'm a fan. You know all these good Stutsman Buffalo, I mean sweet people in my opinion coaching for the right reasons I got to hang out with them. It wasn't like a quick like hey, I'm at the inside ways Here's some of my kids here. I want you to call them.  So, you know, I Mean I could go on and on this was an amazing event and I was really fortunate that Cliff decided to bring it here and What we need to do California is we need to work together for the benefit of the kids We need to set aside any water under the bridge. We need to set aside any Wrong predispositions and we need to think about the kids We need to think about the opportunities and we need to vet these coaches As we hang out with them and all that and then we need to be able to make educated decisions of how we Influence our kids and to what colleges to go to Also, you know those colleges that got a chance to meet these kids and be with them man I'm a fan of Stanford coaching staff. Now. I'm a fan of Army, you know West Point. I'm a fan of this guy now now. I'm like Stutzman I never thought of sending the kid out to Buffalo I want a kid to go have the experience of living in upstate New York and being coached by a man like Stutzman And now I know there's another place that's like, okay, my kid will be safe there He'll be loved there He'll be taken care of there and and now I know a handful more coaches that I'm gonna keep in contact with and I'm Gonna be sculpting and shaping a kid to be a custom fit for that style coach or this style coach or whatever California we need to get it together. We need to quit working against each other We got to work together so that our kids can have the best opportunities possible. That's what we need to do