Yes, Pika, you can go. So thank you for the introduction, for the invitation to this great conference. So I'm going to give a talk more about the random metric theory talk, ready. But a problem will arise in the study of random neural networks. So it's based on a couple of words, one about a couple years ago and one which is ongoing with some in-patient. And so let me kind of give you the model and what I'm going to study. So here we are in the neural network. I mean, you know, maybe we know from all the talk we saw yesterday, you know, X can be an input vector and actually later in the input matrix, if you take four and several samples of this data and W would be a certain matrix of weight. And then you have this non-linearity or put N2Ys to this vector or this matrix later by the activation function. Okay, so here I give some example of a different function that people use. It'll be a motivation to do the simulation with this activation function. So the signal aid or the absolute value or also something used with the activity in the unit which is the max of zero X. And that's the model I'm going to study is based on that. So here I do several layers. I'm actually going to focus on the singular case, but we still have some result on multiple layers.  So often to try to understand very complicated systems, one good way is to make things random. Okay, so you can maybe take the random weights, random data and see what's happening on different statistics. Of course, random neural networks are used. You know, for initialization, you can take random weights for instance and things like this. So where do random matrix theory comes into place and where, you know, eigenvalues and maybe eigenvector also arises is by the following. So here now X is an input data matrix. So, yeah, I did some more dimension to the data. And let's say A is an output target data sets and the points of the training phase to find the best output weights after one layer of the network. So here W will be random. So it's a random weight matrix by the entries and let's do ridge regression. So let's try to minimize this loss function here. So of course you want the output to look like the target data sets. And you have the regularization factor which to avoid overfitting. And you can just solve this thing by the, what's called the ridge regressor and you have this formula here. So of course it depends on the target data sets. It depends on Y, which is F of WX and on this factor here. And one interesting things especially in random matrix theory that this matrix here is a very particular one if you know a bit of operator theory is just the resolvent. K of pi of minus gamma of the sample coherence matrix and Y transpose Y. Okay. And then you can even look at different statistics at the expected loss of loss function depending on the weights or the random data, et cetera. And you can see that the eigenvalues and eigenvectors arises in this type of problems because of course the resolvent contains information eigenvalues and eigenvectors on Y transpose Y. It's a symmetric matrix. So you get a real eigenvalues and the normal eigenvectors and you can reconstruct this. Okay. So that's kind of the training phase will depend possibly on the spectral measure. And that's where the interest on the field of looking at these several matrices comes from. It is so a problem. So the model I'm going to consider here is the following. So here to low assumption but I'm going to go pretty quick about them. I'm going to take WNX random. Okay. So of course you could ask whether for application it's useful not to take data random. But here on a more theoretical level so some interesting thing happened when we take both WNX random I'm going to take them independent centered also symmetric. So the third one to zero whether it's really important to notice is a question. And I'm going to scale them variance one but this doesn't really matter. And they're going to have a sub exponential decay. Okay. So for instance, you can take Bernoulli works or Gaussian or exponential. Many things work. You won't see me check them. Uniform I guess works. And that's different things. So the activation function has something very strong which is real analytics that's used in the proof for polynomial approximation. For application is not completely, you know, at a way of course I should get so value I should do the real you which are not analytic but they have an analytic counterpart like the soft plus something like this. And I'm going to add an assumption and it'll be later for the second part of the talk which is I'm going to send to the function. Okay. So I'm going to take the function so the expectation of f of a Gaussian is zero. Of course it doesn't matter too much it's just setting a rank one perturbation to your matrix but this might not have a trivial huge I can value somewhere. It's like a nice center.  And I'm going to work in this random matrix regime when all dimension grows together. If I am sorry, our two parameters depending on the value X and different sizes. Okay. And so the matrix I'm going to consider is simple covariance matrix, y, y transpose where y is f of the value X over scrout and zero. So here's a scale to have something over one. And I'm going to look at this matrix here. Okay. So the eigenvalues and in particular I'm going to first look at the EED as the empirical again value distribution we're going to give us, you know information on the whole kind of macroscopic behavior of eigenvalues. So what is the first theorem is the following is that actually there is a, at the limit a deterministic probability distribution mu F for the EED. Okay. And we can describe it but what's called a quartic self-constructed equation for a stress transform. So maybe you don't know what that means and it doesn't really matter. What can matter is that it's more complicated than the usual one. Like the beginner semester color would be quadratic or much of a capacitor will be quadratic. Here's something a bit more complicated because there's a before. Okay. And it depends on F, I am side but it doesn't depend on distribution of W and X. Okay. So it's a universal again distribution. And so this type of problem has been studied actually quite a lot recently. So the first two papers looking at this in a way was being Tanwara and Luar Lioquie then Luar Quie for different assumption most likely think they're Gaussian or kind of functional Gaussian, something like this. Last year, Fan and Wang, I think Joe's going to give a talk later this week. He looked at a similar problem with ID Gaussian for multiple layers also with a nice result. And also very recently he went shorter looked at his problem again by using a different method and which is a bit more robust, the resultant method that we use. So, okay. So let's do some, you know what does the eigenvalue look like? So it looks like this. So here I take the sigmoid so a hyperbolic tangent, the value and absolute value. So I took a different iteration function and look at the ED. So, okay, maybe you just different ED something interesting happening for this one if you look very closely that it's actually the margin capacity distribution. Okay. So it's kind of interesting that if you take WX and you take the absolute value of every entry and you get a simple covariance matrix you recover the margin capacity distribution. And actually it's not just the absolute value. So you can find a wildcast of function that took even more complicated at the end to show that you still get a margin capacity. Here I take an odd function extremely three X plus sine log absolute value and they all get the margin capacity distribution of a certain parameter five of ups. Okay. And so why is that is for the following? So that's the limiting measure. So, you know, that's a big equation but don't look at it too much. It's not that important. What's very important is the first line is that it depends on two parameters only of the function. So an expectation of f of a Gaussian square. So of course it can be zero. We can't be accepted f is zero and theta two the expression of f prime of a Gaussian square but the square is outside of the expectation. So in particular, this can be zero. And I wrote it like this because there's something interesting happening that two special cases you have theta one equal theta two then of course this term just vanish and then you have a cubic equation. And this is not surprising because the only function for which theta one can be equal to theta two are linear functions. Okay. You can just extend them up basically. You recover only linear functions. So of course your model is just a linear one and so you recover the product b sharp. So linear network yesterday and you recover just this matrix. And another special case is that theta two equals zero and this term vanishes and then you recover a quadratic equation and you recover the marching capacitor distribution. So actually you have a whole class of function where theta two equals zero that make you recover the marching capacitor distribution. And actually the interesting behavior is that we have a result for it. I'm not going to talk about it for a time reason but if you look at several layers so if you keep putting ID weights and you choose your function to be marching capacitor at the start you're gonna recover marching capacitor at every layer. If you're normalized correctly each time. So it's an interesting behavior actually the fan wrong paper kind of states exactly what happened for multiple layers for any activation function. So even though I told you to not look at this equation you can look at actually this model of random matrix theory. So actually the distribution is similar to in the same the asymptotic one as a linear model. And actually it's an interpolation between the product b sharp and marching capacitor. So it's really like two extremal cases and depending if you have a function which is not one of those you have an interpolation between the two. You have a product b sharp and a full IID matrix of correct dimension. And you just have the two. So if you want to do that two with this one is zero and two equals zero you just have this thing to be zero. Okay. So that's the behavior for the eigenvalue the empirical eigen distribution to the whole thing of eigenvalues and we saw it was used for the rich regression here. But it doesn't tell us anything about another interesting random matrix theoretical problem which is the behavior of the largest eigenvalue. Of course, if you know the asymptotic EED you don't know anything about individual eigenvalues. You just know that the whole thing looks like this but you don't know about the first or second or third one even eigenvalues. And so we're going to look at this now. So the largest eigenvalue is as you can't really see it maybe directly by looking at the solution of a rich regression or something like this but there's some numerical evidence in this paper that says it earlier that shows that maybe the existence of not an outlier so it means an eigenvalue that pops out of the book could help having an idea of the testing error and the main observation for a certain I think classification problem that the existence of an outlier and the distance of the outlier from the book gives you a better testing error. So the idea is that if you don't have an outlier you actually they can't even classify correctly. And if you have one which is further and further from the book you get a better and better testing error. Okay, so this motivates the thesis study of this larger second value which is actually we'll see a pretty non-trivial some interesting things happen very different than the empirical eigenvalue distribution. So I'm going to look at three different as a behaviors or three different ways to change the parameters and see how it impacts the larger second value. So the first one I'm going to call non-universality is the following. So I'm going to fix an activation function F here just take X square minus one over square root two and just for simplicity. So it's centered so that, you know, I told you I don't have any huge eigenvalue anywhere. Square root two is just so theta one equal one because it doesn't really matter. And theta two equals zero at six five M side to be this one and I take X to be a Bernoulli. So when I mean Bernoulli here, I mean minus one one probably two one half one. Simplest distribution you can take and I'm going to check the distribution of double. Okay, so if I take W to also be Bernoulli of course I get Marchenko pastor because I have theta three equals zero. Okay, so I know I should get Marchenko pastor and I don't get any outlier. Now I can take something with a bit of a heavier tail. I take W to be a quarter of the time Bernoulli and three quarter of the time Gaussian. So now I have a, you know a force moment which is a bit higher if you want. And I start to see something happening which is I get one outlier. And then I can take something with a bit heavier tail which is just Gaussian and I get still an outlier but even further away. So here was two and here is like 2.2 or something like this. And you can see that the heavier kind of you get the the tail of the distribution is the further from the ball to get this outlier. So this is a behavior which is really interesting and very in a way nonlinear. What I mean by this is that if you look at the study of outliers and for linear models in one number six theory the position of the outlier is usually universal, okay? Which means that the existence and the position is universal and the fluctuations sometimes are non-universal. But here it's very different because just the existence or the position is non-universal because it really depends on the distribution of W. Well, it's fixed by here W. So this observation was made in this paper I've said it before while you're looking though they couldn't really explain, you know mathematically why you have an outlier here. So the second interesting behavior will be the following will be about the architecture of the model. So what I mean by architecture with a single radio model so not the number of layers but the different dimension if I am size the number of samples or in zero. So here I'm going to still fix this activation function. I'm going to fix W and X with a certain distribution and I'm going to change by M side. So of course the shape will change because I will again, I'm searching for posture reshaped fire website but the most interesting thing will be about the largest I can find. So here for this fire I'm saying I don't get any outlier and now I get the fire size slightly different. So fire website will get lower, I think and I get one outlier. And if I do it again, now I get two outliers. Okay, so I get something still again very different and you want to understand these two outliers as one for X and one for W. That's the idea and we'll see at the end why is that. So that's this problem where the architecture in a way the way the dimension kind of behaves together also can change the existence of another outlier. And finally, I'm going to talk about the activation function and how a different choice of activation function can also make a rise is an outlier or not. So what I'm going to do here, I'm going to take a certain function so it looks very complicated. Actually, it's not really just cosine of alpha X, right? But it is centered and scaled so that theta one is always equal to one, okay? So theta one is equal to one, whatever I take alpha to be and theta two will always be zero because this is an even function. So I still get much of a capacitor but the two partners are fixed. So in particular, the ED will never change. The asymptotic ED because it depends only on these two parameters and they are fixed. I'm going to fix phi M psi also and I'm going to fix the distribution of W and X also. So everything is fixed so the ED will be the same and whether or not the outlier arises means that something different is happening to the activation function. And actually it is. So here take alpha equal to I get much of a capacitor and now outlier. If I take alpha 1.5, I start to see one outlier coming up and then if I take alpha equal 0.8, even smaller, I start to see two outliers. So really it means that the behavior of this large second value depends on a different way of the activation function than just the ED. If you want to have to go maybe one order later to understand what's happening for these outliers. And so actually, so all of that observation will be given in the theorem, but I wanted to maybe show, you know, what's happening because maybe it's less clear by just reading these phrases that something different is happening. So yes, it depends on another parameters which are called theta three here, which is now the expectation of F second of a gauge. Okay, so whether or not you have an outlier in the position of the outlier depends on F second of a Gaussian and not just F prime of F. Here I'm going to define kappa for a later remark. So here's the kurtosis. The kurtosis is the fourth moment divided by the variance squared. And it'll depend also on that. So on the fourth moment in particular. And so what is the theorem? Is it actually the behavior of the large second value also relates to a linear model but which depends on, which distribution dependent on W and X. So what I meant that there exists to rank two random matrix J hat. And his J hat is not really complicated but a bit big to write, but it depends on W and X. It depends on the distribution of the volume and on the distribution of X. If you want, you have one rank for W and one rank for X in a way. It depends also on N zero is hidden inside the matrix. So they depend on phi and psi but it doesn't depend on the function. Okay, the really only theta three will be the depends on the function not on J hat. And you have the largest second value of M is the same as the simple coherence matrix given by this matrix. You have the same linear model that we saw earlier. So we know that ED is given by this but the largest second value is given by your perturbation of this linear model by your rank two random matrix J hat given by this coefficient theta three and the distribution of W and X. Okay, so here the existence or not of outliers is what's called the BBB phase transition. So it's something in random matrix theory that exists which states basically that maybe if the perturbation is big enough so if theta three is big enough if the fourth moment of W and X are big enough if there are five of upsides because low enough or big enough then you get one or two outliers. And that's a theory that pretty well known in the linear case either using a one moment method or also a super alienation and free probability something which is pretty well understood. So there's some several cases I want to consider. So if there are two equals zero which is an example I showed you in simulation you can match in capacitor. And actually in this case you can have something a bit more simple which is actually for the largest second value you have a rank one perturbation J and one ham is just a matrix full of ones. And it depends only on the on F second and on the kurtosis of the distribution. So on the fourth moment. So the largest the fourth moment is the further away from the both you are and the bigger theta three is also the further you are from the book. Okay, so here it's a bit more simple and actually here the position and the existence of an outlier is completely explicit. Okay, like work one perturbation of full IID matrices are very well understood now since maybe five or four with the first case of the VP phase transition and there's been many, many work to understand this. And so you can even have a formula for the position of the outlier in this case for W and X random. Another observation is that if theta three equals zero then of course there's no perturbation. And so you don't have any outliers. Okay, so one reason for example if your function is odd you don't have any outliers. And also it's a way to look which I'm not showing here but if W and X are Bernoulli so minus one one with probability one half one half then J hat is zero also. So you can see it directly here you see it in this formula because kappa which is the kurtosis of W and X will be zero. Okay, because Bernoulli is the only distribution that kurtosis one. So kappa will always be zero for this distribution. So that's an observation that was made also in this paper by the way you're clear that if you take Bernoulli weights you don't get any outliers. And that's kind of the idea why. So finally the two little remarks I wanna make is that here it's a rank two random matrix. So usually a rank two means that you have possible two outliers. And it seems to be true, right? You have sometimes two outliers. And we believe that the result should be true. But if you look at this model and you look at the possible two outliers they look similar to the two outliers here. Sorry, here. However, the proof is not as robust as this and we can only consider the largest again value. And on the second one, but it should be the same idea that this gives the two outliers. And so finally, I want to finish by saying one thing is that this is not completely surprising even though it's very nonlinear in the sense that it's behave very different than a linear than usual linear models. There's a paper from 2015 by Fan and Montanari who looked at the similar problem for kernel matrices. So F of X X transpose instead of F of W X transpose. And they have actually similar idea and similar result where it depends on the same theta three and also on the fourth moment in a similar way. So of course, when you one outlier you have one symmetry, that would value but you get something similar. And actually I want to thank Joe Fan particularly. Actually, we talked about this and he instigated this problem really of looking at our second value. And this is slightly different because you have in the way to randomness you have W and X to be random. So that's one way where you have two outliers. Okay, so I think my time is up and I want to thank you for your attention.


Hey guys, it's Nate, AKA The Foot Accountant. Welcome back to the channel. We have some more updates regarding EA's responses to questions about the whole black market icon moments, scandal, issue, whatever you call it that is going on right now inside of FIFA 21. There is a interview that happened between the Mirror in the UK and the EA Sports spokesperson, which of course was not name dropped or mentioned by name, but EA have made some statements that has been kind of moving around the internet today and it's causing a lot of stir in the FIFA community and we need to talk about it. Of course, just a quick little rundown. If you are not aware, there is an allegation against FIFA 21 that there are employees inside of the company that were giving out icon moments into people's accounts untradable in FIFA for exchange of a little bit of money under the table. So obviously EA, since it impacts them personally, it is employee at EA Sports that is doing this. They launched an investigation. They released this update four days ago about what is going on. The full investigation is underway and they said they basically stopped content granting in its entirety, which is like giving pro players their pro player cards or all that that entails. I'll link this, if you have not read it down below in the description that was released obviously a couple of days ago, but today we have a new update from the Mirror website, a news source from the UK and I wanna talk about what this article and what these responses from EA are kind of bringing out in the community right now with all of the, just the responses that EA had here. And obviously, we know that EA sometimes doesn't seem to understand their own game, but some of the stuff that they said today is really, really crazy. So let's get right into it, let's cut to the chase because there's some crazy, crazy stuff in here. I'm gonna go through these early on questions in this article really fast because there's not a lot of juice here, but EA said they have launched a rigorous investigation. Their initial investigation shows questionable activity. So they have found stuff, right? They are finding stuff which we know that they should. And again, they go to an extent here to say that it involves a very small number of accounts and items to make it try to minimize the problem a little bit. So they were asked if they would provide full transparency regarding the results and they did not say yes or no to that, they just said we will continue to provide updates, which they have done decently well so far. So we'll keep an eye on that. Now of course they talked about content granting again and how it is even possible. That's in the pitch notes like they linked. And again, they go to this link to say these grants have no impact on the odds of any player in the ecosystem or the market acquiring these players. So they're saying that them granting items into different people's accounts, whether it be for like they mentioned their employees for professional events or for testing, it does not affect the pack away at all or the ability for a normal Joe Schmoe to go get a card on the market. So they did say that there, they mentioned that they are able to track this level of information that they can track any specific player item on the game. Now how, you know, we see a lot of things in FIFA where, you know, maybe you lost your card or somebody quick sold your card. They can track every single card in this game, like down to the card, right? Every single gold, furlough, and mendee that is on this game, there's thousands of them, right? Every single one of those they can track. Anyways, let's move on because this is where it gets really, really, really good, right? Some professional FIFA players have said that they had to spend thousands of pounds when the latest FIFA gaming title is released in order to have a competitive advantage in order to be competitive during qualifiers and in the buildup to major events at the start of the competitive cycle, which in the view of some is creating an unequal playing field for the professional community. Now, EA responds this and says, FIFA All The Team is a multifaceted game of skill. Oh, okay. It's important to know that you can acquire all items in the game without ever spending money with purchases being entirely optional. Now let's talk about, this is technically true, right? But EA is stretching the truth so far. Basically what they're saying here is that, yeah, technically it's possible, right? You can be a top 100 trader and you can get yourself to 15 million coins by the time icon moments come out. You can go buy that icon moments from all of them. But let's think about it here. How many people in this game actually run RTG No Money Spent and trade slash play the game and grind the hours, right? To get to 15 million coins, right? Basically, if you're gonna do that, you're gonna be doing it through trading. Top 100 rewards unless you get insanely lucky pack and pack an icon or like a team of the year card earlier in the year. It's gonna be really, really hard for you to get to 15 million coins. Really, really hard. And I would venture to say that there are probably less than like 20,000 to 30,000 people in this entire world that are first owner RTG No Money Spent and have 15 million coins. I would venture to say there are very, very few people. And that number of my estimation might even be very high. I'm just trying to make a number to throw out there because there are millions of people that play this game. So just think about that though. Think about that though. EA thinks that it is very possible for you without buying fever points to attain all items in this game, whether at one time or separate times, right? Any professional competitive player needs to demonstrate the skills of the meta game in order to be considered proficient enough to compete at an elite level. Okay, now this is where it gets real good. This is where it gets real fricking good, man. Okay, so they say that any professional competitive player, any professional pro competitive player needs to demonstrate the skills of the meta game in order to be considered proficient to compete at an elite level. Which includes, right? Which includes trading, economy monitoring, squad rotation, and challenge completion. So basically, EA in the mind of EA, if you're a pro player, no wonder they tweet all these stuff out about pro players being like the next best thing basically to football players in the world. They tweet about all the stuff like they love them because in their minds, they're masters at playing the game, at the meta, trading, economy monitoring, squad and rotation and SBCs. When a lot of the times, if you're a pro player, you're spending all of your time playing the game to be good at it, you don't want to worry about the rest of this stuff. You don't want to worry about having to, is my team of the year on Boppy right now at 6.8 million coins in early February, is he gonna be 6.5 or 6 million coins at the beginning of March where I need to sell them so that I don't lose a lot of coins? Well then what if you sell the card, then you're not gonna have the best in Boppy in the game to play with and compete at the highest level that you need to as a pro player. So EA here is saying that pro players have to be proficient at trading, economy monitoring, squad rotation and challenge completion. I don't even know where to begin with this comment. It's like every single person that plays FIFA at a pro level in EA's mind has to be an absolute combination of a trader, a pro player, you have to know supply and demand and know when people are gonna be panic selling and especially in FIFA 21 where you have discords and all these traders with their services telling thousands of people to buy and sell cards like this and influencing the market from an outside perspective as well to in order to like know all of that, know when SBCs are good to complete, like what? Like no, they're pro players, they're good at playing FIFA. That's why they're a pro, right? Although the goal is to win matches, Foot isn't just about playing simulation football in the stadium, just as football isn't solely about the performance, match day, there are many areas where players need to be better than their peers in order to create an advantage on the pitch. Now there is a slight, that is true in a sense because that's what we're trying to, that's what I'm trying to do here on this channel, right? Is not so much the gameplay aspect but the economy monitoring, the trading, everything but gameplay with content in this game, they want you to be a pro at it, right? They want you to be a pro and that's what I'm here to do is to try to help you like think about these things using knowledge of the past and trading and in the market and trying to figure out when is a good play to do this? When is a good play to do this? When should you wait to buy stuff because the market's gonna go lower, right? But they want these pros to be not only just FIFA pros but to be market gurus as well. Progression is a prerequisite if you intend to be a professional player in any sport and esports are no different. So I think that this is just crazy, crazy, crazy, crazy from EA that they're saying a lot of this because it is, it's just way too far, it's way too far. I think there's a comment down below here that puts it perfectly, look at this comment. The responses are extremely ingeniousness, not sure if that's a word and paint a picture of sunshine and rainbows. EA holds the football genre hostage and milk the player base of their money because obviously a lot of the general player base does not feel like they have the time or the ability to acquire all the items on the game without spending money and that is where EA has set this game up to be. EA has set this game up to be, okay, you don't have the time or the ability to grind for a 15 million coin card in February. Okay, well, here's a shortcut. Here's FIFA points that can maybe help you get there, right? And so that's how they're gonna see it. They're gonna say, yeah, you can play this game completely free, you can run an RTG, which is what we do. But obviously they're gonna try to format the game in a way that makes you want to spend FIFA points. Why do you think that, you know, the week before we have big promos, like before Team of the Year, we have massive coins draining SBCs or during a promo, they release player pickpacks and icon-repeatable SBCs that drain coins off the market so that, oh, shoot, I'm out of coins because I did this icon moments or icon player pick SBC. Now I need to put on FIFA points when a promo comes that next Friday because I have no coins, right?  So that's why this whole thing is not actually what EA makes it seem, right? So if you're reading this article from like an EA or like an outside perspective, you're kinda like, okay, yeah, they do have a point, right? But then you look inward and you see actually how they are making this game and throwing this game out and it just does not make sense. It does not back up exactly what they're saying. Now I wanna take a look at a couple other things because I wanna talk about, specifically they said trading. They want you to trade, right? They want you to trade and make coins and be involved in the economy monitoring inside of FIFA 21. Now this is from the EA help, right? FIFA rules, don't buy coins, right? They say don't buy coins. There's a few ways to earn coins. Playing matches in foot, completing SBCs that have coins as a part of the reward, selling items or trading players for a fair price on the transfer market. So this fair price that we have right here, okay, we can trade players for a fair price. Yet, you set price ranges on this game that make cards extinct oftentimes. You change price ranges to make cards extinct earlier when they could go higher, making people make less coins, right? Or, you know, you create an extinct player on the game and fail to update that price range for multiple weeks and stuff at a time. I mean, remember some of these cards that were extinct? Bruno Fernandez Headliner was extinct for such a long time. Other cards and Babu What If, like Road to the Final Cancelo was extinct at some point this year on the game too. And, you know, EA, if you're gonna be doing this stuff, if you're gonna want us to sell these cards for fair prices, then we also need to have a fair market where we can trade these players without being inhibited by price ranges not being updated. Okay, that's my biggest issue with the fair price comment that they have here on this game, selling items or trading players for a fair price, all right? Now, let's take it one step further. Now, this is on EA's website again on the be safe with foot coins and FIFA points. When people are asked what not to do with coins and people are asked how do I earn those coins? Here's what they have said. Oh, that's the fair price thing again. But I thought these were two different things, maybe not. The fair price thing is the one thing that gets me because just that hold that word, fair price. There are so many prices on this game that are not fair, right? Icon moments, Ronaldo. If you really want a fair price for this card, you have two options. You up the pack weight, so he's actually on the market inside of your threshold which seems to be 15 million coins for all cards in this game or you just up the price or you just up the price and let the market or the ecosystem as you call it work itself out. So that's why I think today a lot of the stuff that EA is saying is really it's two faced. It seems very, very two faced because they are wanting, that was a really good deal on bid right there in that sun. There's my economy monitoring skills coming into play right there. They are wanting people to be pros at the game but they're also wanting those pros to be able to know when to buy and sell cards and to make money. And I just think that EA saying that you can attain all items in the game without ever spending money with purchases being entirely optional and they're making that sound so realistic. That's the issue that I have with this because it is definitely not realistic at all. It is not realistic at all 100% and that's the issue that I have. So wrapping this thing up, I'm really curious to see what EA do on the end of this and I hope that there's some change from this whole thing in general. But again, as you heard EA mention it multiple times during that article and during the responses that they had to those questions, they keep saying this is for a limited number of items. This is a very small portion of the community. This is not happening. This is not widespread. So just the fact that they're using those types of words to describe what is going on in this game makes me think that we're probably not gonna see too many changes. We'll hear the outcome of this. Maybe pro players and content granting will fall under a lot more stricter guidelines and stricter policies and changes now where pro players might not be able to get their cards. Whoever was doing this whole icon moment stuff might have just kinda ruined it a little bit for pro players or celebrities that get their cards from EA, they might not be able to do that anymore. So that might have been ruined because of the whole situation but I really don't feel like you're gonna hear a lot change about this because honestly, EA thinks they're doing everything right and they don't need to change anything but hopefully this causes a bunch of just talk and hopefully something changes. Whether it's price ranges being updated more fairly, more quicker, because EA knows what they're doing with price ranges, right? They obviously have all of the stats and the knowledge on where prices or where cards should be priced in this game. Day ones of promos when cards are extinct, a lot of times 100% of time that's on EA sports because they should be making those price ranges high enough that when they first come on the market and get undercut, yes, there's gonna be some demand right there but sometimes they're releasing these cards at, you know, headliner, lasano when he first released was extinct at a super duper low price. Yeah, headliner, there was another card too Oh, Mbabu, right? Headliner Mbabu, or not headliner but the what if Mbabu who was extinct right away from the beginning because he got that upgrade, EA would have should have known they released that card on a day or like really close to the day that his team actually played so he was close to getting that upgrade and they would have known his price would have gone up if they would have got that clean sheet which they did and then his price went up and he was extinct. So that's the kind of issue that I have and I just wanted to kind of bring in this conversation today to talk about it, talk it through and just realize what is going on right now in FIFA 21 and how EA paints the picture this way to the media but inside of the game we see it a completely different way and definitely not exactly the same as EA push that to the media. So that's the video for today boys. Let me know your comments down in the description for real. Let me know what you guys are thinking about this. It's crazy to follow that this is even a thing like just to know that it's going on. So drop your comments down below. Of course, if you did enjoy it, hit the thumbs up and subscribe if you're new. It's been Nathan Vodiccount and I will catch you guys later. Peace out.


This is stock number 11287A. We are here at Summit Automotive in Fond du Lac, Wisconsin. You're new and used SUV headquarters. Today we are checking out this super clean 2017 Ford Escape with the S package. This vehicle has the 2.5 liter four-cylinder motor, pumps out 168 horsepower. This vehicle has been fully safety and inspected by our service shop as a fresh oil and filter change. All the fluids have been checked and topped off and this vehicle is 100% ready to go. Also has four brand new tires. Ignite Silver is the color we shoot all of our videos in 1080p 60 frames per second. So if you have HD capabilities on your computer tablet or smartphone device turn them on right now because it is definitely your best way to check out this one owner vehicle before seeing it in person. And if you'd like to check out all the photos on this vehicle in the upper right hand part of your screen is a link to our website. Click those and check us out there. It comes with the factory painted 17 inch rims and it has brand new Kelly Edge AS tires. These are 235 55 hour 17s and they are brand new. We put those on in our safety inspection. They have all the tread left. You can still see the little wear knobs on there. So four brand new tires. Front fender is in excellent condition. I didn't see any dents or dings on there. The headlamp lenses are nice and clear. Front bumper is in great shape. I didn't see any major dents, dings or cracks on there. And the hood is in excellent shape as well. I didn't see any dents or dings on that either. Passenger side front fender. Really nice condition and the passenger side rim. No scuffs or scrapes on there and that tire is brand new as well. As you go down this side of the vehicle, take note of how clean the body is, how reflective and mirror like that paint is. We take these HD videos so if you are far away or even if you're close by and you just can't make the trip down but you're still interested in purchasing the vehicle, you can see the vehicle, hear the vehicle and have confidence in what you were looking at before you even get here so that when you do get here, there's absolutely no surprises. And that is why we offer the Summit Auto Fastpass option in the upper right hand part of your screen. A link right to our website where you can apply for financing, get pre-approved, see a payment quote, even appraise the trade all from the comfort of your own home. So when you get here all you got to do is drive the vehicle, love it and you'll be all set to go. Back tire is brand new as well and this one does have the Capless fuel fill which is really nice and you do get four-wheel disc brakes on this vehicle. Looks like those back brakes are brand new and we probably did that in our safety inspection as well. As we come around to the back of the vehicle, rear bumper is in great shape. I didn't see any major dents or cracks on there. The rear gate is in really nice shape as well. And the shocks work really nice on that gate. Back storage area is very nice and clean and you do get a spare tire back here. Those seats do fold down four extra storage which I'll show you in a little bit. And that gate shuts nice and solidly. Down the driver's side just as clean as that passenger's side. No dents or dings on the rear quarter. And for full disclosure this back rim is in excellent shape as well. Down the rest of this side of the vehicle looks really good. Didn't see any major imperfections on the door there. Does have the power mirrors with built-in blind spot mirrors.  Side curtain airbags, no rips or tears on the seat. You do get factory floor mats throughout. This one does have auto headlamps, tilt and telescopic steering wheel, power windows, power locks and power mirrors. And as we hop inside the vehicle you can see that this one has, you will see that this one has 68,948 miles. Instrument cluster is very nice and clean. It does have an outside temperature display as well. We can dim those a little bit. Instrument cluster is very nice and clean. The steering wheel is in excellent shape as well. You do get Bluetooth and audio controls on the right. Cruise controls and information center controls on the left. Comes with the AM, FM, CD player. That's where your information for the radio is. There are your climate controls. It does have traction control. You can turn that off as the USB right there. And this is also where your backup camera shows up right there. It does have the sync system so you can connect to your Bluetooth phone. Passengers side floor mat and seat are in excellent condition as well. And you do get a sunglass holder and map lights up there. Windshields in really nice shape. I didn't see any chips or anything like that on there. We'll take a quick look at the back seats and then we'll start it up and take a quick look under the hood. Back seats are in really nice shape. Didn't see any rips or tears back here. It does have the latch child safety system for any child car seats you may have. Factory floor mats. These seats do fold down for extra storage like so. And just a really nice clean vehicle inside and out. Has a child safety locks on the back doors. Didn't see any corrosion on the bottoms of the doors. This was a one-owned local trade-in. And we'll start it up and take a quick look under the hood. And I would personally like to thank you for checking out the video today and hopefully from this HD video you will have been able to tell just how clean this escape is all the way around inside and out. Under the hood we have the 2.5 liter four cylinder motor. Engine bay is very clean, runs very smooth. And once again this vehicle has been fully safe and inspected by our service shop. Has a fresh oil and filter change. All the fluids have been checked and topped off. And this vehicle is 100% ready to go. I'll do one more final walk around here. I would highly recommend it from a quality and condition standpoint. And to see more pictures of this escape or one of our other 450 new and used vehicles you can go to our website which is www.summitauto.com. Full pictures and descriptions of every single vehicle from two locations all at summitauto.com. There's that website right there. And then if you want to check out more HD videos you can go to youtube.com slash summit auto. Remember to like, subscribe and share on this video and all the videos that you see there. In fact in a second you will see a link to subscribe to our YouTube channel on your left. A link to more Ford Escape videos like this one on your right. If you have not been to our website on the bottom, a link to this vehicle on our website, click those, check us out. And we really look forward to helping with this super clean 2017 Ford Escape. S package front wheel drive and Ignat Silver Metall. Thanks again for checking out today.


So, this spring, the National Academy of Medicine was asked to, or actually, they published a report, and the report was titled, An Evidence Framework for Genetic Testing. And the academies were charged by the Department of Defense to examine the relevant medical and scientific literature to determine the evidence base for different types of genetic tests for patient management. I think that's a quote. At the May Council meeting, there was a request from Council to have a report on this report, and lucky us, Wendy Chung, ended up being on that committee, she graciously agreed to present today. Thanks, Rudy. Okay. So, just to be very clear, this was obviously a committee's work, and I just want to give credit. Jonathan Berg and Dave Valley actually put this slide deck together that I'm using, so credit to them for doing this. These are the members of the committee, and as you can see, it's an illustrious group, and also in terms of, for those of you who don't recognize all of the names of everyone on the list, just to break it down by specialty, it was quite a multidisciplinary group, as was necessary, folks actually ranging from genetic epidemiology, basic science, and gene discovery all the way through really bioethics, health economics, and everything in between. And I can say I personally learned a lot from interacting with such a varied group. This is exactly the statement of task as we were given by the Department of Defense. This gave us a relatively, in some ways, narrow framework in which to work. This was to, as Rudy was saying, look at the scientific literature in this particular case, and to provide some recommendations to advance the development of an evidence framework, and then also to think about a decision-making framework. And so I'm going to go through those three different aspects that we were given in terms of our job description. The first that we needed to do was to be able to identify just the definitions, and I'll go through a couple of those definitions, and to think about the clinical applications and clinical utility. And then to think about the process by which evidence is generated, how you can actually aggregate that evidence, and then to have a decision-making process for being able to use that evidence. In some cases, as you'll see, there's a lack of evidence, which is not surprising, and that makes it difficult to make decisions when they're not informed by good data. So it won't come as any surprise, I don't think, to this group in terms of what the definition of genetic testing is. It was a pretty, I think, what we would think of as standard definition in doing this. You're really looking for whatever you're looking at, whether it's DNA, chromosomes, RNA, but something to be able to use in terms of hereditary or genetic issues. Very specifically, and especially just following along on this TCGA presentation, this was specifically not somatic mutations. So it was out of scope to think about cancer or somatic mutations. This was really focused solely on heritable things, and it was not ancestry testing or other things like that. It was for specific medical application and medical purposes. The three different use cases in terms of doing this, which, again, won't be surprising to this group, we're thinking about diagnostic testing. So once a patient is already symptomatic, predictive testing in terms of someone who might be at future risk, and being able to predict that based on, for instance, family history might be the reason for doing this, or population-based screening like newborn screening, and then finally reproductive situation. So when you're thinking about this, really, for the purposes of family planning. The one thing that was somewhat different about this, if I think about our Department of Defense task, as opposed to when we think about this more broadly, is the DOD has very specific use cases and very different value systems, perhaps, than you might have if you were doing this for a healthcare system. And so that very much constrained or defined what we were doing in terms of this. So just as an example, when it comes to reproductive issues, the DOD, in terms of their value system, might have a different sort of outlook in terms of thinking about reproductive issues and reproductive decisions than you might for a population at large. So in thinking about this, the evidence framework about thinking about this, there are certain challenges. In many cases, as I said, there isn't direct evidence for this, in part because I think we're still figuring this out. We're still learning about some of the genes for these. We're still learning about penetrants. We're still learning about what the implications are of interventions. So I think there are many limitations in terms of that, but I'm going to go through some specific limitations in terms of each of the three areas. So in terms of analytical validity, what I mean by this is being able to just literally, if I go into the laboratory and take a bit of DNA, for instance, how accurate am I, how analytically sensitive and specific and accurate and reproducible is this if I do this, especially when you think about from laboratory to laboratory in terms of doing this. So this is very much a technical aspect. Clinical validity is really thinking about, for instance, if I take all the women with breast cancer, what's the percentage of those women who are going to be positive for BRCA1 or BRCA2? So thinking about that clinical validity and how well established is that connection between the gene, for instance, and the disease, or that in terms of a clinical use case. Clinical utility is really thinking about how do I take that information that I'm getting back from the genetic testing and how do I really use that then in terms of care, implementation, improved outcome, and clinical management in terms of that particular genetic result. So with doing this, there are some challenges when we think about this. So when we think first about analytical validity, and I can say this in terms of my own experience with running a clinical laboratory, one of the issues is that being able to simply have gold standard samples to be able to test your assay out, being able to actually have validated samples to be able to make sure that your assay can detect what might be technically a very, very difficult mutation to detect for whatever reason. Being able to have these, some of these are available, for instance, through Coriel or other biorepositories, but we don't have nearly as many of these as we would like to have. By being able to have unbiased data in terms and data sets to be able to analyze and from technically, from a technical point of view, there can be technical issues when you're changing over assays from one to another. So some of the evidence that comes from this is what we do all the time in the laboratory when we do, for instance, cap testing, when we actually exchange samples and can do this in a blinded way. That's some of our best data that we get from this, but one of the limitations or the gaps are that we often don't have this aggregated in terms of data, single data sources where we can go back and be able to query that. In terms of clinical validity, again, this is sometimes difficult because we're still in the process of discovering the genes for diseases and being able to know how to interpret the large number of variants that are being identified. And so this is just something that we will continue to produce as a scientific community the evidence base in terms of knowing about things like penetrance, variable expressivity, and we've already heard about things like emerge that will start to be able to answer those questions. But this is still very much in the works in terms of being able to come up with this. This oftentimes, again, when we think about this, how do you aggregate the data or their databases that can effectively consolidate this information and use it in a user-friendly way were some of the challenges that we think could be dealt with. And then finally, in terms of clinical utility, this is perhaps in some ways the most difficult to be able to aggregate because there's a lot that's going on in clinical care and clinical medicine right now that is impacting care, but we're not simply collecting the information on that. So there's a lot that's going on, for instance, with carrier testing that's being used a lot, cancer testing, I just think of as two things that are very much in terms of clinical care, but we're not aggregating that information. It's difficult to be able to do across health care systems to be able to understand, in fact, how much that's actually changing clinical care or what that clinical utility is. And so because of that, we oftentimes have a chain of indirect evidence that's based on siloed data, that's based on, in particular, anecdotal cases and sometimes, but we don't have as much as we would like. The gold standard for doing that are randomized clinical trials in terms of being able to get those data together, but those are very difficult, costly, and the question is, in part, who funds those clinical trials? When you think about clinical trials for a drug as an example, it's a pharmaceutical company that wants to bring that to market to be able to pay for those studies in terms of genetic testing laboratories just as an alternative. They have historically not funded such RCTs, and in fact, that hasn't been necessarily part of the culture or financially something that's been viable. And so that's been a difficulty in terms of being able to have those data. And so oftentimes, this is data that are simply lacking in these cases. Finally, I'll just say that one of the things the committee thought was important is that there are certain ways in which some of us have used the term personal utility, but the way that information is used that's not necessarily directly impacting clinical care, but simply as an example for many people with undiagnosed disorders, being able to end the diagnostic odyssey can be very valuable. We do, as a value statement, believe that that has value and that has utility, but that's not necessarily something in a merge, for instance, that we can pull out in terms of a decision that's easily made. Even though that decision, for instance, might obviate the need for further testing, further additional diagnostic testing, that's not necessarily something that's already collected. So in terms of looking at this and trying to make specific recommendations about what to do, and again, the context for this was very specifically for what can the Department of Defense do, and just to put some information around that. So the Department of Defense, when you think of their purview, includes Veterans Affairs, so all of the veterans, and then it includes the family members. Many of them are getting their care literally from the VA hospital system, but some of them are also going and getting their care throughout the entire United States through TRICARE. And so they're getting that, and they could be coming for me, for instance, even though I'm not employed by the Department of Defense, but they could be going outside an official sort of DOD healthcare system, but insured through TRICARE and getting their information, getting their care in that way. But again, when we're thinking about specifically for the committee what to recommend to the DOD specifically, we would like them to, or our recommendation was for them to try and use whatever opportunities they had to take the information that they already have within their healthcare system, and to be able to use that information to drive better data, to be able to aggregate that, to use that to essentially do a self-study to understand what information could be used for them to improve clinical practice and clinical outcomes. So within this, the high quality things that we would like to see are RCTs, but we realize that that's difficult and not necessarily always practical, and that as we're doing this, we wanted to be able to collect some of these data streams that aren't necessarily usually collected, such as that clinical utility for personal utility, and then being able to also support discovery efforts that would be able to solidify those relationships between variants and those phenotypes. One of the biggest things though is being able to establish that infrastructure to be able to track and to collect those data, and I think that's many ways the most difficult thing to do, because like I said, it's not simply within the VA system, although the VA system is the easiest place to start collecting those data, but really for the DOD, it extends very widely throughout many other healthcare settings, and to be able to collect that all into a searchable database would definitely be one of the ideal things to do. So we tried to think about what else could you do, so besides just querying the VA healthcare system, are there other things in terms of claims data that are coming in, or are there partners with clinical diagnostic laboratories that are routinely used, where you could simply subset and take a slice out of if you're using LabX, be able to take all the members that are serviced by LabX and at least take out and one fell swoop all of your data from that laboratory. Those were some ideas in terms of being able to access those data and aggregate the data for the DOD. And in terms of that being able to document in what ways that information was used in clinical practice and being able to measure outcomes in terms of patient outcomes, at least for those patients served by the DOD. Within this, we also realized that there were many gaps and opportunities, and that this would require, in fact, being able to think forward in terms of some of these things. And so we wanted to make sure that the DOD was at least using the things that the rest of the genomics community had already built, the infrastructure that was available. So for instance, making sure that variants were deposited in ClinVar as an example. One could, for instance, mandate that you were a laboratory that was going to be paid for by the DOD, that that laboratory would necessarily have to donate their data to ClinVar, that at least we were trying to get data aggregated in the larger sense for the community simply by using the power of the purse. And in a similar way, if there are other databases that were developed, that that would again be a requirement in terms of any laboratory used by DOD participants. But to realize that any of this that was done was simply not something that Veterans Affairs had to deal with, but this was part of a larger system in terms of other funding agencies and HGRI, being one of them, PCORI, AHRQ, but other funders who could be able to support this kind of work. So the last of the tasks that we were trying to figure out was that if you, in a very practical sense, had to, as for instance, being at the DOD had to decide for a new test that was being suggested. If someone came in and made a request to be able to cover this test in this clinical use case, how would you make a decision about, yay, nay, do you cover, do you not cover? So as we thought about that, we realized obviously that many others have thought about this before, both in the United States as well as abroad, so we went through a systematic review for all of the other systems that have been in place to be able to evaluate these. And I'll go through these in sort of step by step, but just very briefly that in doing so, you would have to think about the specific use case under which this was done. You would need to think about some things very practically in terms of just how much was this burden? Was this a test that was going to be done as a one-off one person ever? And so you might take different levels of sort of energy, intellectual energy, and going through that use case, then if it was something that was going to be used in terms of financial burden with a significant portion of your budget, something like that, you should really take the time to do it very thoughtfully and very carefully. And that when this is done, you want to be able to use, if others have already evaluated this, you didn't need to reinvent the wheel. You needed to simply be able to go back to those prior evaluations. And similarly that if your group took the time to actually make a decision that that information was transparent to the rest of the community, so that others didn't have to reinvent the wheel if the group doing this at the DOD had already gone through and made a very thoughtful decision about this. So within this, and I'll go through this step by step, but is again defining the problem, doing a triage. Do you need to actually do the review? If you do need to do the review, going through making a decision about this and putting that decision in a repository and making this very transparent, so that for instance, if others wanted to be able to understand the thought process by this, that this was something that would be readily available, both in terms of patients who are covered in terms of the DOD, but also the rest of the community. And importantly that this would sometimes need to be revisited and revised, that certainly we're not in steady state in terms of the information or clinical utility. And so that there may be certain circumstances in which you might for instance, make a triage decision about no, but then that decision might be changed over time as things evolve. And as we're doing this, as I said, to be able to make this information accessible to the community. So the first thing which is important, and I don't think people necessarily think about as carefully, is when you're thinking about a genetic test, it's not just the genetic test itself, but it's also thinking about the clinical circumstance in whom that genetic test should be applied. So as an example, if you're thinking about hereditary cancer testing, there's one use case which you're talking about, people who have already been diagnosed with cancer in a diagnostic sense, but there's another group of people when it's predictive. Those individuals who don't yet have cancer, and those two scenarios might be very different. Or if you're thinking about reproductive testing, for instance, with noninvasive prenatal screening, there's one case if you're talking about women who are over 35, and that might have different parameters and different utility, then for instance, you're thinking about taking all women regardless of age. And so being able to define those in terms of the specific tests being performed, all of the analytical factors that go into that specific test with that specific technology that's used, as well as, as I said, the clinical scenario. And so being able to define that is important because one test, different clinical populations might have a different outcome. The next in terms of trying to think about this is is it worthwhile? Just sort of very quick triage, is it worthwhile in these particular circumstances if it's very clearly not, if it doesn't pass that SNF test, you can just sort of triage it immediately to a no. But if there is, if it does appear to be worthwhile, is there already evidence for this? And again, if that evidence already exists not to reinvent the wheel. And then to be able to think about, as I said, the burden in terms of the total cost for doing this. And that's an important decision, especially in a cost constrained system as the DOD had been in. So thinking about this, if there is insufficient evidence already, the idea is to be able to do a triage for a rapid review. If it's very clear upfront, being able to make a decision within that. And in some cases, as I said, it may not really be worth the time and effort to do the review if it's just easier because it's one case and it's sort of just, it's obvious that there might be some utility to have a triage individual who could say yes, when in fact we'll cover it in this one particular case. If there ends up being a greater demand in the future it might require more formal review later, but for this one case, we would go ahead and approve it as an example. Within this, as the review actually came to doing a very full study and analysis in terms of a formal review for doing this, thinking about the outcome of the test and if there's any other alternative besides the genetic testing that might be used and then to approach something like an E-CAP sort of process in terms of being able to go through that. As this is done, being able to think about the potential harms if it is not approved. So being able to think about the cost and benefit from this analysis and then being able to set values in terms of the healthcare system as a portion of making that decision. And then as I said, being able to make sure that this is publicly available in our repository so that this is something that others can be able to use and ensure consistency across those decisions that are made and then be able to make sure that this is something that's iterative and that continues to learn as information changes over time. So with this, I think one of the things that we realized is that there's oftentimes a data gap being able to make some of, inform some of these decisions and that continues to be one of the challenges to be able to do this. And so it's difficult, especially in the time frames that certain decisions need to be made to be able to have that evidence in place to be able to do this, but being able to support systems for gathering that data is perhaps one of the most important recommendations that came from the group. So this is simply a summary of what I've just stated, but being able to do this, we hope would provide at least the evidence framework in a very rigorous, robust way of being able to think through this, assuming that the data are available to be able to inform that. I think the challenge for many of us is thinking about how can you sort of build the plane as you're flying it and being able to think about how do you collect data when things are changing so rapidly and you need to make decisions on the fly as you're doing this. And I think that's the challenge that's in place but being able to set up those data gathering systems that require the least amount of effort that just happen automatically and aggregate data essentially passively was one of the things that would be most practical in terms of being able to come out of this. So I will stop there and I'll be glad to take questions. Yeah, yeah. I think it was you. Oh, it was me. So wow, that was, that's a big report. I downloaded it and I'm glad to have your cliff notes. But I also am concerned about a couple of things and I want your opinion. I mean, first of all, this seems to indicate that, wow, we're still, there's a lot that is not known. Well, I guess that's no surprise to this group. But, and it's also very hard to get funding for systematic reviews. I mean, in fact, really excellent systematic reviews. You can't get funding. It's also hard to publish them when they don't find much as I've known myself. So given that, I guess the bottom line here is just sort of on the quick and dirty you would passively make sure that you're able to collect information as it exists now, right? And just to keep track of it all. But that's not very systematic and it's not, and also that I'm not quite sure. I guess I think the population you're talking about is military population. Right, the military population. And of course, that's a huge population but it's also got some biases. So I guess I wanted, you know, comments. Am I right in thinking, and my general kind of, whoa, this is really, this is tough. Yeah, no, I mean, it is very tough. I mean, this is the difficult part of this end of the implementation pipeline is really being able to have, to me, it's being able to have systems in place that are automatically gathering the data that you need to be able to inform the evidence basis for review that you need to do. Quite frankly, I mean, I think the review of the evidence, I agree it's difficult to get funding. I think you're gonna have to have committees that are actually mandated, funded, supported to be able to do that. I think the greatest challenge is having the evidence in place to be able to do those systematic reviews to be able to look at efficacy. And I just frankly don't think we're gonna be doing many, if any randomized clinical trials to be able to go head to head, if you do genetic testing, if you don't do genetic testing, that type of scenario. I think it's gonna be at this point, just as an example, I don't think we have as much of the evidence that we need to dictate some of the practices that are already in clinical practice with sort of a hunch, but without as much rigor and as much evidence as we would like to have. And ultimately, payers are gonna push back and that's going to get us back to the point where we need to be able to show me the money, right? Show me the clinical utility, the outcomes to be able to do that. And so the challenge is in terms of who funds that, no one's jumping up to fund it in the biggest way that I think needs to be done. So the question is where do you start at least? What are the pain points? Think about in terms of the pain points, at least being able to tackle some of those and do that as a mechanism for triage and prioritization, perhaps. And just one extra thing, I think the most difficult is when you're dealing with healthy populations, generally. I mean, there's certainly data, whether they're perfect or not, but nevertheless, most of the data are collected from families that already have these conditions. But- Well, exactly as you say, for healthy populations, the problem is you're waiting a long time for an outcome of something bad to happen, and it's oftentimes taking a long time to get to that endpoint. Right, it's the penetrance issue. Like, let's just follow people till they're 100, right? Right, right. Thanks, Wendy, that was a great overview. So I did notice in there the internal problem of complaining about lack of evidence of clinical utility. And I think I quickly saw that as an incentive to generate some of that evidence, there was provisional coverage, what I might call coverage with evidence development. Right. You'll give provisional coverage. But as Gail mentioned that for healthy populations, there's such a long lag time that doing prospective studies is difficult. So can you say a little bit about that whole idea of using modeling? Was there acceptance and more discussion of sort of lowering the evidence threshold and more acceptance of use of modeling? You know, we actually, interestingly enough, we didn't even talk about that. You know, we were going very specifically on, you know, being able, not projections, not modeling, just actual data to be able to inform those decisions. It's great suggestion, we didn't even tackle that. When a new law is passed, often, not always, the first paragraph of the law exempts the federal government from the law. Do we know if military DOD employees are protected by Gina? So did the committee cover protection? That's a big issue clinically that individuals in the military are not protected by Gina. We can argue as to how strong Gina is, but there is some language that's similar, but still it's not Gina. This is a big issue for teenagers who are going genetic testing who are considering careers in the military. So my question is, did the committee address the use of genetic testing to discriminate against military employees for employment and promotion? So that would have been out of scope for us in the sense that it was just specifically focused on what is the clinical utility, not worrying about, you know, what your own personal concerns are in terms of privacy or discrimination. So medical geneticists deal with this daily, weekly, on an individual basis when they order tests for a patient. And they often have to defend it to the third party payer of why they ordered a test. And one reason that third party payers don't readily accept is that this will be useful for other members of the family. And was that addressed as a clinical utility issue? Sure. Yeah, so we did address that as a clinical utility issue. And just as an example, or to give it a little bit of more color for this, one of the issues that the DOD had been worried about is that you could be, for instance, a member of the military and go and request this test at whatever center and that someone there would review it and say, no, this is, you know, I'm denying it in this particular case, someone else in Arizona could go with a similar sort of situation and their local person could approve. So number one is there wasn't consistency across things. I do know, and I happen to know some of the genetic counselors who are involved in the VA system, one of the problems they oftentimes have is that the person who's informative in the family might even be a male, as an example, for hereditary breast or ovarian cancer because they're closest to it. And they try and justify oftentimes that that person should get tested in service of another family member. And it's been very consistent that that's not been covered in terms of things. So you're right that at least the way practice is going now, that's not, we viewed that as the clinical utility, really thinking beyond just the one member and thinking about the family. So yes, that is part of the clinical utility that goes through this. It's always this tension, though, between whoever the payer is and whether their particular member is the one who's being served or if you can have a broader view in this, so. Thanks. Okay, thank you, Wendy. Rudy.


Similar to this, we have added the ability to disable inactive users. So not only can you set expiration dates on a particular user who you want to expire after a certain amount of time, but you can also set up a job in the scheduler application to automatically disable users who have been active inactive for a certain number of months. I will demonstrate that here now. We can see this is the new scheduler application, which I'll get into in a minute. But I'm going to go ahead and just create a new job. This is going to be called disable inactive users. Disable inactive users. And I can select the new job type, which is disable inactive users as well. I can set this to run, let's say every week. I'm going to insert that preset here and I can then select a number of inactive months. This can be anywhere from one to 24 months. And this will disable any user that hasn't been active in the DHS to system for that amount of time. So if I set this to 24, any user who hasn't been inactive in two years will be automatically disabled when this job is run, which happens every week in this case. If I then made this one month, it would basically disable any user account when that user was inactive for one month. So you can tune this as you'd like. You can then save that job and it will begin to run every Monday at 3am as I set it to run and that will disable any users who have been inactive for one month at that time.