Good evening everyone, Joe for jaspyscasebreaks.com coming at you with Jaspys 10 box post draft NFL mixer. It's a random team break featuring those boxes right there. Some pretty solid stuff. There's one box that wasn't pictured. If you had an eagle eye and you were like, Joe, there's only nine boxes in there. Where's the 10th one? All right there, not pictured right there. So hopefully everyone read the item description and all card ship too, nice. And there was an early bird special. And I promised to give away $200 a break credit if it filled tonight. Man, all sorts of things happening here. This is 200 bucks a break credit randomized to one person at the end of the break along with any other randomizers. And that's breakcredittojaspyscasebreaks.com. All right, now there's the early bird special. So the first 16 spots right here in the break have a chance at an extra spot. So let's just grab these names right here and let's see who gets that extra spot. It's up, Eddie. Yes, I think we all do. Two and a four, six times. Name on top gets the extra spot. After six, one, two, three, four, five. Name on top, James Majority. Majority rules. He bought the first spot. There you go, James. Thank you. Now let's get the team list going here. So there's all the names here. All 32 are in and let's roll it. Let's randomize it three and a two five times for each list. All right, good luck. So after five times, we had James's early bird spot down to Matt Buckler. And after five times here, we've got Denver Broncos down to Tennessee Titans. Wait, who had last spot? Oh no. Oh, this is disaster. Did you only buy one spot, Ryan? Then it would be easy. He only bought one spot. All right, there you go. That could have been disastrous. Gotta have that last spot, Mojo, or the break just doesn't work. James with your early bird spot, the Denver Broncos, my rivals. Trevor with the Rams, Daniel with the Pats, Tom with the Ravens, Kevin with the Steelers. Trevor with the Jags, Tom with the Giants. Ryan last spot, Mojo star next to his name, Dolphins. Trevor with the Falcons, the dirty birds. Ryan with the Browns, Bucks for Karen. John with the Texans, Scott with the Bills. James with the Saints, Tony with the Chiefs. Derek with my Raiders, I guess soon to be Vegas Raiders, Las Vegas Raiders. Matthew with the Vikes. Scott with the Jets, Dino with the Redskins. Trevor with the Cardinals. Trevor with the Chargers. Gary with the Eagles. Chris with the Seahawks. Charles, Lions, Daniel, Colts, Jeremy. With the Panthers, Marissa, Bears, Barry, Packers, Chris, Niners, James, Cowboys, and Matt Buckler with the Tennessee Titans. All right, and just for, I think there's only one box that this would apply to, that Leaf Metal box all the way back there, but they're in their college gear in that 2020 Leaf Metal box. So that'll go to the teams that they are currently on, or if they were undrafted free agents, they would go to the teams that they were signed by, post draft, and if they're still, I guess if they're still scrubs, if they went undrafted and unsigned, number one on the list will get the break credit, and then two on the list will get the, when I re-randomize the names, will get the undrafted and unsigned person, poor guy. All right, I'm gonna pause the video and allow for a little bit of trades, trading time, and when we come back, we'll see if there's any trades, and then the break. All right, and we're back, and we have a trade. Matthew and Marissa have made a deal. Let's make a deal. So Matthew is out of the Viking spot, and into the bear spot. Matthew's saying he's a huge bear's fan. All right, sorry, Matt. Marissa in the Viking spot now, so we'll put T next to their team names for trade. So if you're following along at home, you're like, oh, so-and-so drew first blood in that trade. We can figure out who the winners are.  Or maybe both hit, that'd be ideal. Maybe none hit, that'd be sad times. All right, well, let's do this. So there's one, two, three, four, five, six, seven, eight, nine, 10, perfect. Good luck, everybody. We got one quick, so this would bring us pretty much to the end of the night. So we'll have one more quick break to do at the very end, which would be a one-box inception break. We'll give away a couple of transcendent spots, and then we'll call it a night. And then we'll be back tomorrow at two o'clock Pacific, five o'clock Eastern, and we'll do this madness all over again. Speaking of the Saints right here, Saints are gonna have, do I have enough space for this? I think I might. Saints have a new backup quarterback. James Winston, the famous James. Remember that time James Winston wanted us to eat Ws?  I remember. Good luck. Eddie saying, pay 500 for Patriots. I'm not sure what that means. It's cryptic. I wonder if there is, and there's no punctuation, is it, is it a question? Is it a statement? All right, box one of 10, good luck. There's MJD, Zac Proce to 199, and Jordan Lynch for the Bears. So it's Matthew Vosper, who draws first blood in this trade to 149. Mike Evans, wow. Rookie auto Mike Evans for Karen Steele and the Buccaneers. There you go, Karen. Five out of 25. If we're hanging in there, Karen, thank you. Spending some time with us. D'Angelo Williams to 75. You got Philly Brown to 25 for the Panthers. And Rajan Neal, Packer's autograph. That'll be for Barry Roberts to 199 on that one, Barry. Barry, Aaron Murray, rookie auto for the Chiefs. Rookie Relic for the Chiefs to 399, no auto. Tony with the Chiefs. All right, let's stay in the past. Let's get this going here. I think the last few boxes I'm gonna do, remember when they used to give you a checklist here? Last boxes I'm gonna do the two first off the lines and then the metal, I think would be a good way to end it. And I see this redemption right here. I wanna say it was Ryan Redman who gave us the scoop and said that for an expired redemption, he recently, they're giving points for expired redemptions. So that, oh no, I gotta look this up. Come on, maybe CardboardConnection.com will have this. 2011 Crown Royale football. Oh wait, didn't I just have a checklist? Yeah, Rebel, good call. I mean, I forget that they even do this, so I was even thinking about it. Steven Ridley, was he a Falcon or a Patriot? I don't know, do you know Redman? Yeah, back in 2011 he was a Patriot. There you go, so I think it was you, right, Ryan? Redman, that was telling us that points and points are being exchanged for these expired redemptions. Here's Karen, hi Karen. Your CJ Spiller, that's for the Bills, Buffalo. That's to 50, that'll be for Scott. There's Fred Bolitnikov, royalty relic for my Raiders, S-299, that'll be for Derek Melanson. And Redman is saying for those expired redempts, you actually kind of have to email and kind of get in touch with them and talk to them and everything, but that's better than what the original deal was.   That's it, your SOL.  Obviously I think if it's for bigger name players, you'll probably get some larger amount of points. If it's kind of a scrub, you probably won't get a lot of points. All right, what else should we do? These are all kind of a little light right here. Why don't we go into something a little bit heavier right here, certified. Is that CJ Spiller jersey number? Did I miss it? No, just one off the jersey number. Close though. I'm usually one off the jersey number. I never hit those jersey numbers. Rarely hit those jersey numbers. I don't know how I marked myself with that orange pen. When did that happen? When did that happen? What's going on? Yeah, I agree. Ryan Redmond's saying, yeah, they don't want to lose customers over something as silly as expired redemption. I feel like people have made legal arguments for that as well. You can't have gift cards expire. So you got to get something back in return at least for something like Redemption's, even though I know it's clearly stated all over the place, but I think it's just a good call. The points aren't horrible either. Every once in a while they'll do like these Kaboom packs and that you can redeem for points and something nice, better than nothing I say. Out of 100 certified gamers relic for the Steelers, that's going to be for Kevin Carey. It is 925 on the West Coast. There's Chase Winovich to 199. Snell yeah, Benny Snell Jr. Remember that autograph he wrote? Inscribed Snell yeah. To 99, that goes to Kevin again and the Steelers. There's James Winston just ended up on the Saints yesterday, which I think is good. Chris Spielman for the Lions. Can marinate behind Drew Brees. Learn a thing or two. Wow, Steelers box to 199 Jersey and Autograph. Another one for Kevin Carey with Steelers personal box here. There's L.J. Collier to 399. And we got Danny Amondola, Lions edition, 11 out of 15. I don't remember Danny Amondola, Lions edition, but it says so right there, so it must be true. It says signed by Lions on March 11, 2019. I don't remember that. Went through the entire 2019 season. Not remembering that. Absolute. Everyone, I've been watching the, I know it's a football break, but everyone's been watching the last dance. It's been pretty good. There's one interview, there's one interview shot. Got my long hair falling over here. I need to get a haircut. I'm just gonna shave my head. There's a shot, one of the particular interviews that Jordan was doing. He was sitting at a, sitting on his chair with a cigar and a glass of gold liquid near him. That's in various stages of being, being full depending on the cut. There was Teen Tandem, 27 out of 199, Meekle Harman Jr. and Sammy Watkins. I wanna say I heard that that was a, a tequila brand that he, there's Alex Barnes with the Titans. That's to 50, that'll be from Matt Buckler in Tennessee. To 100, Saquon Barkley. War Room, Ryan Finley. One of one Ryan Finley. Sorry, it's for chuckling Kevin Carey with the bangles, but unfortunately he's a little irrelevant now. I guess you're the bangles, third quarterback, practice room, practice squad, QB. Here's Ryan Finley, but nice Nike swoosh nonetheless. And out of fives and under, get the, all aboard the Big Head Express. Woo-hoo, there's Jarrett Stittum. One out of five so far, he's still your QB one. Another train whistle for the Pats. That's gonna be for Daniel Hayes who got randomized the Patriots. All aboard, woo-hoo. Red Zone to 100, Austin Bryant and Irv Smith Jr. Ooh, you even got a Smith out of there? His autograph has been shrinking all season long. Marissa on the other side of that trade from earlier, 42 out of 99, Triple Relic and autograph. Nice, and there's Shady. And let's get into some Stratta, some Supreme, some Origin, some First Off the Lines, and then we'll close out with the Leaf Metal. All right, so this is what, 2015 Stratta football. So your rookie Otto here is Trey McBride to 800 Tennessee Titans. I kinda like Stratta, I don't know why they, they left the Stratta brand, but for now it's, they shelved it for the time being. Matt Buckler with Tennessee, and Jersey and autograph Chris Conley. That will be for Kansas City, that's gonna be for Tony. This is 2014, Supreme. That's another product line that Topps shelved. They had Supreme Baseball too and put it on the shelf. Might bring it back, but. All right, there's all Sean Jeffery to 144. Bo Jackson to 50. Devonte Adams to 162. And it's just a Relic only. It's a Quad Relic, Judevion Clowning. Quad Relic, two out of 36. And now Clowning's got a bruised shoulder. That goes to John Williams and the Texans. I gotta find that soundbite. All right, let's go to 2018 Origins. Oh yeah, goes this way. All right, Todd Gurley for the Rams in 99. Russell Wilson for the Seahawks at 299. Michael Gallup is your on-card autograph. That's Galloping over to James Teal and the Dallas Cowboys. He should have a lot of room to operate next season or this upcoming season. Mark Walton, two-color patch and autograph. Kevin Carey with the Bengals. And two out of 25, Aaron Rodgers. Rise to the Hall Relic. Nice, he's about to get replaced. Maybe not for a little bit, but Packers are planning ahead. And it's pretty smart though. Barry Roberts with the Go Pack Go. All right, let's take a look at this. First off the line, certified. Two autos and two memorabilias. Good luck, trash can's getting a little too full. I think I can make it though. Rebel would have loved to be a fly on the wall in Rodgers house during the draft. Can you imagine? Yeah, cussing up a storm, absolutely. I was gonna say there's probably a lot of F-bombs involved in there, Dino's right. I can imagine, I don't think Aaron Rodgers, maybe probably doesn't watch the entire draft, but he's like, hey, Danica, let's kick back, let's enjoy a glass of wine. Let's see what the Packers are gonna get for me in the first round. You know what I mean?  There's Mel Renfrow, 28 out of 50, nice.  All right, all right, maybe one of the, it's a deep receiver class. Here's Damien Harris, one out of 20. Freshman fabric, Pat's. That's gonna be to Daniel Hayes. And then Goodell from his basement goes Jordan 11. They're like, what? Royce Freeman, the 75, certified gamer's relic for the Broncos, that'll be for James. It's the spot that he got from his early bird spot. And new generation relic, Jared Stidham, the future for now, unless those Newton rumors become a little more fire than smoke. It goes to Daniel Hayes and the Pat's. And there's Derek Carr, the 25, it's gonna be a big year for him. He's got more weapons to play with, so it'll be kind of a big, prove it year for him. All right, first off the line, select. Todd Father's gonna be a big year for Trebisky, he says, for Todd Father's Bears. Well, Bears did say that there was, that there was a, that's gonna be open competition, right? There's Christian McCaffrey, Tiger print, Erv Smith, Jr. to 175 and Dak Prescott. Nice, and Carson Wentz. Some nice silvers there too. It's not gonna be a good sign if Nick Foles gets beat out by Trebisky, I'll tell you that. It's an open competition, allegedly. The Tiger print, Christian McCaffrey goes to Jeremy and the Panthers and the Silver DK Metcalf. Premier level is gonna go to Chris G and the Seahawks. This is first off the line, so there's lots of special stuff here. Dexter Lawrence, Derek Henry, field level, Randall Cobb, Tri-Color to 199, Dwayne Haskins to 11, and Patrick Mahomes. I'm just gonna sleeve all of these just so, they know to top load all of these. We got Kenny G, Silver, Devin Singletary, Josh Jacobs, White Prism to 35 and to 75, Marquez Valdez, Scantling. Who's got my Raiders? Derek Mulanson with the Raiders. No, hoping for no sophomore slump, knock on wood if you're with me. No Paris Campbells yet, Steven Pung. Jack's hoping, let's hope that it's just gonna be a year for Trebisky. There's another DK Metcalf. Nice, Pat Tillman, strong, Adam Thielen, Justice Hill, and Jaws. Ron Jorsky is your autograph for the Eagles. The Fly Eagles fly, Gary Harton, with a 51 out of 99 Jaws. Justice Hill for the Ravens, that's to 149. Can't fight the Thielen. Goes to Marissa, of course Pat Tillman, Trevor, and the Cardinals. And you've got Gardner Minshew. Looks like he's the guy in Jacksonville. Juan Thornhill, rookie silver for the Chiefs. It's gonna be for Tony. Saquan Field Level for the New York Football Giants. Jalen Hurd, not Jalen Hurts, but Jalen Hurd, 18 out of 175. Niners, Chris. And another panther, Jersey and autograph, Brian Burns. Some cat team action there. Going to Jeremy, to 23, Jordan. And Hop Texans. All right, your last, your 10th and final box here in this post draft mixer is gonna be a box of 2020 Leaf Metal Draft. So they're still in their college gear. We're gonna look all of these up and make sure they go to the team that they were drafted by. Hopefully they were all drafted. If they were not drafted, then they, and they were, if they were not drafted, then they were signed, it'll go to that team. If they were still undrafted and not signed, we'll randomize it as a lot to one person to break. Hopefully there's no more than one card in there. All right, good luck. We won all first rounders here. There is Claiborne Chasen, I know he was drafted, five out of 35. Claiborne Chasen was the 20th overall pick by the Jaguars, linebacker, Jags. He went from one cat team to another cat team from LSU to Jacksonville, Tigers to a Jaguar. That will be for Trevor and Jacksonville. Next up is touchdown King, is Deandre Swift. I know he was drafted too, 27 out of 40. That's good. High second round pick, second running back drafted, went from Georgia to Detroit. Charles Teal with that one. We got Shea Patterson, five out of seven for Shea Patterson. I don't think he was drafted, but I think he was picked up. No, he's not drafted and not signed. So name on top, we'll get the break credit. Second name, we'll get the Shea Patterson, the randomizer at the end. We've got Grant Delpit, two out of four. That's a train whistle hit. Delpit is a second round safety. Went to the Browns, and he's out of LSU as well. That goes to Cleveland, that's Ryan Bartow with the Cleveland Browns. And it's out of five and under, so all aboard the Big Hit Express. Whoop, whoop, and we've got Jake from State Farm. Jake from, who I think went to the Bills, right? Buffalo Bills, fifth round pick. Buffalo! Just a couple off the train whistle, right there. All right, and there you go, nice 10 box break. Thanks for filling it, everybody. All right, let's actually get a blank list here. New dice, new list, there it is. Trevor down to Dino and everybody in between. Let's roll it, let's randomize it eight times. Top two, get something, after eight. One, two, three, four, five, six, seven, and eight. After eight times, second person will get undrafted, unsigned, Shea Patterson, Charles Teal.  Could be a late bloomer. $200 a break credit, going to Marissa Boyce. There you go, Marissa. I don't think I called your name out too often in this break, but $200 a break credit going your way. Thanks for watching, everyone. I'm Joe for jaspyskrakes.com, and I'll see you next time for the next break. Bye-bye.


Rishabh today. He got his PhD from Columbia University in 2021 and he is now a postdoc at Harvard and is going to present his work on spectral universality joint with Subabrata San and Uelu. Rishabh, take it away. Hi everyone, my name is Rishabh and I'm sorry I couldn't join in in person because of some of these issues I'm not able to travel outside the US but I am working on it and I hope I'll see you folks in other workshops and conferences. So today I'll talk about spectral universality in high-dimensional estimation. So this talk is going to be about an empirical phenomenon that people observe in high-dimensional estimation problems and some work I did with my postdoc advisor Subabrata San and Uelu to try and understand what's going on here. So let me begin by introducing a canonical high-dimensional estimation problem and so roughly speaking the problem will involve recovering an unknown signal from noisy non-linear measurements. So the way we are going to set up the problem is that there's this unknown signal beta star which is a vector which we would like to recover and we observe measurements y and the way the measurements are generated is that the measurements are a function of the unknown signal and this function is a composition of two parts. The first part is linear and is specified by a design matrix x and depending on which community you come from you might also call it the feature matrix or the measurement matrix or the sensing matrix and then there's a non-linear part which is like a link function and it's a function from r to r and it acts on the vector x v star entry y and you might also want to model some noise in the problem and so there's this epsilon and so many in many applications where you want to recover unknown signals you can cast the problem into a problem of this form. So for example in MRI applications this g function is just the linear function identity function and then applications like x-ray crystallography this g is the square magnitude function. So before I tell you about the empirical phenomenon which is called spectral universality let me tell you what universality means in a broader context. So many people say universality they basically talk about they're basically talking about it's a high dimensional probabilistic phenomenon but what seems to happen is that many large stochastic systems seem to behave asymptotically the same even though the exact constructions might be very different. So maybe one canonical example is the central limit theorem which says that if you have a bunch of IID random variables with mean zero and unit variance if you compute the normalized average then it converges to a Gaussian distribution and the limiting distribution is not sensitive to the exact distribution of random variables that you chose and so that's why this is a universality result. Yeah sorry somehow it gets of my computer keeps getting confused. Give me one second and maybe even more striking examples can be found in random matrix theory. So in the three plots that I'm showing you I'm plotting the eigenvalues of a large matrix so this matrix is not symmetric so its eigenvalues are complex valued and so they have a real part and an imaginary part so I'm plotting a scatter plot of the real and the imaginary part. The first one has the first matrix is a IID Gaussian matrix the second matrix has IID plus minus one entries and the third matrix is not random at all it's just populated by the digits of pi and so the point is that the global behavior of the eigenvalue seems to be nearly the same in all the three cases that in the sense of the eigenvalue seem to be uniformly distributed on the unit circle and so this is another example of universality and so the question this talk is going to ask is can universality help us in high-dimensional statistics? So it turns out that a universality phenomenon that happens in statistics which we call spectral universality so I'll tell you about this phenomenon now and again the setup is going to be recover a non-signal from noising on linear measurements and so the universality phenomenon is going to be about the behavior of the design so if someone gives you a design you can always write down its SVD so the u's are the left singular values the sigma are the the u's are the left singular basis the sigmas is a diagonal matrix of the singular values and the v is the right singular basis and what's frequently observed is that the statistical properties of the inference task depend only on the singular values only the sigma and the singular vectors are irrelevant if they are generic and by statistical properties I mean the performance of your favorite signal recovery algorithm for this problem and so what this means is that if you have two designs which have this which they're the same singular values but have very different singular vectors then this sort of heuristic tells us that we should expect the two designs to behave the same and the reason why this is a heuristic is because there's this term I'm using that the singular vectors need to be generic and I'm not really telling you what that what generic means right now and so I'm not telling you when this principle will apply and when will it fail so let me give you an example so the example I want to share with you is this is in the context of the lasso estimator so it was observed in these papers by Dono and Tanner and Manajami at all and what these folks did they looked at the lasso estimator for compressed sensing and they looked at three different designs the first one was the spike sign design the random t-city design and the hardest line and I'll tell you what these designs are and they plotted the mean square error of the lasso estimator as you change the sparsity of the underlying signal and what they found was that the three designs behaved exactly the same so the green curves are for one design the star curves are for the other design and the red dots are for the other design and it's not just that the mean square error of the lasso estimator was the same they also you could also look at the histogram of the lasso estimator and even the histograms look basically the same so it's as if the distribution of the lasso estimator was same for these three very different designs and what maybe I'll tell you what these three designs were so one of the designs was called the spike sign design so this is a deterministic design which consists of two blocks the first block is the identity matrix and the second block is the DCT matrix the discrete cosine transform matrix it's a deterministic orthogonal matrix the other design was the random DCT design so in the way this design is constructed is that you take a big DCT matrix again that's just a deterministic orthogonal matrix and then you pick some of its rows the third design was a extremely random design so what you do is that you sample a big random orthogonal matrix a uniformly random orthogonal matrix and then you pick some of its rows and what's common among all these three designs is that they have the same spectrum they have the same singular values but they have very different singular vectors and out of these three designs we sort of only know how to understand the hard design well and so there's this work which tells us a lot gives a lot of information about the behavior of the hard design and we sort of don't know much about these other two designs but they behave exactly like the hard design and it's not just in compressed sensing so like the two applications I mentioned in the beginning of the talk for example compressed sensing I told you here the most important design is the randomly row sampled Fourier design and in X-ray crystallography there's a different structure design and this design is called the mass Fourier design so each block is a Fourier matrix that is a random diagonal matrix and in both of these applications spectral universality seems to apply and more generally it seems to apply every time you have a very structured design and by structure design I mean you know that it should have strong dependence and very little randomness which makes such designs hard to analyze but spectral universality seems to apply to such designs and I should also say that this phenomenon is not really about statistics it seems to be a property of dark stochastic systems because it's been observed in other fields like statistical physics in spin glasses and in communication systems okay so okay maybe I've convinced you that this phenomenon happens and so why is this should we care about it and so the reason is that if we could sort of prove this thing happens we could use this as a tool for analyzing systems that we can't understand directly so maybe what I mean by that is that suppose you're interested in a given inference problem which has a design that that's given to you and you might often not be able to understand the behavior of the inference problem with this specific design because this design might have strong dependence or limited randomness so think of the designs like the random DCT design or the spike sign design but using the spectral universality sort of heuristic you can construct a surrogate inference problem which will behave just like the inference problem that you care about but will be much easier to understand and so the way you construct your surrogate inference problem is that you take your original design x and replace it by the surrogate design x tilde and the way you construct the surrogate design x tilde is by preserving its singular values so x tilde and x have the same singular value sigma but the singular vectors of x tilde are random they are random orthogonal matrices and so the exact definition of the uniform distribution on the orthogonal group is not super important but just think of it as the most mathematically convenient distribution on singular vectors that you could possibly think of and so if you construct your surrogate design in this way thanks to the rotational invariance properties of the uniform measure on the orthogonal group this design turns out to be easy to analyze and there's like a lot of different tools that people like using to analyze this rotational invariant design and I have some references for some of the tools but at the same time if you believe in this spectral universality heuristic because the surrogate design and the actual design that you care about share the same singular values you should expect the two problems to be equivalent in high dimensions and that's what happens okay so what so let me quickly tell you about what's known about what was known about this this phenomenon so like I mentioned this was first observed in the statistics literature by Dono and Tanner and actually Dono and Tanner proved some of their give mathematical proofs of some of the empirical observations in the context of noiseless compressed sensing so this is a noiseless linear regression problem for the lasso estimator and this this is like an influential result and it's really hard to beat this result in terms of the generality of their assumptions so the result applies to newly deterministic matrices but maybe the reason why this result isn't really the end of the story here is because of their proof so the proof the way Dono and Tanner proved their result was using some results from the theory of random polytopes and the basic idea is that you can pose the lasso problem as a linear program and they were able to use some results known in the theory of random polytopes to understand the behavior of this universality behavior in the context of lasso and in particular the proof breaks down as soon as you add a little bit of noise to the problem we kind of don't know how to extend the arguments or if you go beyond lasso to any other estimator which cannot be computed using linear programming so it doesn't quite capture the full generality of the phenomenon here and then there's like a there's a long line of work about Gaussian universality which basically says that if you have highly random designs they often behave like Gaussian designs and there's a long line of work in this in this direction and so by highly random designs I would mean designs with iid entries or designs with independent rows and what's great about these results is that unlike the Dono or Tanner result they're extremely general so they apply to very broad classes of inference problems but the flip side is that Gaussian universality is not valid for structured designs that I'm talking about like the spike sign or the random DCT design so it's not that these designs behave like Gaussian designs even in simulation so they behave differently and then there's this third line of work which is sort of coming from communication systems and free probability and so really like maybe if I speak in very rough terms is that what they are interested in is understanding the behavior of the spectral measure of a random matrix which is constructed by adding or multiplying matrices with generic eigenvectors and yeah but it's yeah it's hard yeah but in sort of for example it's hard to relate the performance of lasso estimator to the spectral measure of some matrix so it's not clear how these results would help us understand what's happening in statistical applications okay um okay so before I tell you about our result let me just recall what's the heuristic that we are trying to understand so the empirical observation is that the statistical properties of the inference task depend only on the singular values of x the singular vectors don't matter so much if they are generic right um and we would like to convert this heuristic into like a mathematical principle in the sense that we should be able to understand when should we expect this to happen and when should this expect we should expect that this doesn't happen and in particular we would like to understand what does it mean for singular vectors to be generic uh maybe have a precise mathematical definition you know uh generic um and so I've been a little bit obsessed with this problem and I've written way too many papers on this but I what I'll tell you about is the last paper I wrote with my postdoc uh advisors uh you and Suvratasen and I will also point you to this parallel work by Wang Zhong and Fan which also tries to understand this phenomenon under a different set of assumptions uh okay so let me tell you the setup for the result uh so the um uh so we look at the simpler problems so this is just uh maybe the canonical linear regression problem so uh uh again there's an unknown uh signal or parameter beta star that we would like to estimate uh x is going to be a design matrix or the feature matrix uh and we observe the measurements or measurements y and y are a linear function so it's x beta star plus noise and we'll also assume that the noise is iot Gaussian so we want to understand this when all in this simple setup uh and in particular we we will look at um uh regularized least squares estimator uh for this problem so what these estimators do is that they um they're computed they're given by the minimizer of a cost function uh and this cost function has two parts the first one is the uh squared uh Euclidean distance between the action measurements and the fitted measurements um and then there is this regularization term that you might use to promote some low dimensional structure on your estimate of beta star for example if you um uh um uh believe that your true beta star was passed you might want to use the L1 regularizer here but uh in the term that I show you the assumption is going to be that this regularizer is strongly convex so it won't apply to lasso but if you consider lasso with a small amount of bridge then it would apply to that um okay um so here's the theorem that we proved um and before I say the theorem uh I should preface it by saying that this is an asymptotic result in high dimensions so remember we are trying to estimate a parameter or a signal beta star and b is the dimension of this signal that we are trying to estimate and the result is a asymptotic statement as p tends to infinity um ah okay give me one second it's quite frozen again okay so uh what the result says uh is that um so suppose you're given two designs uh and we impose some conditions onto two designs under which they'll be equivalent um and the right now I'm just calling them uh the conditions as the two designs lie in the same universality class and this phrase is a placeholder for three conditions I'll tell you about in the next few slides so they are these three conditions and the two designs have to satisfy these three conditions and then we say that they lie in the same universality class and I'll tell you what these three conditions are in a minute minute and then uh suppose that you sampled the unknown signal or the unknown parameter from an iid prior so each coordinate of beta star which is the signal that we wanted to estimate is sampled iid from a prior pi and then what you do is that you compute uh the regularized least square estimator for the first design um and the second design so beta hat one is the regularized least square estimator for the first design and beta hat two is the regularized least square estimator for the second design um then what the result is claiming is that in high dimensions um the joint distribution of the first estimator computer on design one and beta star uh which is the unknown signal is the same as the joint distribution of the estimator computer on design two and the uh the unknown signal and the sense in which this approximation holds is that if you take any um pi which is like a nice test function it's a bivariate test function and if you apply it to the coordinates of the first estimator and the unknown signal and average it across the coordinates and you apply it to the second estimator and the unknown signal and average that across the coordinates then the difference between the two is going to go to zero as the dimension goes to infinity and really you should think of pi as your favorite loss functions like for example um if you take pi to be the square loss it would tell you that the mean square error of the two estimators on these two different designs um is the same but there's nothing special about the square loss you could have taken other other losses um so so that's the template of the theorem that we proved but I still owe you an explanation about what I mean by uh the two designs lie in the same universality class and what these three conditions are so these three conditions um the first one of them is not very interesting it just requires that x1 and x2 should have the same singular values in an approximate sense and yeah so that condition just says that and it's not super interesting but I think the other two conditions are more interesting so I focus on telling you about these other two conditions okay so uh the the the second condition is that the design should have generic singular vectors and so this was also a condition imposed in the heuristic but I didn't tell you what generic means so this is a definition that we could come up with under which we could prove our theorem and so I'll first tell you uh what the condition is mathematically and then I'll give you some intuition for it so the condition imposes that for any integer k if you look at the case power of the covariance matrix of the design it should look like a rescale a scale it should be a rescale identity matrix and you are allowed an error of uh one over root p and then you have some typically it's hard to verify this condition with one over root p but you're allowed some slack which is a poly log factor and then becomes easy to verify this condition and also the way you are um measuring your approximation error is this infinity norm which is the entry wise difference map entry wise maximum difference between the two matrices so entry wise the gate power of any k any power of the covariance matrix should look like the identity matrix okay so this is uh this condition is a bit hard to pass but the way I like to think about it is via this thought experiment which we call a permutation invariant design so um suppose so in this thought experiment suppose you computed the um uh you look at the covariance matrix of your design and then you computed its um eigen decomposition so the lambda i's are the eigenvalues and the ui's are the eigenvalues and suppose for the purpose of the thought experiment that the eigenvalues are randomly matched to the corresponding eigenvectors so the coupling between the eigenvalues and the corresponding eigenvectors is random and it's given by a random permutation then you could look at any power of the covariance matrix and you could argue by some concentration that it's going to concentrate entry wise to the expected value of the gate power of the covariance matrix and thanks to the random permutation you could actually compute this expected value explicitly and it's exactly a scalar multiple of the identity matrix which shows up in the condition so for this thought experiment actually verifying this condition is almost like immediate um and what I want to tell you is that this thought experiment is not super crazy because it already captures some interesting designs where people saw uh spectral universality so for example if you're picking um random rows of the DCT matrix one way you could do it is that you first shuffle all the rows of the DCT matrix and then pick the first few of them so that's going to give you a random permutation and this random DCT design is exactly covered by this thought experiment and the same with many big random rows of a hard matrix or a uniformly random thought matrix that's also covered by this thought experiment but uh but the point of these conditions is that you don't need this random permutation to verify for these conditions to be satisfied and what these conditions do is that they sort of capture this approximate decoupling between eigenvalues and eigenvectors um and what is nice about these conditions is that we were able to verify the set of conditions for all matrices that uh that were reported to exhibit universality in this work of monogamy at all which was one of the works which um um empirically found this observation um and so in particular I have some examples like Iod matrices would satisfy this and the left linear transformations of Iod matrices would also satisfy this um um the spike sign design I told you about satisfies this and the mass for your designs which are important designs for the space retrieval problem in extra crystallography also satisfy these conditions um okay um so the last the third condition I want to tell you about is the sign invariance condition um uh so the way I am going to motivate this condition is by saying that just with the generic singular vectors condition we know that it uh even without without imposing further conditions uh universality can fail so uh so in this plot I'm again plotting the mean square error of the lasso estimator um as I change the signal sparsity but the three different designs that don't hurt Anna studied so they looked at the random dcd designs spike sign design hard designs so these three designs have the same singular values and they satisfy the generic singular the notion of generic singular vectors that I told you about um and in the first plot I I'm plotting the performance of the performance of the lasso when the unknown signal has been sampled from a prior which is symmetric about the origin um and in the second plot I'm doing the same experiment but I um sample the unknown signal from a non-symmetric prior and in the second case you can see like departures from universality in the sense that the three designs don't behave are the same um and this was also observed in this paper by um monogamy at all uh they they also observed then for example and the unknown signal was completely positive uh they started observing departures from universality so to uh bypass these cases where you know universality can fail the condition that we impose is that the way you construct your design matrix is uh by taking a deterministic matrix j now this j can be any matrix but you randomly sign the columns of j so these s1 through sp are random signs um so it's not going to be uh satisfied by the spike sign design because its columns are not randomly signed but it's for a good reason because in some cases this spike sign design does not exhibit universality but as long as you randomly sign the columns of this spike sign design it'll satisfy all the conditions um and uh that's really the only source of randomness in the design that we need to prove the theorem so just be random signs in the design matrix the other conditions are deterministic and maybe the point is that it's much less randomness in as compared to other universality results that we know um and it turns out that you don't need these random signings if there's some inherent sign symmetry in the problem for example if you're using a the signal is drawn from a symmetric prior or and you're using an even regularizer then which is an even function then again you don't need it's possible to do away with this assumption and um I think what's an interesting problem here is which would be great to sort of figure out um a deterministic condition that replaces this sign invariance condition so we know that we need another condition here but we don't know what exactly this condition is and this sort of this random signings is right now a placeholder for this future condition that we don't know right now what that condition should be um okay uh so let me summarize uh so what I told you about this in the stock was spectral universality so roughly speaking the phenomenon says is that um there's still securities of interest to us often depend only on the spectrum of your design and not the singular vectors um and then I sort of uh in the context of regularized linear regression I told you nearly deterministic conditions of the design under which you can expect this to happen yeah okay question for Risha so in the second condition for the universality class you consider x transpose x right is it important that this is the case why don't you look at x x transpose because this is saying something about right singular vector about x but not left uh yeah um so uh yeah so this is coming this is something to do with the problem structure so this this theorem was about regularized linear regression right and in that problem it turns out that you only need conditions on the right singular vectors but if you look at more general problems for example like if you're fitting a generalized linear model so your loss is not the square loss but like a general loss then you can prove a version of this theorem where you would have conditions on x transpose x x transpose and there will also be some additional conditions uh on uh uh on sort of incoherence between the left and the right singular vectors yeah so so uh yeah so maybe that's the concept yeah okay thanks yeah more questions okay I'll ask one um could you say two words on what happens if the regularizer is not strongly convex but just convex uh yeah so yeah okay so yeah okay maybe this would be a little bit technical but so the way we sort of prove this result is by um uh uh uh writing down an iterative algorithm which would solve the optimization problem that defines the estimator so maybe you just write down gradient descent and what we sort of need for the proof um uh to work is that gradient descent should converge in a constant number of iterations and if the problem is not strongly convex that might not happen um and so uh uh right so so that's why there's this strong convexity assumption but um and if it's not strongly convex you would need to do an analysis which is case by case right for example if the problem is not strongly convex it's not even clear if the solution is unique right uh so like for example for last so you could probably try to do it but it would have to be more on the case by case from what I understand there are no general conditions under which uh people would be able to tell you that oh the solution is going to be unique and um so the problem is not only the finite number of iterations so it's also the fact that this may be not unique because if it's only the finite number of iterations assuming the iterative algorithm is A and B then you would try to do non-asymptotics no you could do a non-asymptotic analysis like you do in random initialization so there is some recent work on that oh right right uh so so I think I missed your question so your question was uh is the problem only that you uh so is the problem that the the surrogate algorithm that you are using to analyze uh regularizing linear regression uh running for a fixed number of iterations independent of the ambient dimension or is the problem that you have say multiple fixed points and so depending on where initialize you go there because the second problem I wouldn't know how to fix for the first problem maybe there is a way right uh okay so uh maybe what I'll say is that uh uh um so it's more of the first problem in the sense that uh uh uh so that's uh like so the proof is method of moments and it's typically hard to do method of moments when you have a diverging number of iterations um and the other thing is that um um so in in problems like lasso for example for lasso Gaussian designs people know that EMP works in a constant number of iterations but they know it because of some special properties of the lasso call right so that's what I'm saying that um uh the main problem is uh constant number of iterations but in some depending on what the problem is you might be able to get away with constant number of iterations by some specific cool yeah thanks any more questions I'm not sure whether my question can be like directly applied but since you mentioned a few times a connection to free probability and uh I know that in free probability um um in analogy to the classical centered limit theorem if you sum up many free variables then uh you also find a limiting distribution so I was wondering if you can use somehow these results uh in order to find a condition on what this matrix x should satisfy to um display some universality in the sense that um maybe this matrix x is a random matrix and if you take independent copies then they are free and then in that case you would get this universality I don't know can you make some connection like this yeah so maybe I can't make connections to the result that you're the free central limit theorem that that you're talking about because that involves averaging a large number of matrices right so here we just have um like one matrix right so uh and like if the matrix is like divisible like a Gaussian matrix you can sort of construct this matrix by averaging several independent copies of it but the matrices that we're looking at don't have this this property but what you could do is you could sort of make an analogy to other results in free probability which says that so for example if you have um two matrices and you conjugated one of them with a hard matrix then the two matrices become freely independent so you can characterize the spectrum of the any polynomial in those matrices right so what you should like maybe the analogy here is that um uh the two matrices in our example would be uh uh like for example if you have a randomly sub-sample dcd matrix so one matrix is the dcd matrix and the other matrix is a diagonal matrix with zero one entries telling you which which rose you sampled um and uh and somehow it turns out that these matrices are free without even uh uh conjugating by hard matrices um uh and uh and it's uh so so that's that that that that that sort of that's the sort of results which were known in free probability because that's the sort of the third line of work that I mentioned that sometimes you can get freeness without conjugating by hard hard matrices uh but uh uh right and the other thing is that it's not clear in statistical problems that why uh uh how is freeness going to help us so freeness is something about spectrum of polynomials right but in statistical problem we're interested in um like for example the emissive of lasso estimator but how is it related to the spectrum of the matrix so but it turns out that um it's something like yeah it's something to do with the behavior of the polynomials polynomials in these matrices which is sort of shared between the two yeah yeah I don't know if that made a lot of sense yeah yeah any final questions thank you and I think the lotationally invariant amp by Zofan derives a similar result so I wonder if there is a difference between the rotationally invariant amp and this research uh so not uh uh right so uh yeah so like the the bad work I mentioned by Wong, Jong and Phan uh they look at uh like universality of like a universality theorem beyond the rotationally invariant case so even if your matrix is not uh uh rotationally invariant sometimes it behaves like a rotationally invariant one and it's sort of looking at uh when does this happen so um yeah so um so maybe roughly speaking I can say that um um so we uh yeah so uh uh the results are stated differently in the sense that uh they like to put all their conditions on the matrix and we like to put some conditions on the matrix and some condition on the unknown signal which is which we assume is sample iid so but you can sort of transfer some of these like imposing some conditions on uh the signal is same as imposing some other conditions on the matrix so if you do this transfer then to make the comparison I think the comparison the Paris comparison would be that uh the results uh those results are stronger in the sense that they don't assume that your unknown signal um is from an iid prior but they would require that the distribution of this unknown signal is for mutation invariant so that's the strength um and the weakness would be that uh on the design matrix they would assume a stronger assumption they would assume that uh they would essentially assume the permutation invariant model that the eigenvalues and eigenvectors of the covariance matrix of the design are randomly coupled uh so it's not going to be satisfied for some of the designs that I told you about but there's no random permutation uh coupling the matrices yeah thank you yeah so yeah yeah yeah so I think in the interest of time we'll conclude here thank you Rishabh again


What is important what he should realize is that the zombies are suffering you may not know that probably but I think He's been in opposition for how many years? 20-20 something years he should have people to tell him the truth those who are on the ground He should have meals with them call them for a Coca-Cola tea a phanta and they ask them How is the country doing? How are my people that I'm governing? How are they doing as they manage to have three square meals because the whole essence of a voting a president into power I want to tell you this it's because they want to see that they are living standards is elevated by That government and of course through the president But if single get from west to bad from the previous government People lose confidence in you. They don't have respect Because boy, but he lack of home back and I'm coming mother Look at you know food you stay the whole day without food and you don't know where the next to come from You can say anything. That's what I'm telling you say people use vulgar language Against the leader. It's because of that. They feel cheated. They feel betrayed So for me, I think if that is what he says no one should stop him or whatever I want to do but as long as this policy and it will benefit the Zambian people late Zambian's benefit from what he has said Then I think they will respect him. They will give him credit Credit as much as what? All right, just for limited as I started that a lot shedding due to water due to low intervals that Reband up. I think this problem has been there since time my name on Memorial Do you think we have any lasting solution remember? Even the people in the special now the same minister was quoted in some sections the leaders many people to wonder You're really in the curry but I don't feel it and see That is history that was clear. Let's talk about today today. I was looking at the clip We are the president the current president if I to say that we've been in office for over one year Have you seen the Lord Shedding? There's no Lord Shedding and they just a few months later. There's Lord Shedding So let's not talk about the previous about what somebody said in PF. It's now people voted for European before them to be different from PF so that they could deliver but unfortunately they're not delivering now coming to your question Let me tell you what is important is people to have power today. That's what we want We are experiencing 13 hours at Lord Shedding, you know what that does It tools for a lot of things even in our own homes in our households Chickens are rotting beef is rotting butter is melting and so on and so forth Now go into compounds How do you say you are a barber? That's how you survive.  You have a small salon You are a lady and how can you stay for 30 hours without electricity? For argument's sake if life go at 06 hours, then it comes at 21 hours Are you going to have customers at 21 hours? No So there will be the loss of production. The economy actually will not stick It will affect so many industries But why did the president In his wisdom Tell the nation that there's no long-shading and there will be no long-shading even he didn't consult Zesco stuff to tell him to say boss. Hey, this is the Caribbean down As giving us problems or these little water and so on and so forth They should have done that and plan some months ago. That means they are Chingu no plan They didn't plan and they didn't expect us not shedding which is very unfortunate That means he hasn't got a good team of advisers who should have advised him In fact what Zambians expect is that before something happens The president should be able to stand up and then now to the general public and tell them to say this is what will be Experiencing in the next few months so that Zambians are ready for that But unfortunately, this has not been the case. This has caught us and aware all of us We never thought long-shading would be would come back again because we have enough power in this country If not, why are they exporting? They should have stopped exporting first of all They should please is again people we in the power They should please a man not to please the South Africans is in bubbles and so on and so for the Congolese and so on and so For no, that's wrong first of all who put you in power the president That's what you should ask yourself if it is the people of Zambia Why should I and shouldn't I also make them happy by stopping exporting of electricity? That is what you do company should be. I've already said it. All right. We have already said it We move on to the consequence that they don't mean fun that the constructs development fund Has been increased that the new don't have been that I think in the 2020 budget it was increased from 1.5 million to 25 7 million somewhere How do you describe by the usage of what is the constraints that you have been finding and that they You're doing administration. It's nothing. That's a drop in the ocean. How can you use in a consequence 25 million? You know the cost of grinding a road is 1.3 million So if you divide that by 1.3 that is less than 20 kilometers That's the year before you go to other sectors of the CDF fund So for me in as far as I'm concerned that is so little and it will not improve this country economically There's a lot of it There's a lot of infrastructure which needs to be done in all the consequences and a lot of progress Sometimes you find that the roof has been ripped off by the rain and the cost is about 200 300,000 now they are 10 of that nature and that's all pay it Mind you so if you divide in fact at 28 point Whatever million into 12 that is about 2.1 2.2 million a month. That is nothing You are talking of empowering Women and men youth and women in that same fund that is too little I think what I can advise the government is that major project like bridges Roads and so on and so forth These should be taken up by the central government. The central government should take responsibility of of Bringing in or doing this infrastructure because cbf should only be for small projects in