James Charles, Andy Signor, Slazo, and now Pro Jarrod. How long is this going to continue to go on where the internet hate mob attacks people because they're guilty until proven innocent? In this video, we're going to be discussing Pro Jarrod's situation, but I'm hoping we can all learn from it with a little bit of social psychology. What is up everybody? This is Chris from the Rewired Soul, where we talk about the problem, but focus on the solution. And if you're new to my channel, what I try to do is take different topics going on in the YouTube community and try to see what lessons we can pull from them to improve our own mental and emotional well-being. So if you're into that stuff, make sure you subscribe and ring that notification bell. So yeah, much like many, many other people, I watched Pro Jarrod's return video yesterday and he cleared up a bunch of stuff and that's what we're going to be talking about today. But anyway, it's real quick. For those of you who don't know, I was a victim of online hate mob insanity as well. So I wrote a book about it called Canceled Inside YouTube Cancel Culture and I also discussed the Pro Jarrod situation, the situation with Holly Conrad and a bunch of other situations that have happened throughout YouTube. Like some of the ones I mentioned in the intro, like with Slazza, with James Charles. But anyways, my brand new book, Canceled Inside YouTube Cancel Culture, you can get a free copy of the e-book until the end of the month, so you got about three days. So check out the description, check out the pinned comment down below because this stuff needs to stop. All right, so anyways, to get this thing started, like make sure you go check out Pro Jarrod's video. One of the things that I discuss in my book is the initial accusations that get spread like wildfire. Even when you come back and defend yourself and release the truth, there is only a fraction of the people who will see that. So help Pro Jarrod out and share his video. So some of the people who are contaminated by the false accusations in the beginning of this whole thing will get the correct information now. But anyways, I wanna start out by talking about this real quick clip from the intro of Jarrod's video. And I've already seen a lot of posts from people telling me to move on or that the tea is cold. I'm sorry that this is no longer entertaining for them, but just because they're bored of it doesn't mean that the harassment stopped or that the story is final and over. I am so glad that Jarrod said this. I am so glad, like this is something that until you go through it, like you don't get it. And I'm not asking for pity or sympathy or anything like that, but I just hope you, the viewers, you understand, like this might all be like fun and games and entertainment for you. You know, drama channels and commentary channels who do these things and like perpetuate these narratives. Like there is a real person on the other end of that. So even though you guys or some people might not wanna hear Pro Jarrod's side of the story anymore, like that dude and Holy Conrad are still getting harassed. Like everybody who goes through this has psychological effects that happen to them and it's ongoing. I had somebody yesterday when I was talking about something on Twitter say, oh, when are you just gonna move on? Like, do you really think that on a daily basis I'm not getting harassing comments or even threats? Like it still happens months later. So I'm so glad that Pro Jarrod talked about this. But anyways, again, make sure you subscribe because this is going to be part of a series, all right? I'm reading a really interesting new book about social psychology and I wanna use the social psychology of witch hunts to kind of talk about different situations that have happened in YouTube. Like I mentioned with James Charles, with Andy Signor, with Slazo and things like that. All right? And this is the first one. So real quick intro of this topic. So there was a social psychologist, famous dude, named Emil Durkheim, all right? And he studied human behavior, all right? And he was a very smart dude. Well, in the 1960s, a guy by the name of James Bergeson wrote an essay called The Durkheimian Theory of Witch Hunts with the Chinese cultural revolution, all right? So what's really interesting before we jump into what Bergeson talks about using Durkheim's theories is if any of you have heard of Gustave Le Bonne, I've made some videos about him in the past, but Gustave Le Bonne, he is famously known for writing a book called The Crowd. And it's all about crowd psychology and mob mentality. So the more you learn about this and the crowd psychology and mob mentality and you blend that in with the social psychology of witch hunts. Like, it's absolutely crazy. So for example, what Berkheim talked about is how we humans, he calls us homo duplex, right? We have two different levels. So as individuals, we are very rational, right? We are independent thinkers. We make up our minds. But when we become part of a collective, so much of that just goes out the window and we are insane people. And logic just flies out. It's just gone to where'd logic go? It is no longer there, all right? And that's what happens during these witch hunts. So when Bergeson put together his Durkheimian theory of witch hunts, he said it had three main components, all right? The first one is that these things arise extremely quickly and seemingly out of nowhere, all right? Like they escalate. Like they go from zero to a hundred like that. Number two, they involve crimes against the collective. So everybody in the collective, in the community can be affected by this. And lastly, the so-called offender, all right? A lot of this stems from either A, something very trivial, or B, something that's completely fabricated, all right? And there's a fourth thing that we're gonna be talking about, which is that when this happens, people are afraid to defend the person who was accused. All right, so we're gonna take these four components and relate them to the pro-Jerrod situation. These arise very quickly and seemingly out of nowhere. I had never even heard of pro-Jerrod.  But this all started when his ex-wife, Heidi, released all this information about the awful person Jared was when they got a divorce. And also, I forgot to say this in the beginning, but do not go harass Heidi either, all right? Jared has not asked you to do that.  Like that's something I hope everybody learns about this whole cancel culture and mob mentality. Like just going from one, like hating one person to another, it's not getting anything done. All right, I talk about that quite a bit in the book. Anyways, came out of nowhere, escalated extremely quick. When this happened with pro-Jerrod, it was trending on Twitter, all right? So these involve crimes against the collective. So pro-Jerrod was accused of soliciting nudes from minors, which he cleared up 1000%. But how does this affect the collective? How did the entire YouTube community turn against him? Well, we've had stories of people like Austin Jones, who was doing this. There's people like Romeo, whoever the tattoo guy, who was doing things like this. And even Jared acknowledges in his return video that there's somewhat of a power imbalance. And that's something I can make a whole different video about, right? But the fear is, is that this can affect the entire community. And is it irrational? I think that one's a difficult one, right? Because there are many people out there, not just YouTube creators, but people who use their position of power to manipulate people who are lower on the power dynamic. You know what I mean? But anyways, that's how this whole thing started to escalate, because people were saying Jared is doing things that could affect the entire community. Number three, the offenses often stem from something trivial or fabricated. So this is actually both, all right? So fabrication, the accusations against Jared for soliciting news from minors, that was completely fabricated, all right? We now know that. Unless new evidence comes up, where we're at right now, Jared provided enough evidence to at least give us reasonable doubt. And something I try to talk about, and I talked about this in my video yesterday about Dave Chappelle, is we need to start looking at the court of public opinion and trying to hold it to a standard of the actual court of law, all right? Because so many of these things that are happening, people are guilty until proven innocent, and with due process, people are supposed to be innocent until proven guilty, all right? But the trivial aspect, okay? And some of you might disagree with me on this, but hear me out. Trivial, this all started from Heidi claiming that Jared cheated on her, okay? So in a personal relationship, that's a huge deal, but 99.9999999% of the people who have been part of the hate mob on either side, it's trivial, you know why? Because you're not in this relationship.  It's none of any of our business. So the harassment that this man got, all right? It is over something that literally does not affect you when it comes to the relationship with him and Heidi and him and Holly, all right? Also something trivial, and I'm glad that Jared kind of stood his ground on this and even his ex-wife, Heidi kind of discussed this, like they have their own sexual things that they like to do and they were swapping news and it was consent between adults or people who said they were adults and lied to them and stuff like that. Like what you do in the privacy of your own home or wherever, like do your thing if it's not hurting anybody, quit caring so much about other people's sex life, okay? Thanks. So number four, fear of defending the accused. This happens every single time. Like when you watch Jared's video, he talks about how he tried to distance himself away from his other friends so they weren't being hit with the collateral damage but the problem when these things happen, especially on YouTube, is that people don't wanna come to your defense because they'll be attacked by the hate mob as well. Like that was one of the most painful things for me, like you feel completely isolated. Like I had friends who tried to stand up and defend me and they were threatened, they were attacked, like it was nuts. Like I had people who were just like, hey, Chris isn't that bad of a guy. They were getting threatened with physical violence, all right? So this is one of the issues when the hate mob attacks is like nobody can even defend you because the mob will just go after them as well and say, oh, well, if you're not with us, we're against you, you know what I mean? And that is absolutely brutal. I'll tell you this, the only time, the only time I've ever seen people come to the defense of the accused is when it happened to James Gunn, the director of Guardians of the Galaxy and outraged culture just started freaking out but the cast of Guardians of the Galaxy and many other celebrities were like, no, this ain't cool and they came to us defense. I wish that happened more often. So yeah, those are the four main components of witch hunts and I hope you now see kind of how this happened to Jared but I want you to look at a greater whole so this is gonna happen again. We're like barely over halfway done with 2019 and it's already happened four times this year, probably more that I'm not even listening, right?  Like, okay, is this turning into something that's escalating too quickly, right? Is it something that we believe involves harming the collective? Is this stemming from something trivial or is it even possible that these accusations are false? You see what I mean? Like, start thinking in that way. So the last thing I wanna talk about, here's what really bumps me out. There are so many creators and Jared talked about this in his video.  So many of those creators who made videos like that are not going to correct themselves, all right? I believe the right opinion tweeted out yesterday that he took his video down just because there's new information and props to him on that but I guarantee you many other creators won't do that because this is what happens every time but here's the second part of it. If they do and keep an eye on this, all right? Because I kind of like analyze these things. The creators who come back and say, okay, Jared cleared his name about soliciting news from minors, what they'll do after that, completely based on pride, completely based on ego in an effort to save face, they're still going to play into the narrative that Jared cheated on Heidi with Holly even though there is evidence that it was a sloppy, polyamorous relationship, all right?  Holly Conrad is not gonna get anywhere near the justice that she deserves for being caught up in this entire thing. And this video is longer than I planned but again, don't go harass Heidi but it's interesting when you watch Heidi's statements based on Jared's, like Heidi is much more aggressive, all right? And I understand when you go through a breakup, when you get divorced, there's a lot of pain, there's a lot of hurt and everything like that but I think that energy is fueling some people to still send hate towards Jared, towards Holly, all right? But anyways, again, this stuff needs to stop and make sure you check out my book. You can get a copy of it for free until the end of the month. A new book is called Cancelled Inside YouTube Cancelled Culture where I take a deep dive and give you kind of a behind the scenes look about how all this stuff happens and yeah, I will be making more videos breaking down different situations using social psychology and discussing how this happens. All right, but that's all I got for this video. If you like this video, please give it a thumbs up. If you wanna check out the new videos that I'm working on, make sure you subscribe and ring that notification bell. And a huge, huge thank you to everybody supporting the channel over on Patreon. You are all amazing and a huge thank you to everybody else who supports the channel in other ways, like by getting merch, buying my books, all that kind of stuff. I love your faces. All right, but anyways, thanks again for watching. I'll see you next time.


Wow you two team keep it clean what's going on it's engraving here with another video and man the Washington football team pulls off a major upset a major upset and now I know so many people have been calling the Pittsburgh Steelers the most overrated 11 and old team that there was and you can argue that amongst yourselves whether you feel that's accurate or not but the thing that I said they they won all their games whether it took overtime but there was some funny business that happened with officiating whatever they overcame a lot and they won all of their games so they had got there and earned their 11 and no record but here comes the Washington football team saying you know what everybody's sleeping on us nobody's giving us any credit nobody's giving us any respect in this game and I mean who really would besides maybe even fans of the team but who was really thinking that the Washington football team was gonna pull this off we of course knew that it was a possibility but nobody thought that they were gonna pull this off but they did it they did it Montez wet man that dude was a game changer all game all game from beginning to end and something that has been hurting big bin throughout this season is those tip passes at the line of scrimmage and boy it came back to bite them at the very end of this game now I know one thing that a lot of Steelers fans have been concerned about throughout this entire season was the the offense they have been very concerned about the offense because the offense they while they have been scoring points there were some games where they would just be like dead in the water they will be moving the ball but they wouldn't be finishing and it's funny because the commentators in this game they had likened what the Steelers were doing in this game where they were moving the ball and they were getting to the red zone but they weren't finishing the same thing that did last week against the Baltimore Ravens they were moving the ball they were getting to the red zone but they weren't finishing and the offense just was not doing enough defense they were making plays they even got it out to a what 14-0 lead and then it got to 14-3 but Washington football team they said it ain't over till it's over it ain't over till it's over and this this goes to show that football it's a sport where stuff doesn't make sense because you look at a team and just to use them again as an example like the Baltimore Ravens they took care of business against the Washington football team the game was pretty it was a lot more closer for comfort than I would have liked but they took care of business against the Washington football team but against the Steelers they lost not once but twice so you would think that okay since the Ravens they beat the Washington football team but then the Steelers beat the Ravens twice oh man the Steelers they'll take care the Washington football team no problem because the Steelers took care of the Ravens nope no not at all football just doesn't work like that and this has been a weekend hopefully it does not continue tomorrow but this has been a weekend where the NFC East say hey what's good we here I don't care what you think our record is or what our record looks like how bad it looks we ain't backing down you saw yesterday with the Giants against the Seahawks everybody including myself I'm like oh Giants and Seahawks please Seahawks got that they got that oh yeah they got it nope Giants say no we're good on that we getting this win in Seattle Washington football team thinking like oh while I wouldn't mind seeing Pittsburgh lose I was like yeah Pittsburgh they gonna take care of business they about to go to 12 or no whatever watch the football team say nope not on our watch in Pittsburgh in Pittsburgh now and there's straight up is no excuse for this one I know some people might be like oh well this is the the Ravens fall because the game got moved and it just messed up the Steelers schedule and that's why they end up losing this game no there ain't no excuses for this one it's no excuses they just because Steelers have been coming out like this all year but again like we talked about the majority of the year they had end up being on the opposite end of the wind column so this was this was something that Steelers fans have been sort of prepping for sort of that let down game that that trap game because they knew it could happen at any given moment at any given second and while I didn't really think the Steelers were gonna go undefeated this year after the second game with the Ravens I started to wonder like huh could they actually do it could they run the table because they had the Washington football team like oh yeah they take care of them but they did have some tough games coming up against the Bills against the Colts against the Browns so I'm like oh these and those teams are gonna be fighting but they got caught up and now you know Chiefs fans oh they celebrating like crazy they like yes this is what happened last year we fought till the very very very end to get a top spot in the AFC and of course it ain't over till it's over there's still four games remaining for both teams for both the Chiefs and the Steelers but those are the two teams that are essentially fighting for that number one and number two spot so now if you were thinking about possibly getting to week 16 17 and resting your starters that thought and I know it was it was too early for that but that thought has got to be out the window now because both of the Chiefs and the Steelers they got everything to play for especially because it's only one bye week it's only one bye week so that makes things that much tougher now one thing that I had noticed personally from when the Steelers played the Ravens and just watching Steelers period what they do a lot of they do a lot of the short passes they do it a lot of a lot of the short passes because their receivers are so physical and their receivers get that yak they get them yards after the catch and I know they put up the graphic that showed that Ben Rothesberger on the short and intermediate passes I think his completion percentage both were over 60% this year but on the deep passes I think it said that his completion percentage was like at 22% and I was like oh well I didn't really know it was that bad but I could see he hadn't been hitting a lot of the deep passes as much as what I was used to from Ben Rothesberger but in this game this game that almost read his ugly head I know James Washington had that what that 50-yard touchdown where Ben Rothesberger hit him and he got that yak he got all that yak and went in went to the end zone for a touchdown and I remember watching that play and I was thinking at that point just seeing how the Washington football team just they weren't tackling and I was thinking like oh yeah okay well we know how the rest of this game is gonna go we know how it's gonna go but nope they said no we ain't with that shout out to Alex Smith for leading this team shout out to shout out to the red I mean the Washington football team sorry for them losing their Antonio Gibson they lost their starting running back who has been just killing it for the past couple weeks they lost their starting running back early on in the game they lose them in the second or third or fourth no these dudes lost him in the first quarter I believe they lost him early early I wasn't even the first drive I think it might have been the first if it wasn't that first drive then it was their second at the latest they lost their running back early on in the game but what did they do they fought do they you know them calls for them causing Pittsburgh hey and shout out to those Pittsburgh fans that ain't afraid to admit it you know them calls this year been they've been shaky they've been shaky and even the Washington football team they had to they had to deal with it there was that big one where the tight end he caught the pass and he got his yak and he was in bounds knees weren't on the ground he wasn't on the ground and he dove in that ball across the first down marker and it was on a third down easy first down you could see it when it was live when they showed a replay you could see it I think Washington challenged it and the ref said oh nope the call on the play stands I said oh okay okay okay but this game it's a game of inches and sometimes since this is a game of inches some calls are gonna go your way you're gonna get those inches on the play and some calls are not gonna go your way when you're going against Pittsburgh hey man straight up you this year again y'all know whether your Pittsburgh man or not the refs have been doing some stuff for the Steelers man it's been like especially over the past couple weeks but really just throughout but you got to you guys to play better than the refs you got to play better than the refs and your team even though it can be hard and it's not even it's not even your fault because a bad call can just kill momentum it can kill a drive it can kill your spirit but for your team to be able to overcome the bad calls and still be successful that says a lot about your head coaching so round Rivera goes and gets kicked off the Panthers they say you're done goes over to watching the football team gets control of them and things then this season he started off a big yikes it was a big yikes but no the NFC East they I mean you they may have a playoff team at like six and ten that that might be that might be a playoff team for the NFC East but they they show that they continue to fight man so you got to respect it you got to respect it and right now they are right there in the thick of things are getting to the playoffs now while even though with the NFC East it's the Giants it's the Cowboys it's Washington and it's the Eagles whoever does make the playoffs I don't expect them to have any long-term success in the playoffs but still hey all you got to do is get there that's all you got to do all you got to do is get there and you never know what could happen from there you never know what could happen so congrats to them it's funny I tweeted out earlier I was like man I said it would be really nice if one team that plays in the state of Maryland could end up beating the Steelers this year and I say you know what it may even come down to a game between Steelers and a team from Maryland happening for the fourth time and I was meaning like if the Steelers played the Ravens and the playoffs of course the Ravens had to make the playoffs and they need some help but Washington football team say you know what it don't even got to get to that we'll take care of it and they did congrats to them big ups to Washington football team for getting a double with Pittsburgh and I know Pittsburgh fans they they probably upset but I mean as upset as you can be and should be and might be all you got to do to cheer up you just got to go to your phone go to ESPN you ain't even got to look at the box score from this game all you got to do go to ESPN.com click on the Pittsburgh Steelers logo and look oh oh 11 and one okay cool I'm sure that'll make you feel a lot better first in the AFC North fighting for the number one C right now so why they did lose tonight it's not like this season is over it's not like oh that's a wrap now oh man we're terrible oh man what is yeah Steelers have always had some things they needed to improve on more so on offense than defense but they 11 and one 11 and one so it's really not that bad anyway team keep it clean appreciate y'all I love y'all and just like the Steelers all when it comes to the race for chasing 072 dolphins I'm out


So, this is a very special moment for our university, our larger community of learning, the state of Vermont, and our nation, and we're very grateful for all of you who have chosen to stay through the course of the day to enjoy this moment. We have a few folks that will come up and speak, but I want to just talk for a second about how we came to this point. There are people in this room that have led Norwich University, whether through strategic direction or through partnership, to be an international leader in cyber education and its related fields. I'm looking over to my left to your right, I'm looking at President Emeritus Richard Schneider and the things he did for this university, programmatically, strategically, and just through sheer force of personality charm, you know, he did incredible things for this school. We would not be in the building like this, we wouldn't have programs we had if not for Rich. So, sure, thank you again for being here today. And sitting near Rich is our greatest living alumni, Gordon Sullivan, retired general and former chief of staff of the United States Army. General Sullivan was the chairman of the Board of Trustees and led in partnership with Rich, our university, to reach the point that we are the institution that we are today. And if not for General Sullivan, we would not be in a position we are as an institution, as a brand, and as such a critical piece of the national security system and the global community of learning. So, General Sullivan, sir, thank you very much for being here with us also. I'm looking in a room and I can thank many more people and acknowledge many more people that led us to this point, our incredible faculty and staff. There are so many faculty members, current and former, that also led our academic programs and did incredible things for our university and our community, and most importantly, our students, that are our highly successful alumni across all fields, but today we celebrate our alumni working in the cyber, computer science, information security fields, and the incredible work they're doing that furthers the Norwich brand certainly, but also in service to their communities, our state, and our nation. I'm looking at some of our trustees and there's one I'd like to acknowledge, especially because he's a cyber expert and he's our newest member of our Board of Trustees, and then there's Frank Leja, who is sitting here in the second to front row. So what he's bringing to our community as a cyber expert, as a refined gentleman, and as a great thinker, is going to be transformational for us moving forward, but every single person I named and those others in the room, including Phil Seussman, our director and president of Nuwari and vice president for strategic partnerships, everyone that has had us hand in bringing Norwich to this point will say very clearly and unequivocally that we would not be here without Senator Patrick Lee. We would not have been able to do the things that we have done and expanded our programs and to develop our brand to the level it is without this incredible partnership. So we're going to have a very special dedication here in one moment, but before we do that, we have a special video presentation for Senator Leahy that we'd like to play right now. Please roll it. I remember the very first time that I actually met the senator in person. I had an informal discussion and Senator Leahy came over to me and said that he had grown up in Montpelier, Vermont, and he had rode his bicycle from Montpelier to Norwich as a kid and had ridden around campus. And as he's talking about this at that moment in time, there was imagery. I could see the Vermont connection there. It was really, you know, I got that connection of the unique environment that we both grew up in, albeit I was a couple of years later. The love of Vermont, that was his touchstone from whence he went out and did all kinds of wonderful things for the whole country and for the world. We grew up half a block down from the state house. At a point, my mother rented rooms to legislators. My father bid on a lot of printing jobs for the state, so always topics of conversation at the supper table was state doings and political doings. The conversations we would have, the discussions about state and local national politics, I believe very much influenced Pat to run himself for political office. My campaign is a campaign to preserve the Vermont we cherish and we want to keep. And to prepare for the Vermont we want to move forward into the next century. This election will speak of the future of Vermont and the direction of America that together we shape a better century and a better world for ourselves and our children because it will be our century and it will be the best of centuries. The U.S. Senate was that place that would pull together his enormous talents and his drive to really make a difference. We're becoming more and more of a high-tech state, so certainly legislation that involves high-tech computers, the ability to compete worldwide, that's important. He's always been a bit of a technology guy, so he could make the connections between how technology was evolving and how we need to anticipate where it was going to go and not only the opportunities it creates for economic development, but also issues around security. And he saw the importance of cybersecurity as a military national security issue. Norwich had done a good job building the beach at on those issues. He saw that and he supported that and he tried to resource that. Senator Leahy sponsored the National Center for the Study of Counterterrorism and Cybercrime at Norwich University in the 2002 Justice Reauthorization Act. And we got funding that first year in a program element line for special operations. Our first real initiative was to do some type of educational program which ended up being decide. And Senator Leahy, the cyber senator, got it and he understood it. He said, this is a really important program because it's going to build resiliency into the elements of our national power and critical infrastructure. So as a former Army officer and the former commander of the Cyber National Mission Force, I realized that the greatest strength our nation has and the critical element of defending our critical infrastructure are people. So the Defense Cyber Institutes in Norwich, they feel a critical need in providing the right kind of education and training that our cybersecurity workforce needs. And with the support of leaders like Senator Leahy, these institutions have been able to stand up and during training programs to provide the hands-on technical training that our leaders need. They provide a great foundation of leadership and practical experience. And because of the diverse course loads that these students take, they also provide the critical thinking skills that are current and future cyber workforce is going to need to be able to compete in the future. I think what we really have to just take a moment and reflect is to really appreciate and thank Senator Leahy for his vision. For the trust that he's put in Norwich University to lead the way, not only are we leading the way in STEM, AI, and now quantum computing, but we're developing the entire student. We're giving them the critical thinking skills to build the next generation and continue it to be leaders in cybersecurity. I chose Norwich because of the computer security program, as well as the experiences and opportunities that would come with gaining an education from here, such as having an internship here on campus where I got to study how network security works for the state and understand how to identify threats and help if there's a problem and basically helping to keep the state of Vermont safer that way. Senator Leahy has been a very strong friend to Norwich University and to education in the state as a whole and also very supportive of first what Vermont needs, of course, but also how the Guard could help supply what Vermont needs. The Senator's steadfast support for this concept turned into a building, turned into an eternal concept because it's going to keep evolving and it's going to keep changing into what our country grows to need. He got it right away that we needed a flexible space that was going to be able to educate everyone, but we didn't know even what we were going to educate in because the technology was changing so quickly. That led to this building built in 2001 and without his support, this building would have never happened and we wouldn't have been able to deliver the great academic programs to civilian students, to cadets, to adults in the master's program and then the degree completion program for the cyber professional wannabes and for the United States military enlisted personnel. Man, what a powerhouse and we have been roaring ever since, but without that building, we wouldn't have had the space, the flexibility or the equipment and Senator Leahy made that happen. Senator Leahy has been a great senator, great congressional leader and the nations are going to miss him. I am personally grateful for the friendship that you've extended to me over these years to be able to build these programs and to support Norwich University. Thank you, Senator Leahy, well done. You can retire knowing that you have left Vermont in the country in better shape. So Senator Leahy, in my service, the United States Coast Guard, we would say, Senator Leahy, fair wins in following seas. We have left America, Vermont and Norwich in a great position of strength and innovation so that President Anarumo can build on that capacity and generate the next group of leaders to protect America. Hello, I am President Mark Anarumo at Norwich University and on behalf of the entire Norwich community, I want to thank Senator Patrick Leahy for his long support and partnership with Norwich. As we all know, Senator Leahy is one of the first to understand cyber as a domain and as an area of great threat to our nation and the world. And through the partnerships that he provided and supported to Norwich, we have been able to create some of the great leaders in the field, and not just for cyber, but the areas that are related to cyber that are now emerging in the 21st century, like data analytics, artificial intelligence, machine learning, and of course, quantum computing. These are all areas in which Norwich University will lead because of the great support of Senator Leahy. So on behalf of our international community that is Norwich and on behalf of our wonderful community here in the state of Vermont, please accept my sincere gratitude for our long and successful partnership. Norwich together and Norwich forever. So I'm about to make a very special announcement after which I'm gonna ask Representative Welch to come up and make a comment. But before I do so, we are proud to welcome the person for whom today is named and who we have the symposium honored for, Senator Leahy. Thank you for joining us, sir, very much. I've been looking forward so much to being there. And I'm sorry that this recent fracture I got, I'll be walking soon and I will be up to Norwich Field to see you. But there's one good thing about not being there. We ever had something where virtual technology was fitting at this time. It's allowing us to come together. I can visit the wonderful campus of Norwich from hundreds of miles away. Technology brings us together. And it's accurate to say as many have that technology creates a huge number of new threats, but technology allows us to stave off those threats and to enjoy the benefits. And of course, the most important thing that technology could do is the relationship that will create between people, not just in your campus, but all over the world. The response of place where people connected is not only important to life, but it's a very fabricated life. When Marcel and I are home, we might have dinner on Church Street and brunch on our visit to the gas station in Montpellier, our old friends and Vermonters I've known for decades as well as people I'm just being feel comfortable and coming up and just sharing their thoughts and we find how connected we are. They talk about Kevin Bacon and three degrees of separation in Vermont. There's never more than three degrees of separation. Everybody's connected to everybody. So it's natural for me to use cyberspace as a continuation of connection. I created one of the first websites in the Senate. I've done hundreds of video chats with Vermont schoolchildren and encouraged them to learn about our state and our country. And that's why I've ensured among the billions of dollars in the federal budget, it's appropriated for cyberspace programs that the executive agencies pay attention to the special perspective we in Vermont bring. And I must say that while they may listen to me as chairman of the committee, they listen even more when they hear the wonderful things you're doing, you make the case. And because the real connectivity is in cyberspace, the people linking into them, the people coding the software, the people fabricating their hardware, any cybersecurity specialist will tell you the weakest link in security is the humans who usually create the programs. But humans could be the most powerful part of our network. And so much of my work has been about how we improve the way human connections can strengthen the technological ones. I created the Trusted Foundry program years ago in support of this evolution. So people telling them can have confidence in their chips and can fight cyber attacks. I provided funds to establish the Leahy Center for Digital Investigations at Champlain College. So there'd be a place where the work of protecting and serving people in the real world is made easier by the online. When I ensured the National Guard had a cyber unit which helped the people, not only the nation, but our state and cyber disaster. So it's a web of human connections. But then I look at what you've done in Norwich. My gosh, everything, everything comes together. And I, this year I'm looking forward so much to being here because I think of all the years we worked together. And I think of the times walking around the campus just talking about things, maybe we could do this, maybe we could do that, well, let's try. And I never found anything in Norwich that we can do it. I mean, we talked about some of the most impossible things, we can do it and you have. And I can't tell you how proud I am. And Marcel and I have talked about this, the school bearer by name because we recognize that technology is connecting people. You're gonna train your cadets and civilian students, not only how to protect cyber space, but how to interpret, use the complicated information from new frontiers like cyber training. But because you are in Norwich, because you are in Norwich, you're gonna teach them ethics and the importance of a human-centric perspective. That's gonna make them better people, better students, better cadets, but it's gonna help America at a time we need it. You show them the Norwich way, the Vermont way, they'll learn the strength of the connection between people. They'll go back to the earliest days of Vermont. We've always had to rely on each other, but that's made us strong and resilient. Everyone here knows we continue the tradition into the digital age. We've used technology to create new bars between us. It has been my honor to sport and strengthen that during my Senate career, and I must tell you, even after I leave the Senate in January, you'll see me roaming around the campus. I know you've always had an open door for both myself and myself. We'll be there because you are in Norwich, because I am proud of you, and I can't tell you how much this day means to me. Thank you all very, very much. So thank you very much, Senator Lay, for your words. One of the beautiful things about living and working in Vermont, especially at a university like Norwich, is how deeply personal it is. And I really want to thank Senator Lay, he's a professional staff who are sitting here today. They asked me not to because they want to be anonymous. I won't call him out by name, but they hosted us, and Senator Lay, he hosted us in his chambers and throughout the building. And what was supposed to be a 30 minute office call turned into about an hour and a half of really deeply personal and beautiful time. And then his handlers kept coming in and saying, Senator, you have other senators waiting for you, you're late. And he said, tell them I'm with the president of Norwich University and they're gonna have to wait longer. And I asked him, and I said, well that case, please have to know my name is Rich Snyder, Norwich Forever. So in that note, I'm about to make the formal announcement, and then I'll ask Representative Welch to please come and join me on the stage. But ladies and gentlemen, at this moment, you're gonna hear the formal proclamation from our chairman of our Board of Trustees, but it is my absolute honor and pleasure to announce the naming of the Senator Patrick Leahy, School of Cybersecurity and Advanced Computing. Correct? Congratulations, sir. Thank you. Ladies and gentlemen, I'd like to introduce Representative Peter Welch to make some comments. Thank you, sir. Thank you very much. Patrick, I love that video. I love seeing your mom and dad. I remember when I first went to Montpelier and your sister Mary in the bicycle, that was good, but they left something out. You kind of messed up on this. You left out Senator Leahy as a young boy on his tricycle riding around the governor's office in the state house. That's true story. And I love the comments that are so true about Patrick just being so grounded in Vermont and loving it so much. And one of his, Chuck Ross, his former chief of staff would tell stories about riding around the state with Patrick and he'd be talking to somebody at the gas station or he'd be talking to a neighbor and then the next call might be to the president, right? Or the president's calling Patrick and they'd be riding around and suddenly Patrick had to tell the president, you know, I've got a call coming in. It's from Rick Snyder, let's say. But a Vermonter like took precedence and that grounding in Vermont and caring about it is what has totally anchored this man's career of 48 extraordinary years of service to the people of Vermont and to the people of this country in the United States Senate. That is so extraordinary. Patrick, we are so, so indebted to you for every single day where you acted on our behalf always with integrity, always with ethics and I want to lead a round of applause. And I want to just say a couple of things too because how does this happen, all right? That Norwich University is now a preeminent location for teaching and learning about cybersecurity. Well, number one is the vision, as somebody mentioned. Patrick had a vision and he was committed to it and he knew that he had to act day in and day out to try to achieve this. He saw over the horizon where cybersecurity is fundamentally what security is now about and is absolutely vital to the national interest and it was absolutely vital that we have a way in which young people are going to be able to help our country protect its secrets and protect its economy. Number two, with this position in the Senate as the chair of the Appropriations Committee and as a long-serving member on the Appropriations Committee what Patrick understands is it's not just money, that is really, really important. You've got a resource, but it's a sustained commitment. It's not a one-and-done kind of deal. It's like we've got to create an institution that has the capacity to deliver, to upgrade, to recruit students, to recruit faculty, to do all of those things that require institutional foundations that then allow students with motivation and energy and intelligence to be able to get the training and go forth and help us and you see that commitment of sustained support for an objective through an institution in what he did here and it was mentioned. 2009, a $15 million contract. 2013, $9.9 million homeland security contract. 2018, $7.3 million in three different grants and of course in 2021 Norwich University receives $19.5 million grant to lead the Defense Department Cyber Institute program and that's a sustained commitment by a leader who understands the importance of institutions to our society so that we can accomplish what our society needs. But the last part, and this is the secret sauce, he knows how to operate in the Senate. That is not easy. That is not easy. It's true. Through all of the noise of politics, when we're reading and mystified and what's going on here and there and who's up and who's down and all of the distractions, there's this steady force that's just moving forward just beneath the noise and it's Patrick talking to the center, talking to that center keeping things moving. It's not a straight line getting from here to there to accomplish what he's done but he knows when to whisper here, when to whisper there, when to pull back, when to push forward. That is like a skill. And my goodness, it's invisible to us here because we see fortunately the output but I remember being on a trip. Senator Leahy invited me on a trip with him and Senator Shelby who's the ranking member of the Appropriations Committee at that time I think and I saw him in action because we're on this trip, Patrick and Marcel, of course and it's all together most of the time but every once in a while I would be looking for Patrick and I would see him in the back of the plane or in the back of the room where we were meeting talking to Senator Shelby. Now if I had Patrick's acting skills I could mimic the southern accent of Senator Shelby but I won't try but you know somehow in some way this conservative senator from Alabama and this wonderful senator from Vermont found ways to do things that needed to be done in that case for both of their states. So the ability to actually do the work that's required to have the relationships that you have to sustain over time to have the trust of your colleagues where when you are asking for their support that is going to benefit in this case Norwich University that's trust that's earned day in and day out by the integrity that Patrick has brought to his work every single day and what I've always you know admired about Patrick is the advice he's given to me when I was serving in Congress your word is your bond your word is your bond and Patrick you know we're here in Norwich today we're so proud of what Norwich has done but we're really here just to acknowledge that none of this would have been possible without your dedication, your integrity and your extraordinary political skill so Patrick Leahy I just want to express the gratitude that all of us here at Norwich but all citizens of Vermont have to you and Marcel for the dedication you've given to us and to our country thank you. At this point I would like to invite the chairman of the Norwich University Board of Trustees Mr. Allen DeForest to the podium. Allen? I'd like to start by recognizing two very special people President Mark Anarumo and President Emeritus Snyder and without your leadership we would not be here today thank you for that leadership I'm so happy to recognize Senator Leahy in this formal manner his support sir your support will now live on for generations in 1998 the Norwich University Board of Trustees led by General Gordon Sullivan who's with us today gave the mandate that Norwich University become a leader in cyber education research and training it was the trustees who had the vision to say this is important for Norwich University this is important for the state of Vermont and this is important for the nation Senator Leahy has been a steadfast partner in supporting the growth of this signature program at Norwich University therefore it is my great pleasure as chairman of the Norwich University Board of Trustees and by unanimous approval by those Board of Trustees that we dedicate the Senator Patrick Leahy School of Cyber Security and advance computing and with the Leahy family please join us on stage for the unveiling also Mark is here Dean Temkin Michael Baddick and Frank Leija would you also come up and join us? So now I would like to invite Phil Seussman up to make some close remarks thanks Phil What a wonderful day Thank you Senator Leahy for all that you've done for Norwich University and really giving us a chance to be a national leader in cyber security and critical infrastructure protection Fabulous If I may just to interrupt a moment I want to tell you all part of the view of this session in Norwich I feel fortunate as a kid from Montpeyre that I could be in a position and do this but as Marcel has heard me say so many times I fought for this money because I believed in Norwich I knew it wouldn't be wasted I knew Norwich would make the investment worthwhile I knew that you would do the things needed to help protect our country and this is something that protects our country greatly and Phil you and I and the presidents of this and it means so much to me that you're there I can't tell you how touched I am by this I would have done this whether my name was on there or not because you're the ones doing the work I believe in Norwich I believe in all of you and I just say thank you very much and thank you to Peter Welch for the wonderful things he just said thank you sir and we're ready when you get on your bicycle again to host you on campus sir I will I'm walking slowly but I am walking and I will be on the bike but I'm going to be walking with you around the Norwich campus sometime soon thank you that sounds fabulous


Unscored interval IOA, inter-observer agreement for those of you that haven't caught on yet. So in this case, we're going to look for agreements of non-occurrence, right? So non-occurrence of a particular behavior. In this particular interval, Bob did not pick his nose, okay? If one person doesn't see it, then that's an interval that we're going to use to count, right? So we're going to count only the intervals that have non-occurrence, and then we're going to count the numbers of those intervals that the two observers agreed. So if both people didn't see Bob pick his nose during that interval, then great, there's an agreement. And the next one, maybe one person didn't see him and the other person dead, hey, there's a disagreement, right? So they disagree on whether it occurred or not, but it's going to be one of the ones we count because somebody didn't see it, right? So it grossly underestimates behavior. You take the total number of intervals of non-occurrence, divide it by, or take the total number of times when they agreed that the behavior did not occur, divide that by the total number of intervals that contained a non-occurrence from either one. You end up with a ratio of multiply that by 100, and you end up with the inner observer agreement.


As part of the enhancement of the efficiency of the production distribution supply chain in the fruits and vegetables sector project, the Department of Agriculture in collaboration with the Taiwan Technical Mission joined the Food and Agriculture Organization in commemorating World Food Day with the Farmers' Market. The Farmers' Market offered an opportunity for farmers as well as agriculture-related small businesses to sell their products directly to those who work, live and visit Castry Central. The initiative, which took place at Constitution Park, forms part of a series of farmers' markets hosted across St. Lucia under the Servant Crops Project. The products on display included organic produce, potted plants, and locally made products. Marketing specialists of the Taiwan Technical Mission, Eileen Chen, explains that with difficulties brought on by the COVID-19 pandemic in the agriculture industry, this initiative provides farmers with a good avenue to showcase their products and increase their sales. I think the most important purpose for farmers' markets is we let the local consumer be aware of the high-quality produce in St. Lucia, not just fresh produce. You can also see agro-processed food. So, along with that, it brings the consumer awareness about how St. Lucia's product could be high-quality. Project coordinator of the Servant Crops Project, Adeline Nurovik, indicated the farmers' market is in keeping with the goals of World Food Day as it promotes the healthy eating and sustainability of the agriculture industry. We decided to have our usual farmers' markets with a twist. So what we did for this World Food Day activity, we invited our usual supporters, that would be our wine producers, our flower. Different farmers have always been supporting us and we extended to other farmers throughout the island. But we have to remember that we're in the COVID season, so we were restricted as to the number of farmers. But lots of persons are interested in what have been happening under the Servant Crops Project and we have many positive outcomes and this farmer's market is the next outcome of the Servant Crops Project. Because it's World Food Day, we decided to have it and to make it a bigger one for our farmers to take part in. Ms. Nurovik noted that the Ministry of Agriculture, Fisheries, Food Security and Rural Development will continue to collaborate with the Taiwan technical mission under the Servant Crops Project to reach the overall goal of reducing the food import bill in St. Lucia. From the Communications Unit of the Ministry of Agriculture, I am Anisia Antoine reporting.


This is the momentous occasion for the Indian culture, science and the world to be celebrated. We are here to celebrate the moment of the world's most well-known founder, Swami Ramdev. This is a momentous occasion! It is such a life-like… It's difficult to figure out which one is real. The resemblance is really uncanny. With this, I would like to thank all the dignitaries, and I would request them to kindly take a seat in the front row. This is the momentous occasion for the Indian culture, science and the world to celebrate the moment of the world's most well-known founder, Swami Ramdev. This is the momentous occasion for the Indian culture, science and the world to be celebrated. I don't know how many thousands or how many lakhs of hair are there in this. Each and every hair has been put by hand. And I had applied this after the veneration in the morning, so I said, Baba ji, you apply it. Now, in this whole of Pursharth, in 2018, I went to the London attraction of Madam Trusad. There are 24 such attractions in the world. And the biggest attraction of New York is… As I told you, there is a number of them. There is Amitabh Bachchan ji, his daughter-in-law, Aishwarya Rai. They are their figures. And there is Diti Kroshan, Priyanka Chopra and Shah Rukh Khan. These are the five most famous celebrities. But along with all of them, along with the world's 200 renowned icons, they included a Sanyasi of India. And today, the whole of the world is dedicated to India. They are dedicated to Ayurveda. Now, even in America, made in the US and made by Americans. What concept is going on? The movement of Yoga, Ayurveda, Swadeshi, went from Patanjali. Its wave is in the whole of the world. Nathic ethics and spirituality. Nathic and spiritualism are based on such a teaching, in which there is a lot of language, knowledge of language, knowledge of subjects. Along with this, character based in curriculum. Spirituality based, root based education, the whole of the world. For this, a lot of work is going on in Patanjali. And I would like to say that in the next 5 to 10 years, our goal will be to introduce 5 to 10 lakh schools to Guru Kuli. The whole of the world is dedicated to Nirmal, Charit Nirmal. And a new direction of youth Nirmal, in our youth, in our students, is such a programming of their brain, which is from intellectual terrorism, religious terrorism, from political and corporate terrorism, saving the whole world, and in the correct way, the world will get the victory, and the children in that direction will be able to move forward in the society, the country and the world. The health seekers in the whole world, they want to get rid of their BP, their sugar, their thyroid arthritis. This has to be done with synthetic medicines. That is why the whole world is dedicated to Nigahe Bharat. And within Madam Tussade, I have asked this audience, to present a statue of a mother in the form of life. That is why they call it the wax figure. It is not presented with the statue or other words. It is presented with the statue of a mother in the form of life. So, they get about 2 crores of money. And to make a figure like mine, it would have been a little more expensive. So, the matter is not about the cost, it is about the concept. So, in the world, 24 such attractions have been made by Madam Tussade, and that is the biggest attraction of the world in New York City, on Times Square. And in that, she has given a statue of a mother in the form of life. For her, Kratajyatap Urvak, he is not respected by Swami Ramdev, he is respected by Yoga, Ayurveda. This is the cultural heritage of India, which we have inherited from our ancestors. This is its respect. And most of the people come there. And in the world, icons are not only from Hollywood or Hollywood. In the world, icons are not only from politics, not only from corporate world, not only from scientists. In the world too, icons can be seen. The respect they have given has been honoured. Here too, a visionary attitude is shown to them. So, today, I was asked how do you feel? I said, it feels like me.  So, when I went to Madame Tussaud Attraction in London, I was measured in about 200 ways. Now I have forgotten about it. And I was with the statues. We are still standing on the statues. If we are standing next to each other, they would have asked me to stand empty-handed. So, today, because this is a tradition, I can do only one action. I can do many. If you liked our video, then don't forget to like, share and subscribe to our channel.


Hi, I'm Stéphane Dascoli with a PhD student at ENS and working with Facebook. Stéphane, we cannot hear you. Okay, here I am. Okay, so hi everyone. Thanks Marc for the introduction. So today I'm going to be presenting some joint work with collaborators at ENS and Facebook, Maria Refiniti, Yelvet Sagon, Julio Biroli and Florence Akala. And we're going to be looking at the generalization behavior of neural networks and try to reconcile the surprising behavior with all the right ideas. So, yeah, on the first part of this talk, I'm going to be trying to reconcile the double descent with the convention wisdom of the bias variance trade-off, which you're probably familiar with, but I'm going to restate it here. So the original citation is that the price to pay for achieving low bias is high variance. In other words, when we increase the number of parameters or the complexity of the machine learning model, we expect to see underfitting when the number of parameters is too small, overfitting when it's too large, and therefore there should be a soft spot where the test error is low. So this is a U-shaped curve, but this seems not to be the true story in deep learning. Instead in deep learning, if you plot the generalization error of a neural network versus the number of parameters, you'll observe a particular interpolation threshold, which is denoted here as this dotted line, to the left of which you get the classical U-shaped curve here, and to the right of which, instead you get a monotonous decrease. And so at the interpolation threshold, which is the point where the training error vanishes, you get a characteristic peak in the test error. Now notice that this peak can be suppressed by performing regularization. For instance, in this paper, it was performed by early stopping, or even by ensembling, which is averaging the predictions of differently initialized networks. So what we like to do is understand what in the situation is happening to the bias and variance, as we knew them before. So to study this question, we're going to be looking at the random features model, which you'll probably hear about a couple of times during this conference. It was introduced as an approximation for kernel methods on large data sets, and essentially it can be viewed as a two-layer network where the first layer of weights theta is fixed, and we only learn the second layer of weights. Now this model was recently shown in a paper by Song-Mei and Montenari to display the double descent curve. But in addition to simpler models like linear regression, which also have a double descent behavior, in this model, we can disentangle the input dimension, in other words, the number of nodes in purple from the number of parameters in blue. And also we have a notion of non-linearity in the projection. But more crucially for the current study, in random feature models, you have this fixed first layer, which can mimic the effect of initialization in neural networks, and therefore can enable us to study the effect of ensembling. Now to introduce notations, because they're going to change a lot throughout the conference, I'm sure, we're going to denote as n, the number of training examples in the data sets x, as p, the number of hidden nodes here, and as d, the number of input dimensions. We're going to have a non-linearity sigma, which is going to be taken to be ReLU, and all the random quantities here are going to be sampled from a standard Gaussian. So our learner is going to take the following functional form, and it's going to be used to try and approximate ground truth, which is a linear function, plus some added noise epsilon. To learn the second layer of weights, we're going to perform ridge regression with parameter regularization parameter lambda, and we're going to evaluate the tester of the model on the fresh sample, again, Gaussian IID. So what we're going to want to do here is disentangle the various sources of variance, which come from the sources of randomness in our system. So this is going to be sequential decomposition. We're first going to remove the variance with respect to the noise and labels epsilon. And as you can see, once you've averaged over the label noise, you get an average predictor, which is closer to the true function in blue. Once we've done this, we're going to again remove the variance, but this time respect to the first layer of weights, theta. So the random features. And again, we're going to get an average predictor, which is closer to the ground truth. And finally, the last source of randomness is the sampling of the datasets. And once we've removed all the source of variance, we get the final bias, which is the distance from this fully average predictor to the ground truth. So to study this analytically, we're going to go to the high dimensional limits where N, D and P all go into infinity with their ratios fixed. And in this limit, we're going to use the replica method from statistical physics to compute the tester and the various sources of variance. And what you can see is that, first of all, the tester displays the double descent. But if we look at the various terms, which intervene, we'll see that the noise variance and the initialization variance both diverge at the interpolation threshold when there is no regularization. In stark contrast, you'll notice that the two remainder terms, which are the sampling variance and the bias, reach a plateau at the interpolation threshold with a distinctive kink here, which reminds us of a phase transition, and it can be smoothed out by regularization. So the conclusion of this is that the noise variance and the initialization variance lie at the heart of the deep double descent behavior. And in particular, the fact that they are monotonously decreasing after the interpolation threshold shows that over parameterizing is beneficial just due to the fact that it amounts to removing some variance. Now, what's nice with this decomposition is that it also allows us to study the effect of ensembling, which is again averaging the predictions of learners with different first layers, but trained on the same data set. So we're going to call K the number of such learners. And what you'll see is that as you increase the number of learners K, you'll drastically reduce the noise variance and the initialization variance, which we're causing the divergence at the interpolation threshold here. And when K goes to infinity, the divergence is completely suppressed. So the summary of this is that we reconciled this classical bias variance decomposition with a more modern one which takes place after the interpolation threshold, where in stark contrast with the previous bias variance decomposition, the variance is decreasing and the bias remains constant. So in the second part of this talk, I'd like to reconcile this double descent with an older version of double descent, which was known since the 90s in statistical physics in the perceptron. Indeed, if we look at the parameter wise profile, which is the test error you obtain when you plot versus the number of parameters as we just saw in your networks, you get a peak when P is equal to n number of training examples. If you look at linear networks, you won't get such a peak. However, you will get a peak when you plot the sample wise profile, which is the profile you obtain when you plot versus the number of training examples. However, there is one difference between nonlinear and linear networks is that in linear networks, the peak appears when n is equal to the input dimension D rather than P. And so if we plot this in the form of a phase space, a P and phase space, you'll notice that these two kinds of overfitting correspond to different lines in this phase space. However, they have one thing in common is that they both occur at the interpolation threshold exactly. And so what we'd like to answer is, are they the same thing? Are they due to the same phenomena? Well, the answer we provide here is that no. Indeed, if you look at the regression task with a lot of noise, what you'll notice is that the sample wise profile obtained when plotting versus n will present two peaks when the activation function is a weakly nonlinear. You observe a peak at n equals D, which is the linear peak, which is vertical on the phase space, and a nonlinear peak at n equals P, which is diagonal here. Now notice that when the function is purely linear, you only get the n equals D peak and when it's strongly nonlinear, you'll only get the n equals P peak. And so what we'd like to know here is what are the mechanisms underlying these peaks and how are they different? So to answer this question, we consider two different models. One, which is our usual RF model, which I've already introduced, where we get an RF student and a linear teacher. And we're going to denote by Z, the projection matrix, which is simply the data set once it's projected through the first layer and the nonlinearity. Now one quantity, which is important is the covariance of Z, whose bad conditioning, in other words, small eigenvalues are going to cause peaks in the test error. We compare this analytically tractable model to a numerical model of, which is a toy model for deep learning where we have both the teacher and the student, which are three layer networks. The teacher is randomly initialized and fixed, whereas the student has his weights trained by gradient descent, again on mean square error loss to try and mimic the labels of the teacher. So this is the, in both models, if you look at the P n face space of the test error, you'll notice two characteristic lines. The linear peak, which is at n equals D, which is the first overfitting peak, and then this nonlinear peak when n is equal to P, which is diagonal in the face space. To provide some description for this, we again go to our high dimensional limits, and we'll notice that one important quantity which emerges is the degree of nonlinearity, or rather the degree of linearity of the activation function. This quantity varies between zero and one, and for instance, to illustrate it, we can take the piecewise linear functions and continuously vary from the absolute value for which r equals zero to a linear function for which r equals to one. And what you'll notice is that by doing so, you're going to be reinforcing the linear peak at n equals D, but weakening the nonlinear peak at n equals P. Now, what are the mechanisms underlying these two piece? Well, to understand this, we consider the matrix Z, which we already discussed. And we know that in the high dimensional limit we consider, we can decompose this nonlinear projection as a linear term plus a nonlinear term, and the relative importance of these two terms is controlled by the degree of nonlinearity. These two terms are both going to give separate contributions to the covariance of the random features, which we denoted sigma previously. And investigating the properties of this covariance is going to help us identify the sources of the peaks. So we do this using some formulas established in random matrix theory. And so if we look at, first of all, the purely linear case where r is equal to one, we'll notice that the spectrum, the spectral density of the covariance matrix sigma has a vanishing gap, in other words, vanishingly small eigenvalues, only when n is equal to D. And this is what is causing the overfitting peak, which we called the linear peak. If we look at instead the purely nonlinear case r equals zero, we'll notice that the vanishing gap again occurs, but not at n equals to D, but instead at n equals to P. In both cases, the linear peak and nonlinear peak are caused by small eigenvalues. However, when we go to the intermediate case, for instance tanh, but I could also have showed ReLU. What you'll notice is that the n equals P gap survives. In other words, again, we see here vanishing eigenvalues when n equals to P. Instead, the n equals D gap is regularized. As you can see, the spectrum is cut off at some point. And so we don't have vanishing eigenvalues. So something else we'd like to do here is understand where the divergence is coming from in terms of bias and variance. So we already know from the first part of this talk that the nonlinear peak is caused by the noise and the initialization variance. But what we can see here is that the linear peak at n equals to D is only caused by the noise. Nothing special happens to the initialization variance. And one important consequence of this is that if you go to the noiseless case where there's no noise in the labels, the linear peak dies. However, the nonlinear peak survives. And this explains the curious phenomenon in deep learning that even when there's no noise, you can still observe this overfitting peak. So now I'm going to discuss two important differences between these two peaks in terms of phenomenology. So in the top row here I show the RF model results and in the bottom row I show the deep learning toy model. And what we see is that when we apply regularization, we can notice that the nonlinear peak is reduced both in both models. When we apply regularization, we'll notice, sorry, in the case of on something or regularization, you can notice that the nonlinear peak is reduced, but the linear peak is weakly affected. And the reason for this is that the linear peak is already implicitly regularized by the nonlinearity, as we saw from the analysis spectrum. And the important difference between these two peaks is that the linear peak, which again appears as this horizontal line in the face space, appears much earlier during training than the nonlinear peak, which appears diagonally here. And again, the reason for this is clear, because the nonlinear peak is caused by vanishingly small eigenvalues in the spectrum. And we know that those vanishingly small eigenvalues are much slower to learn, which is right there appear later during training. All right, thank you for your attention and my collaborators here.


Subramanian, our president of TNGA, Tamil Nadu Government Employees Association was suspended one day before due retirement. With a view to victimizing him, with a view to see that he is not getting his terminal benefit, to see that he is not getting his pension, by that way they want to harass him. By that way they want to send a message to the other employees that if you are indulging in struggles, if you are indulging in strike, if you are indulging to oppose the government's stand, then you will meet this type of victimization. They want to send that type of a message, but it is a very bad action. The government should withdraw the suspension, reinstate him and allow him to have his retirement and all terminal benefit and other things he has to. Otherwise, all democratic organizations like Trade Union, like Bank Employees Association, LIC Association, all those joint together, they will come to the street to oppose this, to see that he is getting all dues, pensions and other things. We will try to see that, that will be ensured. The government should come forward to reinstate immediately and stop this type of victimization.