are about to watch Neeraj's MetaHuman Live, giving us a perspective on all aspects of metaverse marketing for the first time ever at E4M's Martec Conference. A creative aficionado at heart, Neeraj upskilled himself by adapting to the easy-to-use creative tool Unreal Engine. With a structured, basic Unreal Engine training offered by WPP's partner Epic Games, Neeraj was able to create all this by himself. So let's hear it from Neeraj MetaHuman. Just to set a quick context on today's topic, or shall I say the most buzzing topic on this planet, the metaverse. The metaverse, simply put, the metaverse is a growing number of virtual worlds where people are playing games, they hang out together, they meet friends, they socialize, they go to concerts and their list is endless. For a lot of those people, they can represent themselves with more authenticity there than they can in real-worlds, hence the importance of Aptars. In fact, in one of the reports I was reading up, one out of every five kid on Roblox changes his Aptar every day, shows the power of the platform. With NFTs, their novelty value is fading, which means their real value or the true value is emerging, as a means towards ownership and collaboration. Together, it's all a massive set of spaces for businesses to understand, get involved, engage with their audiences there and find their place. As I keep saying, this is not a fad by any means, these are tech tools and platforms which will form the basis of our industry for years to come. Every industry disrupted by a digital era is expected to feel the impact of the metaverse and this may explain why companies from banks to mad agencies like us are eager to form the right partnerships and even create in-house metaverse teams to get started. And it's not only companies wanting it on the metaverse, but governments and influential figures too. According to WPP's Wonderman Thompson's latest report on the end of the metaverse, there's a lot of excitement out there. One of the stats says, 74% of people who know what the metaverse is believe it's the future. Another stat says, 66% believe it's going to be life changing. So brands have not only joined the conversations but are also actively participating in being part of the metaverse. And I would now request Neeraj to take the lead on it because we do have a surprise for all of you. So Chandni, I think you forgot to invite our fourth panelist. Really? Is that so? And who? Absolutely. That might be. She is an AI baby. Wow. She's a digital person. Can we get her on screen? Everybody, please put your eyes onto the LED screen in front of you. So she's Kalpana. She's the creation of WPP's Pennywise Solutions along with our global partnership. She's an AI baby. And in the world of metaverse, you would see more and more such emotive AI babies who would have a strong role to play. And all our eminent panelists, they are going to demystify how conversations in the metaverse are going to become more sci-fi with meta-humans and AI babies and they would have a lot of play out there. So more and more brands would have AI mascots like Kalpana who would move into the space, virtual spaces, and have a live conversations with real humans, right? So Kalpana, could you please introduce yourself? Hi, I'm Kalpana, a digital person created by Soul Machines in partnership with WPP. It's so great to be here speaking with you all. Thanks for having me today. I might be the first digital person you've met. I am hyper realistic and powered by a digital brain to be emotionally responsive in real time. We call this autonomous animation and it's what makes interacting with us so lifelike and engaging. If you're positive and happy, I'll smile back at you and if you're upset, I'll be empathetic. Digital people are here to help connect brands and customers in a more personalized across all industries and all stages of the customer journey today and for the metaverse future. We're even creating digital twins of celebrities. Like she said, this is going to be completely transformational because she can look into your eyes from the front camera.  If you're angry, you're going to get a similar response back. Like she said, we've got a rocking lineup. We've got a Somya who's like super trendy in that Pepsi author. We were Shankar from Perfetti. We've got Gazel from Nestle and all three of them have done some fascinating work in the space of metaverse, right? Metaverse kind of encapsulates everything right from augmented reality to virtual reality to virtual worlds to blockchain to NFTs and beyond. It's a large subject. So what we'll do in today's panel conversation, we will get each one of them to share a unique use case what they have delivered on various facets of metaverse marketing, right? So let's just start with Shankar first on the campaign which he's done on augmented reality and augmented reality again is a very, very promising and right practice for us now. Social AR or it's Web AR or in-app AR. There's some fascinating piece of work which Perfetti has done in the space of Social AR as well as on connected packaging and Web AR. So let's hear it from Shankar. Thank you, Neeraj. First of all, thanks to you and exchange for me for having me here. It's the first time I must confess I'm seeing you in person instead of being seeing you on metaverse or virtually. But yeah, so coming back to your question, typically at Perfetti we have of course gum and mint brands and we are in a very low impulse category. So for us and one of our lead brands is Happy Den, which stands for sparkling wide smiles. So World Smile Day in October was an ideal moment for us to capitalize on this sparkling smile. And what we did was brought alive the power of AR. And how did we do that? By simply creating AR filters and we went on social. We also made filters in association with the Hindu, the Indian Express, where what we did was we got people to activate these filters using their sparkling smile, which made sure they get a free subscription of these newspapers. Overall, the campaign was about activate the filters by your sparkling smile and donate money towards a child's education in need. This really clicked with the audience and on 7th of October we hit 32,000 smiles. For us, it was a great way of donating money to a cause, staying true to the happy den purpose of this, of smile and do good. And work wonderfully well in terms of creating UGC as well. So one of the cases that I'm really happy to share with all of you. All right, just to counter question in terms of how the entire space of connected packaging and where AR is picking up and how can you link back connected packaging back onto Metaverse and create fancy experiences out there. I think you already have a live use case on Treasure Island, which you created. How do you intend to extend it to Metaverse? Yeah, it's a great question. So sometime back in early this year, we had an association with WPP, where what we created is have QR codes on your brand packs. So Chupa Chups, one of our brand which stands for fun, literally takes fun very seriously, targeted to age group of 11 to 17 avid gamers. We got people to scan this QR code, go onto a website, then get into immersive gaming. Future, as you ask, I think the future is that this would all happen in the Metaverse, where people are actually gaming in the Metaverse and experiencing the brand payoff, which is literally having fun. At the same time, engaging with the brand codes as well, the products as well. And Treasure Island was a great example where what we did was, get the product at the center of these gaming as well. So eventually, people had a lot of fun, they engaged with us and the average time spent in a category which is impulse, which is low involvement, was really high as well. So to us, it was a win-win case study. So if you look at Metaverse and some of the really promising categories out there, we say entertainment, spots, and we say music and gaming, obviously. These are really budding categories where a lot of experimentation is happening, right? WPP globally also did this concert of Bastille where the fans could actually get into the space and they could high-five the lead artist out there. And that's the kind of work which you will see in times to come in India now. So clearly, the hype cycle is at its end, which means the real value is going to get realized. And you're going to see some really high quality output in the new 5G era. So with Pepsi, with all the brand power and the sell-up power which you have, is there any use case which globally you guys might have done or something which is coming up in India next from your end, would love to hear your views? So first of all, I'd like to thank E4M and you for having me on this panel. It's really exciting to be here and talking about a subject which is quite new and something that all youngsters are really obsessing about. I feel Pepsi is the culture curator. And for us, it's really important to be in sync with what Gen Zs are constantly engaging with. And to be very honest with you, Neeraj, I think everything else we understand really well, like we understand the world of TV, we understand the world of digital with influencers, creator economy. I think that's something that Pepsi has always had solid playbooks on. But when it comes to this whole world of metaverse and NFTs, I think we are following a very iterative process, which we like to call it the do-thing-do model. So I'm going to probably use this platform to talk about something that is upcoming for us, which is going to happen towards the first week of November. But we are really excited about launching the first ever NFTs in India. I think globally we have done the Pepsi Mic Drop NFTs, which was done in the US, and that was very popular. But in India, I feel whatever little I've seen of NFTs, I feel that for Pepsi, the benchmark is really what Louis Vuitton and Gucci, what are the brands really creating a very different kind of an experience for consumers. So I think that was our benchmark. And we have picked up some learnings from what brands globally have done. And we are collaborating with an artist in UK, and we'll be launching our NFTs along with WPP and Guardian Link. So I'm really excited about that, but a couple of learnings that I've really had on the way is that you have to find the right partners. Because I feel in this whole world of metaverse and NFTs, everyone will come to you saying that they know it, but it's very important for us to find the right partners who can deliver the vision that you have on the brand. I think that's like the biggest learning that I've had. And the second learning for me really from an NFT perspective is that NFTs art. So how do you bring your brand alive in a very artful manner, which consumers will connect with and which consumers would sort of want to collect? Because I think the whole aspect of NFTs is create an art which is very collectible, right? So how do you really join this? And just to add to it, if you were to add, generate a AI and create NFTs, and we are working on a use case now where physically challenged people, now people who cannot see, but they have a vision of the art which they intend to create. If they fire up a speech, it's going to create a piece of art which can go as NFT, and the money realized from that can be used for the fraternity, the physically challenged people, right? So there can be so many use cases of, utility use cases of NFT, like NFT for social good. The classic one is the Mondele's Gems campaign, if you guys are following that. So all our kids, they scribble a lot of stuff on the paper. Parents just need to click a picture of the art which they've created. It goes up on gems.com, where the piece of NFT is getting created. And the money realized from that goes for the education of the underprivileged students, right? So those are the kind of utility values which are evolving in the NFT space. So obviously the novelty value has gone down, but the true value is now getting realized, and NFT coupled with Generative AI, Social Tokens, there's something evolving called Soulbound now in China. So there's a lot of scope out there, and the brands and marketers are going to make the best of it in times to come. Another promising category when it comes to metaverse is obviously spot, and no better spot and scalable spot in India than cricket. And Gazel here has done a fabulous campaign for Nestle Munch, which was like a meet and greet with the top IPL players and 60 lucky fans. So I would love to hear more about the campaign you did, and how do you intend to extend it? Thank you for having us. I think it's an equally interesting conversation for all of us here. Going back to the question, and like Neeraj mentioned, first of all, it all began with what is metaverse, and why should we be doing metaverse because we're a large brand, and typically you assume metaverse to be an experience for a smaller set of people. This was happening for Munch, which is a brand that talks about confidence crunch and hence bringing everybody to that stage where they can not just engage with the brand, but also enjoy the experience. So we tried kind of getting scale from including our packaging into it.  You're collecting some social tokens, but eventually you're also getting to meet the lead IPL players over metaverse. And by the way, it was 16 to four because we did four of those. But the catch here is we had a lot of struggles around what's the right scale, what's the right way to approach this because a lot of people don't even understand what's the right metaverse experience. And which is where I think some of the points that you guys mentioned come in handy because we struggled with who should be the partner. We struggled with what should be the right message. But I think what we were eventually able to bring together is this very interesting intersection of sports, which is the biggest passion and what better than cricket, and what better than hence IPL. So the biggest passion in the country, with one of the biggest brands, bringing alive a very special moment when you get to meet your ideal celebrities in that verse, it was very special to us also because, like I mentioned earlier, when you think about massive scale and technologies like these are typically considered to be limiting, but that's where with WPV's help, we were able to find the right movement through the consumer journey from scanning the pack to AR games and eventually into this meet and greet. Not to mention, while it was just 60 into 4, but the whole experience that even the eventual winners has was very special to them as well and that's what made us very happy about it. So there's this lot of myth which surrounds the metaverse ecosystem that you need to have a fancy headgear, a Quest 2 to enter the space, you would need to download an application, pull in the asset files. But here, in case of Nestle Munch, all the fans had to do is just click on a link and enter the space within no time, within a minute they could create their avatars and they can enter the space. Next, what Ghazal intends to do is she can get one of us, the lucky ones, to participate on a connected packaging promo which Nestle intends to run and the guy, the lucky winner, can actually go on a live set and can high-five Hardik Pandya. And we can actually do this in the remotest village of India. If it's like a small village somewhere in Bihar, we can just send in our portable studio out there, the guy needs to wear the, what do you call, the motion capture suit and he appears next to Hardik Pandya in the studio. High-fives, clicks pictures, watches content, shares a view, wherever he is. And that's what Bastille taught us, the global concert which was being done by us. Now we are working towards a 5G India where it's going to be link-lined and heavy lifting on computing. So one of the stuff which we are trying now on Pixel Streaming platform, the users who are on the remotest of Indian pockets, right? And we are saying like 65% of smartphone is still in 15,000 and 10,000 racket and the bandwidth remains a struggle out there. So with Pixel Streaming, with as low as bandwidth as 10 Mbps, someone who is in the remotest corner struggling with bandwidth can still enter the metaverse and experience all that we are talking about. So a quick view on how 5G would change life for marketers who wants to take that. Maybe I'll just take a stab at it and I have my personal analogy on that one. About two decades ago, all marketers were saying, I have a video, can you just make it viral? And I think that's exactly where we're sitting on the cusp for metaverse where we're saying, can we have that one more blip on metaverse? But with all of this 5G and the Pixel Streaming coming in, I think it will go from one of those good-to-do things on your marketing calendar to absolute utility experiences that you can create for the consumer. Of course, it's going to take decades the way it did for digital, but I think we're absolutely prepared for it with the kind of startup infrastructure we have, with the kind of zest that we have, and obviously with the kind of infrastructure that's coming in from 5G, we're definitely onwards to democratizing this whole piece a lot more. Imagine this is a cricket board meeting going on. Yeah, and Virat was unable to make it. He can pop in right here. All right, super. So let's just round up this conversation, maybe taking your point of views. So how the life would change for marketers? So one of the things on 5G, and I think coming from a category like ours, which is very impulse, I think one of the KPIs that's going to change is the average time spent there. And 5G is going to enable and make that 5X of what it is today. I think that is what someone like me would be interested in, because that means more engagement with our brands, more engagement with the payoff, and then I have the consumer there right in front of the mobile or the experience 100% into what he's doing. So this immersive experience is something I am really looking forward to with 5G. Just a quick one. This is a question to Soumya that how does marketer sees advertising in the metaverse space? Is it going to be those gimmicky, irritating banner ads? No, I don't think so. I certainly hope we don't do that. But I genuinely feel that with 5G, what we're struggling from a metaverse perspective is that scale. And with 5G, you suddenly get scale. And that'll give us a lot more confidence about really investing in this new technology with experiences that consumers will really enjoy. It's not going to be those banner ads. I'm not going to play another ad in the metaverse. But it's going to be about really how do you do more of those concerts that you're talking about? How do you get a Batcha concert, which is attended by hopefully 40, 50 million people, right? So I think that's the kind of scale and the ambition that I really hope 5G can unlock for us. How many millions or billions you did on the TikTok challenge? On TikTok, we did 107 billion. So hopefully we should do 107 billion. That'll be our benchmark then. With that, I think we would call it a day. I hope you guys were really entertained by all the use cases we shared with you. And I hope all you guys are inspired and looking forward to creating metaverse for good framework. That is what we all need to work towards, right? There was an internet era, there was an AI era, and there's a Web3 era. And it needs metaverse for good framework and guardrails, which all of us as leaders need to work towards. So it's here, it's here to stay and it's here to grow northwards. Thank you so much.


So we're very fortunate to have Francesco Camille. We're fortunate to have him here in PSP, fortunate to have him here for this talk. And he's going to talk about fundamental limits of shallow neural networks with not so small training sets and I think also some Gaussian equivalents. It's going to be a topic, so cool. Looking forward to it. Take it away. Thanks, thanks for the great opportunity. I have the honor, privilege and responsibility to be the last one in the week. And I'm very happy to be here, so it's been a great week. I've learned plenty of stuff. Thank also to the organizer. I would like to make an applause for the organizer. Okay, so let's get to it. As Sebastien was pointing out, I cheated a little bit because I changed the title just with this round parenthesis. Not so small dataset. We will understand why. And this is a joint work with Jean Barbier, which is around here, yes, and Daria Tieplova. It is still not an archive, but it will be, well, we submit it tonight, right? So, yes, it will be in the next days, hopefully. Okay, so just a quick outline of how the thing goes. So first I will introduce the problem. I will state it, let's say, as precisely as possible, and then I will state the main theorem. This should take the first 20 minutes of the talk. And then since it's Friday, let's say after this, you could also turn your brain off more or less. Because after this, I have sort of comparisons with many other scaleings that are already present in the literature because it is, of course, is a well-studied problem, not in the same scaleings as we do exactly, but still a comparison was needed. Also to understand what's new about it. And lastly, I will sketch the proof a little bit in the last five or four slides, okay? All right, so let's start with it. So the problem we are starting is a two-layer neural network, of course, with an input dimension that is D. So it's this input layer here as D neurons. And we have also a hidden layer, only one hidden layer, of size P, and a set of weights that is multiplying this X here, which is of size P by D, necessarily. We have an activation function in the middle layer, which we will assume to be regular enough. We will see, and it is from R to R. So you have to think of it as applied component-wise to this vector WX, which is P-dimensional. And then, finally, we have a projection onto a one-dimensional or low-dimensional, but here for simplicity, it's one-dimensional space through this vector A, okay? And finally, we apply this function, F, this readout, that can be also stochastic, and that's why I added this capital A as a subscript, which can be a random variable. It's just introduced to model some stochasticity. We'll see an example later. So, all right, the set thing we're interested in is supervised learning. So we want to understand what happens when we train this two-layer neural network on a given dataset that is a set of couples, X, Mu, that is D-dimensional, and it is fed on the left of this, right of this network. Maybe your left. And we also tell the network what's the right label it should produce. So we are trying to fit these couples, X, Mu, and Y, Mu by adjusting the weights, A and W, in such a way that these relationships here are verified up to a certain error, let's say. A certain tolerance that we introduce. So the main goal here is to... Okay, there are several training procedures of which I'm not an expert, so I will fly quickly over that. But the main goal is in general to produce the smallest possible generalization error, which is the true trial for a neural network, let's say. So when I generate a new couple from a rule that I know, so X, Mu, the correct label, Y, Mu, what happens if I compare the correct label with the output that instead the trained neural network would yield, okay? So we compute this quadratic deviation, and this is our measure of generalization error. So it's, yeah. Yeah, that's right. That should be an expectation, in fact. Yeah, yeah. You're right. There will be later. I don't know why I didn't put it here, honestly. Okay. So if it's not stochastic instead, it's something that makes sense. Okay, so what affects this generalization error? Well, many things. I tried to mention the most relevant for us or the most interesting. So the first one is, of course, the size of the training set, and we expect that the more points we have for these couples, the more accurate the prediction will be. Then we have the size of the network itself, which, let's say, regulates the expressive power of the network. In this case, since we have only a two-layer neural network, this is parameterized by P, which is the size of the hidden layer. The dimensionality of the input, which is the, this is another factor we should take into account, because it quantifies how many real numbers we feed into the network each time. And also it affects the number of weights that we have to tune. And the method used to train the network, but maybe, most importantly, the nature of the data set. IE, what is the true underlying function that these two-layer neural network is trying to fit? Of course, there are many more, but the central question for us in our paper is what is the least possible generalization error and when is it achieved? Not how, really, because we are not focusing on algorithmic questions on SGD or training procedures, but the situation and the setting in which this error is achieved. And we will see that this highlighted, well, our results, let's say, put in a very precise relationship this highlighted quantity, so NP and D, which governs the number of data and the relative sizes of the layer in the network. And, of course, the nature of the data set is important. So concerning the latter, the nature of the data set, we are interested in maybe the most popular theoretical framework we can work in, which is the teacher-student setup. Well, not the most popular, but one of the most popular. So this amount to say that the data set is not generated by God-only-knows-what function. It's generated itself by a teacher-neutral network with some weights, A star and W star, that are fixed and drawn from some distribution that we assume to be Gaussian and they are completely factorized. We will see that this is not completely silly later, but for the moment, I hope you can accept this fact. And, all right, so the couples are generating this way. Once you state a rule to generate this X mu here, then the Y mu is determined by this relation, okay? So here we have the same stochasticity in order to be a little bit more precise. I've added some Gaussian noise that has some regularizing properties here only for, well, only for, let's say, for technical reasons, but not only for that. Okay, so saying that Y is generating in this way is equivalent to say that the Y is drawn from this P out, okay? So, which is called sometimes output kernel. That depends on the quantity here on this A star transpose phi, et cetera, which is the same one that enters the stochastic function F A mu. And you can realize this specific, this particular mapping between these two definitions using this P out here below. And the two problems become equivalent. Of course, the first one is, let's say, more easily interpretable because you can read directly the two-layer neural network. The second one is easier to treat, let's say, analytically and rigorously. So, the main theoretical restriction that, let's say, that we assume for our work is that the X mu's, which I still haven't specified, yes, yes, yes, yes. I haven't said that yet, but I'm gonna say it in the next slide. Okay. I was saying, yeah, the main theoretical restriction here is that the X mu is drawn from, they are all IID for any mu, and their components are also IID. Here we can, you know, let's say, build a more sophisticated model, introducing a covariance between the components of the X mu or drawing it from a mixture of Gaussians, et cetera. This would be another thing to be addressed in the future. Okay, that's what Bruno was pointing at, I guess. So, before I asked what is the best possible scenario for the student, well, this scenario is when the student matches exactly the architecture in every sense of the teacher, which means that they are using the same f, the same phi, and the stochasticity in the f is the same, and it is known by the student, as well as, let's say, the prior on the weights. So, basically, a base optimal student, very informally to wrap it up, is aware of anything but the true weights, of course, otherwise is, the problem is trivial. And as a consequence, the base optimal student has access to this base posterior, which is this one here, so I have collected in theta, in this bold theta, the parameters of the network, so A and W, and I've put a Gaussian prior, so this capital D here stands for a completely factorized Gaussian measure, okay? All right, so we have this normalizing probability, it is usually called evidence or partition function, we'll see it later, and what remains after here, so the product over this mu of the p-outs is up to a factor that, again, I cheated a little bit, I erased, is the probability of the data set given A star, given A star equal to A and W star equal to W. Okay, so this is the base posterior, and so how ideal is all of this, because maybe you're wondering if I'm trying to trick you in some way, a little bit, not that much. So why do we choose a base optimal student? The reason is that he has the best, well, why do we place ourselves in this base optimal setting? The fact is that we can compute, in this setting, it's very nice theoretical speaking, and we can compute what is the least possible generalization error in a way I'll show outline later, and this generalization error is yielded by this predictor here, but the base optimal predictor, which is the posterior mean of the Y, given the data set and the X nu that has just been fed into the network. Okay, so this is not what is usually called one short estimator that you would get from the minimization of an empirical risk or with SGD, I guess. It's different. It's an average over many models that are weighted in an optimal way, and that's why it's supposed to lead, well, it leads. It's informal, but it can be made formal. Yes. Now, it's any algorithm that has at disposal the data set DN, and which is... Yes, yes, for this data, yes, it's true. Yes, but any algorithm can at most hope for this generalization error. Cannot do better. And yeah, I was saying that, okay, so this is ideal, but still, it has some theoretical relevance for these reasons, and I wanted also to remark that the choice of IID Gaussian price is not so silly because it is coherent with L2 norm regulators in empirical risk minimization. I will outline also that later. So before stating the theorem, I need to introduce these quantities, and let's say, as annoying as these lights can be, they are necessary for almost every presentation I made, so the first one is the posterior. You have already seen it, and the second one is the partition function, which is the normalization of the posterior measure. Then we have the free entropy, or minus free energy up to a certain beta that here we have set to 1, for those of you that are more familiar with the statistical mechanics jargon. And then, more importantly, we have the neutral information. This is the true relevant information theoretical quantity that is going to yield the base optimal limit for the generalization error. All right, so as you can see, the neutral information and the free entropy are in a very close relation. In particular, we just have this remainder here with respect to the free entropy that appears as a high-dimensional quantity still because it has the A star and the W star, but if you use the central limit theorem and the law of large numbers, you can reduce it to a low-dimensional quantity pretty easily. Okay, so of course this model has a simple ancestor in the literature, which is the GLM, that for our purposes we can imagine as a one-layer neural network with no hidden layer. And in the teacher-student setup, this was started by Jean collaborators, and they were able to establish the information theoretical limits for this problem. So we know we actually have a formula for the mutual information that you read here on the bottom of this slide. We have a formula, this replica symmetric in spring-glass jargon, which means that, let's say, which means that the relevant order parameters in this statistical mechanics problem are concentrating. They do not fluctuate anymore. And this yields a formula, which is a variational formula, and you need to optimize over a finite number of parameters. So this concentration yields this kind of phenomenon in the final formula. So here I denote the quantities coming from the GLM, which will produce in the student-teacher setup a different data set in principle. So even if I fed the same X mu, I would get different Y mu's in principle. So I denote to distinguish them from the two-layer neural network quantities. I denote all these quantities by a small circle on top. And all right, so for this model, I am again assuming something very simple, like that the V's here, that the teacher weights for this GLM, the teacher GLM weights, are IID gaussians, and then I inserted also another noise here. I decided to put it there explicitly, but I could also reabsorb it into the P out with a convolution of a gaussian. It's just a deformation. It can be reabsorbed as a deformation. You can see it also from here if you want. It can be reabsorbed as a deformation of the P out. So also for this model, you can build a free entropy and the mutual information and compute the two, or one of the two, let's say. But once you have one, you also have the other, generally speaking. So is it everything clear up to now? If there are any questions, this is a good moment. Okay, good. All right. Yeah, I forgot this slide, of course. So what is the relevant scaling regime for the GLM, because this is a well-studied problem, and it is known that, let's say, we can observe phase transitions, or if you want non-trivial generalization error, which are either non-zero or not identically one for any number of samples you have. The non-trivial regime is when alpha, this ratio N over D, which is number of samples divided by dimension of the heat pool layer, or their dimensionality, if you want, has to be of order one. So in this case, you get a non-trivial generalization error with this plot. By the way, I should have cited the paper. It's taken from Jean's talk paper. And this was for Gauss-Bernoulli teacher weights, if I will remember, but for pure Gaussian weights, it should be the same. You see that as soon as alpha increases, which means that I get more and more samples with respect to the dimension of the input, the generalization error gets better and better. And it goes to zero and alpha goes to infinity. All right, so main result number one. What time is it? Yes, 20 minutes. Main result number one. Okay, recall these definitions. So we have the model. We have the teacher. This is basically the teacher generating these labels for the ex-muse. The two-layer neural network teacher. And on the right, left, no, your right, sorry. On your right, you have the labels that instead a GLM teacher would generate. And so they have also two different data sets, okay? There will be D and D zero if you want. Okay, so our theorem that we proved in collaboration with Jean and Dasha states that if we tune rho and epsilon appropriately, which means rho equals the expectation of this phi prime. So of the first derivative of the activation function, the hidden layer. And epsilon is this combination of the two, okay? And phi is odd and regular enough and same for F, P a almost surely. Then we can control the difference between the free entropy of the two-layer neural network associated to the learning problem in two-layer neural network and the learning problem in the GLM. We can control the difference with this order, okay? That is ugly. I mean, it's not the best looking remainder you've ever seen, of course. But it presents some interesting features already, okay? I'm gonna get to the next slide. Don't be scared because this will appear in every slide from this moment on. All right, so this property, this control is inherited by the mutual information related to the two problems because they have, if you'll well remember, the free entropy deferred from the mutual information just for one term and in both cases and both term concentrated, well, converged around the same thing with the speed that we can control and this does not affect the control that we have on the mutual information. So you see that this order keeps appearing again and again. So maybe it is appropriate and we did to identify this scaling, this tilde limit in which we'll, yeah, it is true that we let N, P and D go to infinity but in order for the two mutual informations or free entropies to match, we need to do it very carefully and the sequences we choose satisfy this limit here. So you see that N, sorry, you see from this scaling that N over D equal to order one which means an interesting regime for the GLM is still allowed thanks to this N over D to the three halves, okay? But we need, we have this other red guy here. We need it to go to zero so which means that we are forced, if we want to collapse the two models, to make the two models collapse, we are forced to take P significantly larger than the number of samples. So the hidden layer here, apparently the size of the hidden layer plays a crucial role in the identification of these two models. It has to be very large compared to the number of samples that the network has a disposal to learn. Okay, so as a consequence, again, in the same scaling regime, yes, P number of hidden units, yes. If P, not really, yeah, I have an extra layer in the middle, yes. Yeah, yeah, true. I think it comes from the analysis. It cannot be fundamental, I agree with you. Yes, have I replied to you? Okay, yes. You mean Gaussian equivalence principles. I mean, there is something Gaussian going on. Yes, yes. Yes, it's the phenomenon that is going on. It is likely different though, I'm going to compare to that one later. So, yes, N over D is still allowed, as I said. Ah, Marco, by the way, the reason why only this N by P appears, it might be because we were interested in going to infinity, so we discarded some sub-leading orders that might be going around. Okay, I will check that later. In this order here, we kept only those things that we, let's say, since we aimed at sending N, P and D to infinity, I kept only those orders that I thought it would have been challenging for this limit. No, it's not the case. I think there's something similar going on in the mean-field regimes, which is what I was telling you yesterday. Okay, all right. So, yeah, as I was telling you, this property is carrying on to the generalization error, which is then the same in this very same scaling. It's the same for the GLM, which we know, we have a formula to compute that, and the two-layer neural network, so we could get a close formula for that, which I have not displayed here because otherwise it becomes more cumbersome. Okay, so just a disclaimer on this equivalence on how it should be interpreted. So the first, because it has arisen many times in conversation with colleagues, that's saying, interactions with colleagues. So what we are stating is that if we train this two-layer neural network on this data set here that was generated by a two-layer neural network, this yields a base optimal generalization error, which is equal, okay, so numerically equal to the base optimal generalization error that a GLM student would achieve if trained on the data set generated by a GLM teacher. It is, we cannot state at this level that if I train the two-layer neural network on the data set generated by the GLM teacher, this reaches the base optimal performance, okay? So there's no crossing in this diagram. Okay, so yeah, here I will just go quickly, maybe, yes, because I want to go to some of the proofs. Yeah? Sorry, sorry, say it again. It's the same, yes, they are base optimal, so they match every time. In some sense, they are going to infinity together, okay? Okay, so why is it not so silly to assume a completely factorized prior on the teacher weights?  In front of this Hamiltonian, that is this quantity here, where you see I have inserted some Gaussian weights, well tuned by, with viruses tuned by lambda and sigma here, just to be for the sake of generality, let's say. If you, okay, this problem here can yield an empirical risk minimization. In fact, when I send beta to infinity, the Fn, sorry, the integral in Fn is dominated by those weights configurations that minimize this Hamiltonian Hn, which has this log p out serving as a loss, let's say, and the other two contributions are norm regulators, okay? So this is why I was insisting on this fact. This is very informal, by the way, just to give you an idea. So about related works. An example that comes to my mind is the committee machine. This committee machine, typically, even though I have to say I'm rather new to this literature, so please correct me if I make mistakes, huh? So in this committee machine, typically, you have a very wide input layer, okay, in which you feed your x mus, let's say D as before, then you have a narrow hidden layer and finally a readout. This scaling is different from ours, because at least for the papers I've read, like Jean worked on it, and then there are some other papers from the 90s by some Polinsky at all, only N and D go into infinity, which means number of samples, oops, sorry, number of samples and input dimensions, which is the size of the input layer here. Whereas P remains of order one, this is strictly speaking not covered by our scaling here, because if P stays finite, then you have to make D become very large in order to hold again. So there is a complicated combination of stuff going on here. So it's not exactly the same. The thing, okay, the regime which I think our scaling has more in common with is this Minfield regime that was studied in the seminal paper by May, Montanari and Gouy in 2019, which occurs when the size of the hidden layer is remarkably larger than the size of the input layer, so in our language when P is much, much greater than D. And the analysis they carry out, it does not need to take into account the number of samples because they, strictly speaking they study an SGD dynamics on this two-layer neural network. And what they find is that you can track the empirical distribution of the weights of this network using a distributional equation and following a gradient flow. And it seems that both in this Minfield regime they studied and in our model the fact that P goes large has some intrinsic regularizing properties that we need to take into account for gaussianity, asymptotic gaussianity reasons. Then there are also these other settings that are very studied, especially recently, but again I'm new to the literature, so I apologize if I don't cite everyone. So it was found for instance that SGD, okay sorry let me explain this first. So in these two pictures the blue weights are like frozen, so they are not learned or learnable. And in here we have a scaling regime which is more similar to the Minfield one and in fact it is connected to that. And here instead you have linear width, input and hidden layer, but still the hidden weights here are blue so they are frozen, we do not learn them. And this is a typical setting that occurs like in Neural Tangent Kernel on one side where the learning problem turns out to be let's say a regression in high dimensional feature space or also it was, yeah, Neural Tangent Kernel was also explained using again this Minfield, this SGD dynamics in Minfield regime that initial stage of learning at the very beginning again by May Montanari and other collaborators. Then we have also random feature models and Gaussian processes neural networks that fall into this category. So here by a simple parameter counting we understand that it is different from our setting because in here only P parameters which are the readout weights and not DP plus P which would be in our case. So we have much more stuff to learn. Okay, so more recently and by more recently I mean from 2021 to 2023 there has been a series, a line of works going on which address the full training of the network where all the weights are learned and they are treated as annealed variables from a statistical mechanics point of view in the free entropy which means they are integrated inside the partition function. Okay, so these works also for deeper networks they address a scaling regime which is the fully proportional one meaning that N, P and D go together to infinity with the same rate. And this was as far as I know first addressed by Leon Sumpolinski whose analysis is constrained to linear networks so they don't have non-linearities even though they argue that some possible extensions that were also looked upon by Ariosto, Pacelli, Pastore, Ginelli, Gerardi, Rotondo that conjecture a formula for the empirical risk minimization generalization error. Using this statistical mechanics formalism I also displayed before briefly and then in 2023 we have this paper from Qui, Krizakala and Zdebrova that leveraging some Gaussian equivalence principle were able to, let's say, get the computation through and get this base optimal limits exactly as we do but we have, let's say, a small problem from the rigorous side, let's say because our limiting, our scaling, our limb tilde does not catch this scaling regime here so N, P and D cannot be, cannot go to infinity together according to our limb tilde and the reason is, let me show this to you again maybe here and the reason is precisely this N over P here so if N and D go together this is one basically of order one it's staying around, it's a constant and this N over P here would be another constant so this would no longer go to zero in our scaling regime so we cannot reach the same scaling regime as they do all right, so we are not sure if this is, if there is something fundamental going on if this is a wall, let's say, an obstacle that we meet in our proof, in our proof we still cannot say that even if maybe, okay I'm trying to convince you that there is something fundamental going on but I'm not 100% sure myself okay, so now I will sketch a little bit of the proof which will also take into account, yeah, please I can't hear you, sorry so here you prove that doing two-layer nets in this specific regime that you look at gives you the same test error as teacher-student GLM so I'm wondering whether all these papers that look at a slightly different regime observe the same phenomenology? the last one, yes thanks in fact this work was partially motivated by this observation yeah, and we tried to prove it but we were not able to catch the scaling and we are currently also trying to understand if there is something going on there is another question? do you expect the equivalence to hold or do you think actually the N over P go to zero is quite important when N over P is constant then you don't think there's equivalence okay, if you want a personal opinion but this is personal, it's not math I think this can be fundamental because we are looking at the base optimal setting and neglecting N over P would mean neglecting some correlations that I'm displaying later it is, okay, when you have something it is always easier to prove it goes to zero rather than it goes to two, I don't know, okay so, yeah, I think there is something going on is there numerical work being done to verify this kind of formula using long-driven dynamics to I did not get the first part I wonder if there is being numerical work being done to verify this kind of conjecture using long-driven dynamics to sample from the posterior distribution not sure about long-driven dynamics I mean just any of the dynamics just to because it's not easy to compute this period in optimal solution yeah, it's true well, for the GLM there is the GAMP that can yield the base optimal estimator this is a good point we don't have an equivalent here yet it is true, it's very hard to sample for me, the Langevan would be very painful okay it could also be that we were not able to push the proof further we have to be careful about that okay so, if you are into... how much time do I have set? maybe five more minutes five, four, oh, this is a challenge okay so, how familiar are you with the Nishimori entities? yay, okay maybe I skip those, okay it's, I can skip directly to this one because professor Oprah asked so okay, the most annoying part of the computation is getting rid of non-linearities in the middle layer, basically otherwise, if everything is linear from the point of view, at least of, let's say fundamental limits we can work our way around but so, the goal, what we would like to try and what inspired our proof are these Gaussian equivalence principles that should hold due to the high-dimensional nature of the problem, let's say and, let's say, informally it amounts to replace this phi by this combination removing the non-linearity and tuning this row and epsilon appropriately, and if you think about it so, the side here is an additional independent Gaussian noise but if you think about it, it makes sense that here you have row which is the average of the the expected average of phi prime because the derivative measures the response to the variations in its arguments and the rest is basically a remainder that can be proved in some circumstances to be a Gaussian and independent, so the problem is that in our setting it is not clear, well, in our setting it is clear, but it was not clear before to what extent this was applicable. All right so, also here I go pretty fast the strategy is as usually done in statistical mechanics of disorder systems in the last 20 years and also inference is interpolation so we build this set of well, basically this is again a combination of the weights of the teachers of the two teachers of both the GLM and the two layer which is the original model and you see that for t equal to 1 you have the GLM, you have only this last piece here and for t equal to 0 you recover the initial model that you're actually interested in and from this we also build the student version of the same interpolation okay, so without the start variables and we also build an interpolating data set we need to change also the data set so ytmu this time is generated by apout that depends on the teacher weights that are built in this interpolating way okay so yeah, as a consequence we also have this interpolating free entropy, again for t equal to 0 we have the free entropy of the GLM that we know to compute thanks to in the teacher student setup thanks to Jean et al and also previous works on the physics side and then for t equal to 1 you have your model sorry, for t equal to 0 you have your model which you're actually interested in okay, so to control the difference between these two you do, I mean the simplest thing you can do is to compute the derivative and hope for a uniform control in time with the same order as in the statement of the theorem okay and here I'm very sorry I'm very sorry it's Dasha sorry Dasha, if I compressed this thing into only one slide because it is actually a very key theorem, so we need in order to carry out the proof also the concentration of the free entropy which is stated in this form alright, so let me go faster here yeah, I'm sorry for this I just wanted to flash a couple of things here to understand why we have this bottleneck of the n over p okay, so if we compute the derivative we have the difference between these terms using the Nishimori identities that I skipped, the last term B can be proven to be zero immediately and this is this is fast so what we expect is that minus A1 counters the A2 and A3 and they cancel each other in this specific regime that we have so inspired by what physicists like Lee and Sompolinsky, but also Rotondo at all did in their paper we tried at first to integrate directly the weights of the networks which in this more rigorous settings amounts to performance integration by parts with respect to these Gaussian variables but there is a problem because these Gaussian weights here are inside phi so we cannot use Gaussian integration by parts right away so we focus a little bit on this guy here and we employ the oldest trick in mathematics I guess so you subtract what you would like to have there and then you add it back and you control the difference between this red and blue guys and then here in this first line here that you would like to go to zero because this is linear and you can treat and it magically cancels the other two contributions A2 and A3 so you integrate by parts the A star here in the first line and what you get is this more or less ugly guy okay alright so here the first part goes roughly to zero as the order that you see in the round parenthesis thanks to the concentration of the free entropy this you can show then you have this sum of n squared terms of these differences here the quantity that you can prove to be of order one it's fine but the problem is this difference here so it's n squared variables and the only thing that makes you lucky is that this guy in the square brackets concentrates around zero okay so it is helping to go to zero but unfortunately I mean if you study the order of this guy you find this small o here of n times square root and this n over p which n has to be simplified with the one of n here that gives yields exactly n over p that it's our bottleneck it's actually there and the reason why I personally think that this is there is something going on here it might be fundamental is that up to this slide everything I've shown you is exact I have taken no limit yes you control it as the norm of the norm of the matrix of the difference norm of the matrix you have red minus blue mu and nu and do you control it for fixed mu and nu the size of red minus blue is small no I take mu and nu I need to take them together look at the norm of the matrix for example the operator norm of the matrix that is red minus blue I think we sorry sorry can you say that again say red minus blue is small but then there is issue of n over p yes do you look at the difference of red minus blue entry wise or do you look at the difference of red minus blue as a matrix and then the operator norm of that matrix might be small no we don't look at the operator norm but the fact is that here I've compressed a lot there is some subtleties here because this red minus blue part is not independent of the first part that is here and also this expectation there is some stuff going around there and if the first part were not there I could prove the concentration I would like it to be getting rid of this n over p and actually I can also estimate further orders the problem is that the further orders are correlated with the first part and I cannot get rid of these correlations and that's my bottleneck that's why we cannot improve that and typically in empirical risk minimization you don't have this part because you're interested in the infimum of the Hamiltonian so you don't have this expectation log z in the first part that's why I didn't manage to replicate your results and that's it I'm sorry sorry everybody take your time to conclude so what about the n over p of order one this is an open question I would like to discuss with people if they have an idea how to improve this because it might be either fundamental or very technically hard we're still not sure about that 100% at least so this is another idea of a project I'd like to pursue which is what happens if I have partial information on the W star so I introduced this additional channel here and when sigma which is the signal to noise ratio in this channel goes to infinity I know the way it's exactly and I get back to the random feature model for instance sigma is zero I know nothing and this is what I've studied so far so this could interpolate between the two and maybe some interesting phenomena arise from that then why not more than two layers this is also another rightful question the fact is that at the moment we only miss the concentration of the free entropy which needs some combinatorial argument that we hope to be able to import from other papers by Hongbin Chen and Jamingsia that did that for the multi-layer GLM in which the inference problem is different from our learning problem but maybe we can import some of their machinery and instead the other part of the simplification of these terms I've shown you can be carried on to a multi-layer setting if one integrates carefully the weights of the network one by one with an inductive argument at least I'm confident I've already started sketching something and last question is what happens if we add structure to the data so this is the simplest thing you can imagine X to be still Gaussian or drawn from Gaussian mixtures why not structure types of data yeah I thank you for the patience also alright thank you very much you already managed quite a few questions but I think we have time for maybe a few more before we go to lunch thanks for the nice talk and I'd like to get some intuition for this result so so this generalization error or the mutual information I mean is this an upper bound I mean because what happens if I understand right in this limit when P goes to infinity that your model should become more expressive so essentially the mutual information should go up but at the same time your data becomes more structure less in the sense that yeah so and so I mean if I take can you take this as an upper bound of or a lower bound of what you would get with finite P this is an excellent question okay I don't know the correct answer is that I don't know okay so if I increase P I would need more data to have more information about all those weights right I don't know I have to think about it honestly no I can't say just of a very simple model yes but I have to correct the very simple model there is a okay let's say that the properties of the more complicated model is are say collapse into these coefficients rho and epsilon that I carry on that said I'm not sure there is a monotonicity between the two I can't say here right now on the spot because I'm not very familiar with what happens in commuting machines that is for finite P I should first have a look at that for sure okay we have time for maybe one last question I think maybe more of a comment than a question I guess I didn't think so much about it but your question makes a lot of sense but it's not clear that when P gets small let's say P is 2 you should have equivalence because when P is 2 you have this commuting machine which is of a very different nature of a linear model which is the perceptron so I don't think you I don't expect this kind of results to hold for P small I don't expect this kind of two-layer network to recover a commuting machine with more than with at least two hidden units because you can suddenly model non-linear rules which are much more complex than the linear JLM so I don't think it's the same thing at all yes yes but we should comment yes comment on this you are right alright that was a very interesting session let's thank Francesco again for his talk


Welcome to Access a Trader, the number one community for those who are committed to taking control of their trading in order to achieve success, profitability, and longevity. Thank you for joining us. Here's Dan Shapiro to help you find your edge, master your process, and own your future. Hey guys, good evening everybody and welcome to the Wednesday edition of the AccessaTrader.com weekend update show. So the market ended very, very well yesterday, right? We talked about, we reclaimed supply two days ago, had a nice move yesterday in the indexes. Obviously, again, we were very bullish going into today's session and today was the first day of the third quarter, right? First day of the third quarter and 20% move like we discussed last night on the video in the S&P and today basically the market participants woke up, beta names woke up and they said, listen, if you thought second quarter was good, hold my beer. We'll get to that, we'll get to individual pivots in a second. So, I mean, there's really what's there left to say about this tape? Again, you see all these incredible market pundits who've been around for 600 years and they're using all these great words to try to convince themselves, not even to try to convince the public, not even to try to convince the investors, but they're trying to convince themselves.  It doesn't make a difference what financial news network you get your news and again, it doesn't need to be a CNBC or Bloomberg or Fox. There's 30,000 sites, 300,000 sites and they're going to give you every single reason. Why you are dumb? Why you shouldn't be in the market? Why buying at these levels two months ago was wrong? Why chasing momentum is bad? Which it is, again, unless it confirms technically. So, one of the biggest issues and this kind of trickles down to what a lot of traders are doing that really missed this whole ride. Again, you can make an argument that, you know, I forgot who asked me today in the webinar, is the market, and this is pre-market, is the market ahead of itself? The market's been ahead of itself for two and a half years. Okay, so this is, we're just seeing kind of like literally the cherry on top and, you know, when you want to trade for a living, again, it could be a passive way of trading, long-term investment, whatever the case may be. Again, the first thing you have to convince yourself is that your money is safe. Once you convince your money is safe, then you can obviously go into whatever process you're trading, whatever stocks you're trading, you could go in there with a clear head.  For any market, when you're going in eyes wide shut, this is where you always catch, you know, the rich hand, right? You get the elbow into the neck. This is where you get hurt. But this is the type of tape.  That's your clue. That's your market. That's your green light.  People are chasing, you know, people are overextending themselves. It's your job to figure out how to exploit them.  So the idea that you're sitting on the sidelines making excuses to make you feel better, okay, the sarcasm is cool, right?  But again, without those idiots, there's nobody there to take advantage. Again, why are you not taking advantage, right? If you're, you know, if you're so smart, why aren't you taking advantage? And I think that's where a lot of the fund managers really dropped the ball, right? And you heard this two, three months ago that after the initial debt-cad bounce of this horrific COVID virus, and again, I was one of them the first two, three weeks. Okay, this is a debt-cad bounce. Let's see what happens next. Okay, the debt-cad bounce is not a debt-cad bounce anymore. It's a full-blown rally. Oh my God, it's not a full-blown rally. Now we're testing the high, the all-time highs. Again, at some point, you have to turn around and say to yourself, look, I have to take, you know, I have to take control back, right? I have to stop making the excuses. I have to, you know, I have to look at the market for what it is, not what I think it is and what I think I need it to be. And again, this is where, you know, self-accountability takes place, right? Everybody knows this market, so crazy, right? Everybody knows this market's irrational. Everybody understands that the economic landscape as we see it right now has no way near the reality of where the stock market is. We get all that, right? But isn't our job to kind of, well, figure out how to get around it, whether it's long, short, and indifferent, right? Isn't our job as a trader to take the landscape, to take the ingredients, what we have in front of us and make ourselves a delicious dish, right?  Not painting ourselves in the corner, not putting ourselves in a situation that we are so biased in one direction that we refuse to see the light, right? Isn't that what it's all about? And the one thing that we're going to look back at this stock market, and again, whether it ends a week from now, tomorrow, whatever it is, right? You know, whatever it is, the moral of the story is this will go down in one of the most aggressive stock markets. And again, I don't care how you trade, you don't need to trade beta to figure this out. You have $2 stocks going to $30, okay? This will go down as one of the most aggressive stock markets you could remember, all of us can remember, going back to the internet craze, going through the generational bottom of November of 2009 and fast-forward going on here. So I'm hoping everybody's doing incredibly well, more important, staying healthy again. You're seeing crazy numbers starting to pop. The governor of California today announced they're closing things back down. Restaurants, bars, movie theaters, you know, so the cases are real, right? The cases are real and we're getting a lot of states that are really, really coming out with some aggressive numbers. The most ironic part of all this is states like, for example, like New York, New Jersey, we're kind of cool, man. We're kind of cool. Surprisingly, considering the stagnant numbers that we had, now California has surpassed New Jersey in cases, which is amazing. Now we're in New Jersey, we're getting like 300 to 400 cases a day statewide. We were getting, what, a month ago, $3,000, $4,000 a day? So ironically, the bigger populated areas like Newt Manhattan, right, and Manhattan, the suburbs of New Jersey, we're kind of, you know, we're kind of cool. Maybe people are taking a lot more fiscal responsibility with their health and not just kind of going out and living their life with blinders on. So that's kind of cool. So I hope everybody's staying healthy. I hope everybody is putting themselves in the position, whether it's mentally, financially, relationship-wise, in a good, good place. Because again, at the end of the day, our mentality, our mindset, our happiness, again, is nothing to do with trading, right? Trading is just kind of a thing we do, right? Some of us have been doing it for a long time. Some of us have just started, but again, life and happiness is not about trading. So please find yourself in a good area, mentally, have a good life, right? And just be happy, learn how to smile. So let's talk about the tape. Obviously, again, the market's nuts. If you thought yesterday's moves were crazy in stocks like Tesla, again, as the market says, hold my beer. Technically, this is kind of where we are. The Dow is down 77 points. Nobody cares. I had no idea. I never have any idea where the Dow is. I really do. I just kind of look up and say, oh, the Dow's up. The Dow's down. It doesn't make a difference. The Dow is down 77 points. The Dow's that composite. Two days in a row now of triple digit moves. And we'll talk about the pivots in a second. When you see some of these moves, it's phenomenal. It's just absolutely phenomenal. Today was officially, and again, I didn't get the memo, but officially, you know, officially when you have a move of like 6, 7, 8 points, apparently a stock will go up another 60, right? That's from what I understand. So today was kind of like national no-down tick day in shares of Tesla, Roku, Amazon, Netflix, Facebook, just crazy, crazy moves. Really good day. Really good day. Going into tomorrow. Again, we got this move. I don't want to call this kind of a similarity to what we saw last week, but we did get a move into the upper Bollinger Band. Again, this is not quite an inverted hammer because you can see the body of this candle is really thick and it's closer to the top than the bottom. So I mean, I could see a slight pullback and that's why I have a couple of names that I do like that I'm watching for tomorrow. You know, names like Boeing, for example, that just didn't rally. Just didn't rally. They're very, very close to losing the five days. So just in case we do get a pullback, I like Boeing to see if it confirms the channels. BYND, like for tomorrow, I kind of like it as well. It gave back its whole gains, like really its whole gains. So I'm going to start watching this whole bottom channel right here if it starts really giving everything back up. But obviously, you know, this is a very, very strong market. Stocks that are taking their 52-week highs are taking their two 52-week highs. We'll show you in a second in shares of Amazon when bananas today, Netflix, obviously, Tesla, Facebook and everything else.  We are closed on Friday. Okay, we are closed on Friday 4th of July, right? 4th of July tomorrow. I think we're going to get some really good value. I think it's about lunchtime or so, maybe until about one o'clock or so, which is basically our majority of our day anyway. And I think after, you know, after lunch, I think the volume is going to really, really die down and the range is going to start really contracting because again, a lot of people are, you know, going away for the long 4th of July weekend. So if you are trading tomorrow, I will definitely be trading tomorrow for all you guys in the live webinar and the Twitter feed. So normal day, normal day tomorrow. And for all you guys who are taking the rest off, again, unwind, decompress, get your mind right, have some good food, have some laughs, have a drink. Again, smile, smile, smile. That's what all is life about. What's the point of anything that we do, okay? If you don't have a smile on your face. So I'm wishing everybody a phenomenal, phenomenal 4-day rest. So let's get into the pivots today. Just nuts, just absolutely nuts. Right, so here was the game plan. If you notice, everything was to the long side, like literally everything to the long side. We talked about Tesla yesterday. I said any week open, okay? And this has been the play now for years. When you see a stock put up a monster, monster candle the previous day, okay? Like a Tesla did yesterday. And the stock opens up a week. You need to be. We talked about this last time in the video. You have to buy this thing on Rising Support. You absolutely have to. Tesla opened up down, you know, down like, what, down four or five, maybe $7, whatever the case may be. And I wrote this out. I go, look, a couple of different scenarios today to consider. It never got down, obviously, to the 1057, 1058 area. This would have been, this would have been like phenomenal. It never got there. But again, the second scenario, red to green, obviously, no. This is not a pivot, just momentum. But here is the pivot, right? Here is the pivot right here. 1089, 1090 needs to build.  We talked about the option flow yesterday. 1120. That's so child's play, 1120, right? So here was the 1090, right? Obviously when red to green, here was the 1090. Here was the 1120, right? Da da da da. Nobody cares, right?   Tesla went nuts, went absolutely nuts. Amazon, this was definitely the move of the day. 2771 needs to build. I said, look, there's a shot this thing gets to the 52-week highs. Here is Amazon, right? Here is Amazon 2771. This is this whole channel here, 2771. Yada, yada, yada.  It's amazing. Just when I say the moves today were exaggerated. I think that's an understatement. Amazon went nuts, Square got upgraded today. Again, nuts was definitely the theme. A couple of levels to watch this morning. 107 and 10785. Again, if you notice the 52-week highs today, they were going eight. They were just jumping over each other to buy shares. We're talking about cases of FOMO. 10710785 is the June 26 highs. Square went nuts, right? Here is Square. Here is the 107. Got upgraded. Took out the 10785. Square went to 117. Again, I have no idea what's going on today. FSLY, not a big move. 87 needs to build. Only went up. Again, I kind of knew it was a little bit overextended. I said, look, the stock is a little bit extended. Atat extends. Only used a 50-cent risk. Again, more for experienced traders. Stock only went up like 70 cents. Facebook went bananas today. Obviously, I think my dyslexia definitely kicked in here. It was 227.60, 227.80, not 226.80. Needs to build for cashflow spike. And that's what I thought it was going to be. I didn't think this was going to be this majestic move. I thought it was going to go right into this area here. 229.5, 230. And Facebook just went nuts. Again, just like everything else. Just absolutely went nuts as well. Apple, very, very weird today. So it had its initial cashflow move. 336 needs to build. So Apple was kind of weird today. So Apple takes out this whole 336 area and trades into supply. Which again, we kind of understood that. So it put up like a dollar 40 move. And it just never confirmed back to the upside. It actually wound up red on the day, which is very, very odd considering the nasa composite was up 122 points. Very, very interesting there as well. Nvidia really didn't do a lot. Nvidia put up a dollar move. 382 to 383 and then kind of did nothing for the rest of the day. Kind of like Apple. Spotify, I still like this thing on the short side. Chewy, I stopped watching Chewy. Let me see if Chewy did anything. I just wasn't watching Chewy, so I apologize. 4420. 4420. No, I never got down to the 4420s. Okay. So that's that there. Big spike there.  Squares going nuts. Apple was actually the most ironic part. The first one to explode, which was very, which was amazing that it stopped. Amazon just sweet morning and then, and then, and then, and then, and then, you know, again, all these things are going nuts. Towards the end of the day, there was an 1120, 1119 second entry that actually flushed down to like 1107. But Netflix, and here's the afternoon move on Netflix. Netflix, a buyer comes in and this is, once you start seeing these multimillion dollar bets, it's like a, it's like an absolute flight, you know, flight to the insanity. A buyer comes in for the August 500 calls to put down two and a half million dollars of premium. 5474 is 52 week highs and look what Netflix did. I mean, just look at this move here. It takes out the 474 and puts up another $14 on top of that. So yeah, you get the point. I mean, this is, again, this is why, you know, this is why we trade beta.  And oh, oh by the way, by the way, by the way, by the way. So yesterday I put on my first swing probably in a long, long time. So I bought this net net net yesterday at 34, excuse me, can I say 34? 3594, it just sat there all day. It went, actually went green today and then went red today. It held the previous days low, which is very, very important. This is how you hold onto a swing and make sure it never takes out the previous days low. And then it just caught fire. So I'm up a buck in this thing right now. I still am very, very intrigued by all those $40 call buyers yesterday. We saw some $40 call buyers today as well, not as aggressive as yesterday, but I'm still very, very intrigued. I would love to see this thing take out this 38 and a half, 38 and a half level, which was the 52 week highs and maybe get a push to 40. You know, we'll see. You know, we'll see what happens here. You know, close just under $37. Just close under the $37. I'm up a buck in the trade there. So that's it. That's it, ladies and gentlemen, support you guys who are trading tomorrow. I will see you normal bad time, normal bad channel. Guys have a great rest. Have an awesome 4th of July. I love you all and God's willing. I'll see you all tomorrow. Take care guys. Congratulations for putting in the time to take control of your trading. You're one step closer to owning your future and achieving the success you desire. Want daily trade ideas directly from Dan? Straight off his personal watch list? Unlock our free PS60 Vault where you'll get nightly updates on pivot opportunities we're watching for the next day's session. Click the link in the description to get started today.


We have discussed ideal systems at length during our discussion on statistical thermodynamics. Most of the expressions which we have derived belong to ideal system ideal gases, but we know that even if we first talk about the gases all the gases are not perfect gases or ideal gases. We may say that the ideality or perfection is approached under certain conditions. For example, when we talk about the gases when the pressure is very very very low the gases may start behaving ideally and when the system is not ideal then there have to be some different sets of treatment for that. So, in today's lecture we will talk about equations of state and in today's lecture we will restrict our discussion to gases. We have been discussing many equations many relations in terms of canonical partition function and the canonical partition function which we designated as capital Q is a function of n number of particles volume and temperature. So, this canonical partition function which is a function of volume temperature n number of molecules was connected to various thermodynamic parameters and one such connection was between pressure and canonical partition function. If you remember we had derived this equation pressure is equal to k t times del log Q by del v at constant temperature. We can call this is an equation of state because what is an equation of state? When we talk about ideal gases or perfect gases the ideal gas law ideal gas equation or perfect gas law that is p v is equal to n r t this is an equation of state for a perfect gas. So, we can call an equation of state as an equation which connects p v and t the simplest case is p v is equal to n r t, but remember that this is only for an ideal gas.  This is a purely general equation we did not specify this that whether it can be used only for ideal gases or it can be used only for non ideal system we did not specify that, but let us first apply it to monatomic perfect gas monatomic perfect gas. Let us do that and see what happens what we have p is equal to k t del log Q del v at constant temperature. Remember that if we are talking about monatomic system we are only discussing perfect monatomic gas. So, Q canonical partition function in terms of molecular partition function is going to be Q raise to the power n over n factorial because the molecules are not distinguishable. Therefore, we are going to use the connection of canonical partition function with molecular partition function which is used for indistinguishable molecules. If Q is Q raise to the power n by n factorial then log Q is equal to log Q raise to the power n minus log n factorial. Can I write this as n log Q minus n log n minus n sterling approximation fine. So, that means my log Q is n log Q minus n log n plus n and let me write this on the next page. Log Q is equal to we have n log Q n log Q minus n log n plus n this is what we derived that is log Q is equal to n log Q minus n log n plus n, but we are interested in pressure. Pressure is equal to k T del log Q del V at constant temperature and since we have already an expression for log Q we can take derivative of log Q with a respective volume. So, when you take the derivative of this with a respective volume this is the only term which may depend on volume n log n and this n these are constant. So, derivative of this these two quantities with a respective volume is going to be 0 ok. That means the pressure is equal to k T then we need to take derivative of this which is n by Q into del Q del V at constant temperature. Since we are talking about only monatomic perfect gas. So, Q is equal to Q T is equal to V upon lambda Q where lambda is equal to H over 2 pi m k T ok. So, Q T is equal to then 2 pi m k T raised to the power 3 by 2 into volume by H Q V upon lambda Q. So, let us now put pressure is equal to k T n upon Q upon Q I am going to write Q T which is 2 pi m k T raised to the power 3 by 2 into V and here I have H Q. This is n by Q then multiplied by del Q by del V at constant temperature I need to take the derivative of this with a respective volume at constant temperature constant temperature means entire this factor becomes constant. So, let us write down this becomes 2 pi m k T over H Q then del V by del V is 1 of course, this is raised to the power 3 by 2. This and this cancel H Q and H Q cancel and then if I rearrange I get P V is equal to n k T is n times n a that is number of molecule is number of moles times the abogadro constant into k into T k times n a is R is P V is equal to n R T this gives me P V is equal to n R T because k times n a is equal to R. Now, I can consume n I can take n on the other side and V by n I can write molar volume. So, I have P V m is equal to R T this is the equation of state. We were saying that this is equation of state by using that first restricting to ideal gases we have shown P V is equal to R T or P V is equal to n R T. This is an equation which we have been using for a long time and we called it as equation of state, but this is applicable to ideal system ideal gases. The gases may or may not be ideal first of all we should remember what is the definition of an ideal gas. How do you differentiate between real gases or an ideal gas right sometimes real gas you call as non ideal gas what is the difference between them. When I say ideal gas that means no inter molecular interactions between them irrespective of the distance. When I say non ideal gas or deviation from ideality that means the attractive or repulsive interactions come into play when you vary their distance from very far to very close. And if you remember at some point we define pi t is equal to del u del v at constant temperature. This is equal to 0 for ideal gases and this is not equal to 0. I repeat this is not equal to 0 for non ideal gases ok. So, if we were to address the difference between real gases and ideal gases there can be several points of differences. For example, one we talked about pi t pi t is equal to 0 for ideal gas pi t is not equal to 0 for non ideal gas. Why because sometimes the gas molecules when you mix two gases and if these two gases are non ideal then they will interact with each other. So, interaction energy can also contribute to non-ideality. There is existence of non zero joule Thomson coefficient there can be difference in transport properties. I repeat there is difference in transport properties and there can be existence of non zero joule Thomson coefficient. So, therefore, how to address the deviation from ideality in gases. Looking back at what you study in class 11 or class 12, Wendervals equation of state in which A and B were introduced which reflected upon the attractive and repulsive parts of the interactions. Here we will talk about the virial equation of state and we will talk about virial coefficients P v m upon R t which you remember was also called as compression factor, but here we are not going to talk in terms of compression factor. Here we are going to talk in terms of virial coefficients.  If b c all these numbers or all these coefficients are 0 then you simply have P v m is equal to R t then it becomes an ideal gas. In this form of the equation b is called second virial coefficient c is called third virial coefficient and so on. Obviously, these virial coefficients are going to depend upon temperature why because when the temperature is increased the intermolecular interactions between the gas molecules will also be affected. Therefore, these virial coefficients b c etcetera these virial coefficients vary with temperature. So, our purpose is to somehow connect these virial coefficients with canonical partition function then we can move towards bringing in non non ideality into our discussion. Let us do that, but before doing that let us recall some of our earlier learnings. One of the earlier learning was that the total kinetic energy of a gas of molecules is some of the kinetic energies of individual molecules even when interactions are present. When we only talk about kinetic energy I am not talking about potential the total kinetic energy of a gas of molecules is some of the kinetic energies of individual molecules even when the interactions are present. That means, now in a real gas the canonical partition function must factorize into a part which is arising from kinetic energy that is going to be same as that for the perfect gas and another factor called configurational integral. That is what is commented here thus in a real gas the canonical partition function factorizes into a part arising from the kinetic energy which is the same as that of perfect gas and a factor called configurational integral this we will call as z. Obviously this z which is configurational integral it is going to depend upon intermolecular potentials. Now this canonical partition function is represented as z by lambda raised to the power 3 n. For perfect gas you have already studied that this is equal to v raised to the power n by n factorial where v is the volume of the container and this is according to this definition because q is then partition function for a collection of non-interacting particle. That means, if z is equal to v raised to the power n by n factorial we are talking about perfect gas we are talking about non-interacting particles alright. Let us move ahead from the ideal gases now let us move towards a real gas. When we talk about real gas we are talking about intermolecular interactions. That means, we have to now in addition to the kinetic energy we have to bring in the potential energy and let us represent potential energy as v n.  What are these d r 1 d r 2 d r this integration of each one is equal to volume. We are not getting into the derivation of these you have learnt at some point, but we need to know in order to evaluate this we need to know the potential. Though we are not getting into the derivation of this, but we can go reverse that we can show that this converts into the configurational integral for a perfect gas when you ignore the potential energy. And when do you ignore the potential energy when the interaction between all the particles is switched off to 0 perfect gas. If you put v n equal to 0 then exponential 0 is 1 in that case z is equal to 1 by n factorial d r 1 d r 2 etcetera and since each one is volume then this is v raised to the power n by n factorial and this is what the configurational integral was defined for an ideal gas. Distinguish between this v n and this v n this is the potential energy and this is the volume of the container ok. Now, let us consider only pairs of particles you can consider two particle interaction you can consider 3 4 5 you can keep on increasing and that will bring in more complexity, but for a simple example let us consider the interaction between only pairs of particles.  So, that means, if we know the potential somehow if we know the potential then we can have b this f is called mere f function remember this is called mere f function. Obviously, when you put v 2 equal to 0 then exponential 0 becomes 1 1 minus 1 becomes 0 and if f becomes 0 that means, then in that case b also becomes 0 right it says it goes to 0 when 2 particles are so far apart that v 2 equal to 0. So, if you are talking about the particles which are very far apart that there is no interaction between them then the interaction potential energy is 0 v 2 is equal to 0 if v 2 is equal to 0 then your virial coefficient is also equal to 0. So, this explains that when the particles are very far apart in that case this f f value turns out to be 0 and then you can see what happens to the value of b. Now, consider that the interaction intermolecular interaction depends only on the separation of the particles, but not on their relative orientation. Please listen carefully when the intermolecular interaction depends only on the separation of the particles separation let us represent by r, but not on their relative orientation. We can take an example for example in closed shell atoms and tetrahedral or octahedral molecules. In that case this equation b is equal to minus n A over 2 v into f d r 1 d r 2 where f is this is the same equation that we discussed earlier. This b transforms to minus 2 pi n A 0 to infinity f r square d r and this can be evaluated numerically provided we have this mere f function which is dependent upon the potential. You can have depending upon the potential energy you can calculate you can evaluate the second virial coefficient numerically alright. Now, what we have is the expression for b which now we have derived for when the potential energy only depends upon their separation. So, consider hard sphere potential this hard sphere potential is infinite when the separation of the two molecule r is less than a certain value. Remember the potential energy curve it will go take a dip and then again it will increase. So, we are using the same principle that when the separation of the two molecules which is represented by r is less than or equal to certain value right then the potential becomes infinite. For example, you see here if the potential energy for a certain values r less than sigma it becomes infinite. Then you put v 2 equal to infinite over here exponential minus infinite 0 0 minus 1 is minus 1 in that case f is equal to minus 1. So, if f is equal to minus 1 then b becomes you can connect this over here b becomes 2 pi n A r square d r from 0 to sigma sigma is that minimum value which turns out to be 2 by 3 pi n A times sigma cube. This is the case when the separation of the two molecules r is less than or equal to certain value. On the contrary if r is higher than sigma let us say the molecule they are far apart then you can take potential energy to be 0 in that case f also turns out to be 0. So, what we have derived is that the second virial coefficient b is equal to 2 pi n A integration 0 to sigma r square d r which takes up this form 2 by 3 pi n A times sigma cube. You remember that Wendervals equation of state p is equal to r t over v m minus b minus A over v m square where A took care of the attractive parts of the interaction and B took care of the repulsive part of the interaction. So, here also the hard sphere repulsive part of the intermolecular potential b that contributes to b is exactly in the same way, but the these are some of the extreme cases which we derived. The question is can we find a potential the actual potential? Here we need to have the value of v 2 to evaluate the exact b state c or any other virial coefficient. So, can a potential be found which when the virial coefficients are evaluated provides the full Wendervals equation of state? The answer is yes such a potential can be found. We have talked about second virial coefficient similarly the extension of the discussion can be made to other virial coefficients and you will also require to have knowledge of how to find a potential which actually represents those intermolecular interactions, but that is beyond the scope of the current discussion. What I have discussed in this lecture is that the gases which do not follow ideality which are not ideal which are not perfect will show non-ideal behavior and will undergo intermolecular interactions. How to incorporate those intermolecular interactions? How it is reflected into virial coefficients? That is what we discussed in this lecture. I hope it was useful and it will form a basis for your higher revenue. Thank you very much.