Hello, and welcome to the session of the Denver Democracy Center. I'm Jonas Prowler-Plesner, Executive Director at the Alliance of Democracies Foundation. I'm also really delighted to be joined by Digital Minister Audrey Tang from Taiwan for this keynote session on protecting democracy from disinformation, the case of Taiwan. Welcome, minister. Hi. Good luck, Otaim, everyone. Great to see you again. I went to Taiwan last year for the presidential election in January and met with you there and witnessed firsthand some of the knowledge of tactics for combating disinformation and also foreign-driven election interference from China. And as part of your approach, minister, with radical transparency, that conversation at that time was recorded and is still sort of publicly fully accessible. And I know all of your official meetings and texts and everything with people, they are all sort of publicly available, again, as part of that sort of openness to combat disinformation. So it's a pleasure to have you here again virtually. And together, we'll focus on Taiwan's effort to engage both citizen, business, and civil society and the government's fight against disinformation. And we'll also talk about whether other democracies can learn from Taiwan. So my first question is really about what Taiwan has done to curb disinformation both in elections and during the pandemic. Yeah. The Taiwan model of countering the pandemic, easy to remember, it's called countering the pandemic with no lockdown and countering the infodemic with no takedown. And we do so via humor. The playbook is called Humor Over Rumor. That is to say, whenever there is any trending myth or disinformation, we make sure that within a couple hours, for example, as you can see here, there's a very cute dog here that explains physical distancing in terms of indoors, three Shiba Inus away from one another, two Shiba Inus away from one other. If it's outdoors, it's very difficult to unsee that clarification. And because it's so cute and harmless, people remix and share it all the time. As soon as the clarifications and the science reaches more people in a way that's humorous, the outrage that disinformation travels dissipates. People stop sharing conspiracy theories and so on, because there's something equally interesting and even more scientifically correct for people to share. But the real challenge is how to detect the trending misinformation, disinformation before it reaches majority of population. This is not unlike detecting a virus. And so we do so by what we call a people-public-private partnership, having the social sector forming those early advanced warning systems, including virus like companies like Trend Micro, including, for example, the co-facts project from the Gov Zero collective. And many people who just report any misinformation, disinformation, but not to the government, but rather to the social sector fact-checkers and journalists. That's really interesting, because you, of course, come from a background, having been a software engineer, hacker of sorts, even would put on your bio and from the sort of civic tech community of Taiwan. Do you think that's unique to Taiwan that you have this very engaged citizenry and very engaged software community around this or something the rest of us can also replicate? I think it's quite pervasive in pretty much all jurisdictions. There are, for example, chapters of the Code for All Network around the globe. So there's no shortage of civic technologists. But I think in Taiwan our unique configuration is such that we're a very young democracy. Our first presidential election was in 1996, which is already after the World Web gets popular. So in our month, democracy is a set of technologies. That is to say, instead of having hundreds of years of traditions to us, democracy is not just uploading three bits per person every four years, which is all voting, by the way. But from the very beginning of our democratization, we think of day-to-day democracy through deliberative meetings, participatory budgeting, sandboxes, presidential hackers on e-petition, you name it. All these increase the bandwidth between the social sector and the public sector when it comes to collective decision-making. And so that in turn informed our social sector to form like the Taiwan Fact-Check Center and many other response systems that just as we countered the spun decades ago by setting up voluntary report as spun buttons in all the email reading agents and having a global spun house network to send out the signals that can flag something as spun before it hits our inbox that goes to junk mail folder. Similarly, an immune system like this operated by the voluntary sector is configured in Taiwan much more easily because people think of democracy as something that we can contribute not just for the professional career public servants. Any words? People actually do this? Yes, the K-12 curriculum, which I'm one of the designers specifically emphasized instead of media literacy like we did in last century. We now say media competence. Literacy is when you are readers and viewers of media. Competence is when you are producers of media. And because Taiwan has brought in as a human right, so any primary school student, middle school student have access to broadband at just 16 US dollars per month for unlimited data. Otherwise, it's my fault. So they can actually Fact-Check very easily. And they did that. For example, in last year's presidential election, there's plenty of student that Fact-Check all the three presidential candidates, public debate and platform on the kind of cross-check platform so that they can contribute to the newsroom in conjunction with the social media and the professional mainstream media. They just look at everything that the presidential candidate has uttered. And once they have this experience, it's actually very difficult for them to go back to conspiracy theories because they have engaged in finding out the facts together. That's very interesting. Of course, in Europe and in the US at the moment, we're discussing a lot on this information of what the social media platforms have a responsibility that they should take down more than something and should label and so on. But your approach, do you think that you don't even need that? Or would there still be a need for better regulation of social media companies on this information? So instead of regulations here, we have the norms set up by the social sector. For example, the Gamzero movement a few years ago started a campaign to get the National Auditing Office to publish the campaign donation expenditure as open data, raw data for all the investigative journalists to look at. Now once they get what they wanted in 2018, and the National Auditing Office did publish these, then we discovered that none of the social media expenditure is filed as campaign finance. And that's obviously a loophole because in Taiwanese law only domestic people can contribute to campaign donation, but foreign interference kind of find a shortcut through the social media advertisements. So we talked to Facebook and other social media companies saying, look, there's already a very strong social sector norm and they convinced the public sector to comply to do the same. And now it's your turn. If you do not publish your advertisement library in real time in the same sort of raw data and open data as our National Auditing Office did, well, we don't have any regulations against you, but you may face social sanction. And in Taiwan, social sanction is really strong. So in 2019, Facebook worked with us so that when became, I think the first jurisdiction that they published in real time, the advertisement library pertains to political or social sponsorship advertisement. But the other thing that you just mentioned, which is the public labeling, we do that too. And we call it notice and public notice.  That's of course, like intentionally wrong information, but the photo is legit is the Reuters photo. So the time fact check sent him after enough people flagged it for review, actually found the alternate caption of the original Reuters photo said nothing about being paid to kill police, which is that there are young people in the protest. But the alternate caption came from Zhuang Zhenfa Wei, Zhang Anjian, the central political and law unit of the Chinese Communist Party. So instead of taking anything down, we just put a public notice and with the collaboration of Facebook and other social media platforms, people who share this alternate caption, they can still share it. But everyone see that the Taiwan fact check center has already shown that the original alternate caption came from the way board of the Zhuang Zhenfa Wei. That's a fascinating example. Minister Tang, we're actually about to come to a close, but like the panel that follows after us is of course going to continue the discussion of disinformation. So if you have just one piece of advice to them, what would be sort of the most important takeaway on disinformation for their discussion? So I tend to think of the infodemic as just like the pandemic that if people wear a mask and wash their hands, then it's of course reduced our value. And this is similar to the notice and public notice that I just described. But to actually fix it, it really requires a contextual understanding in a more holistic way, the policy question. And so through radical transparency, through citizen deliberation, I do believe by listening at scale, that's the vaccine of the mind that can truly guard us for the long run against infirm dynamics. Thank you, Minister Tang. These have been really illuminating examples and a great way for the audience to hear about Taiwan and your personal experience with combating disinformation. And it's also a perfect introduction to the next panel, which I will also discuss the information and can of course draw on some of the experiences which Taiwan has brought forward. Once again, thank you. Thank you, Liflo and Presby. You too.


So it's a pleasure to introduce Guido for this course on probabilistic modeling and vision inference. Guido, it's up to you. Excellent, thank you very much Matteo. It's a great pleasure to be able to teach in this spring college and I start sharing the screen. So I think the format, if you have questions, do put them in the chat. I know it's a little bit difficult sometimes because when I'm sharing the screen I may not see whether there is something in the chat but as Matteo was suggesting, every 10 minutes I'll take a mini break and enable questions in the chat so that that would be hopefully a good way to keep it interactive as much as it can be possible, that of course is not as interactive as if you were here in person. So I'll share my screen, I'll share a whiteboard page and minimize Matteo so that he doesn't cover up most of my whiteboard. Excellent, so probabilistic modeling and vision inference. So what is this all about? Well, this is a spring college in complex systems and you probably are very well aware that most complex systems are not fully reproducible so they exhibit some stochasticity. Now what do we intend by stochasticity? What I mean by this is that if we repeat the same experiment, let's say you get a different result in general. Not deterministic and this might be because of a variety of reasons, it might be because there is extreme sensitivity to initial conditions like you're in a chaotic dynamical system or it can be because there is intrinsic stochasticity, say quantum mechanics but also in systems for example of chemical kinetics in very low molecule numbers. Obviously the strategy, if you don't get the same result every time you do the same experiment is to try to model the variability instead of modeling directly only the outcome and what we'll be doing is to formulate all our models in terms of probability distributions. Now I kind of assume that you have been exposed to a little bit of the foundations of probability calculus. Now there are very different levels of mathematical rigor you can teach this from. I'll take a somewhat practical but reasonably rigorous level and when we talk about probabilistic modeling what we mean is that we're going to consider the outcomes of the experiments, let's say the variable that we eventually measure is going to be a random variable and we're going to model its probability distribution. So the probability distribution is essentially what gives us a weight for each possible outcome. So probability distributions have to be greater than zero for all possible outcomes and these outcomes, this index here might be even you know non-numerable index set if we're thinking about continuous random variables and we must have that the sum of the probability associated to each event over all possible events must be equal to one. So it defines a measure over the set of possible values taken by the random variable X. Okay so this could be written, okay so let me write it just to give you an idea of the type of notational shortcuts that we would employ that I will employ regularly this would be written really as this so the probability that the random variable X takes the value Xi this must be always greater or equal to zero. It's possible to have events that never happen and then I will systematically shorten this p of X equal Xi into p of Xi. Okay the thing that we will be primarily interested in is the situation where we have more than one random variable and the simplest case is when we have two so we have a random variable X and a random variable Y and we will define a joint probability distribution over the pairs of random variable X and Y and by this I mean the probability let's say p of Xi Yi that X is equal to Xi and Y is equal to Yi. This is the joint probability sorry sometimes I have difficulties with my pen so that's why I suddenly stop. Now another question that we may ask when we have more than one random variable is the relevance of one random variable with respect to the other really and so we may want to ask what is the probability what is the probability of Xi given Yi. So if I know that Y has taken a particular value how does this affect my belief over the variable X so this is called the conditional probability. The final ingredient in the basic things we need to know about probabilities is what is the probability of X taking the value Xi regardless of what Y is. This is called the marginal probability and the reason why it's called the marginal probability is very neat because in the old days a lot of these probabilities were just simply tables where you would have the values of the two variables and you would have you know say here are the possible values of X here are the possible values of Y and when we on the last column on the margins of the table you would have p of Y so here would be the value of Y equals one let's say first value regardless of the three values that X take so it's called the marginal probability because it was written at the margins and was given by summing the rows all the columns of the table. So the final fundamental ingredient is the celebrated base here. Now base the reverend base was a student at my other institution so I'm a professor at CISA but I'm also a professor at the University of Edinburgh where base studied theology many many years ago about 300 years ago almost and he observed that a trivial consequence of the fundamental laws of probability allows you to say something really nontrivial. So the first law of probability we've already seen and it's this marginal is it's the normalization property but the second law is a relationship between the marginal and the joint so the probability of X i is equal to the sum over all possible values of Y j or p of X i Y j that's the marginalization property the third fundamental property is that the probability of X i joint with Y i is equal to the probability of X i given Y i times the probability of Y i okay so if I want to know what is the probability of observing simultaneously X i and Y i well I can think first let's observe Y and see Y i what is the probability of this Y i and then given that I've observed Y i what is the probability of X i and now obviously this is purely symmetrical it doesn't matter whether I choose to observe Y i before X i and so we get also that this is p of Y i given X i times p of X i and therefore base theorem simply takes this observation of this equality and tells us that the probability of observing Y i given X i is well it's the probability of X i of X i given Y i times the probability of Y i divided by the probability of X i so this is base theorem so let's take the first mini break so I've done a very very quick run through what I mean by stochastic modeling by random variables the basic concept when you talk about random variables is that you shift from modeling the precise value of the variable and how it depends on other variables to model the distribution of the possible values of the variable of interest and how it connects to other random variables of interest and then we've listed the basic properties of probabilities including base theorem so are there any questions that people want to pose in the chat can I please explain marginal probability again so sure so marginal probability is the probability of observing X i regardless of what Y i is and that is obtained as I have a joint probability law so I have a probability distribution of the joint values of the X from the variable and the Y on the variable and what I need to do is to sum out over all possible values of Y and let's call it Y j actually instead of Y i so I don't want you to be confused it's not the case that X must have the same number of possible outcomes than Y so it could be this could be that I don't know X could take three values and Y could take seven values or something like that or in general they could both be continuous variables on one or one of them could be a continuous variable and the other is discrete variable and so on so this is the definition of the marginal probability probability of X taking a certain value regardless of the values of Y and you obtain that by considering the probability of X being that value and Y all the possible values of Y and summing that is this clear now professor yes what is the difference between marginal probability if I calculate the probability of X i individually and what's the difference between these two quantities no so the marginal probability is the probability of so we're always thinking about the scenario where we have two random variables yeah I don't know temperature and pressure for example yeah and the two things are linked so you want to know what is the probability of a certain temperature regardless of the pressure is yeah but they're interrelation exactly so what you need to do is to say okay you have some probability distribution of the pressures because you know that is your system for example and you sum the probability of the temperature being that value that you're interested in yeah and the pressure being a certain value yeah across all possible values of pressure thank you okay yeah I have a question yes please ask this phase theorem how the how is it different with frequency frequency theorem in probability with frequentist probability yes yes yes yes okay so that was going to be what we were going to talk about after the question session so maybe it's a good time to move yeah so there are two broad schools of probability of statistics really yeah so there is frequentist and frequentist versus basin so this dichotomy is really about what is your interpretation of probability okay so if we're thinking in terms of measure theorem and and so on you know the way I've introduced the concepts of probability are really the probability gives me a measure over a set of possible events and these rules have nothing to do with being Bayesian or being frequentist now the question that you're opposing I think is you can't read what I'm writing is that true of everyone well you can't read because I write in an incomprehensible way I see okay that's that's a more serious problem yeah yes I think the next times I'll go on a blackboard at ICTP so frequentist versus Bayesian and so what I've written it doesn't really matter this is just the title of the section what I've written is so far I've just looked at basically rules of probability yeah defined as a measure of a set of possible events marginalization products rule Bayes theorem there are all obvious true statements now the difference between frequentist and Bayesian is what is our interpretation of what probability means okay so in the frequentist scenario so for frequentist this is not something that I was particularly going to get into for frequentist statisticians the probability of x taking a certain value xi is defined is defined as a limit in the number of infinite number of experiments of the number of times x is equal to xi over the number of experiments number of experiments goes to infinity so in this way you're supposed to have observed the world a lot of times and then you can define what the probability of a certain event is as an approximation of the limit however there are many scenarios where we are interested in situations where this limiting approach is not even conceptually possible and for example you know if I wanted to ask a question you know suppose Italy is going to play football against England next month what is the probability that Italy will win now that's a question that is well defined and very often asked in the common life but it doesn't make any sense in the frequentist statistician world because it's impossible for Italy to have played against England an infinite amount of number of times and of course even if it has played a large number of times it will have been different teams on different conditions in different days so for basic statisticians the probability of x taking a value xi is defined in a somewhat hazy way as the degree of belief in outcome x equals to xi so from the Bayesian perspective you're supposed to think that there is an expert or someone that knows something about the system you're interested in that will be able to quantify these probabilities and then you can operate on them how would you operate on them well let's see we have seen that Bayes theorem tells us that the probability of y taking a certain value given that we've observed x taking a certain value is given by the probability of x given y times p of yi divided by p of xi so how do we interpret these terms okay so the expert has a prior belief about what y should be and also the expert has a belief about what x should be given that y is something so the prior has a model of the how the the expert has a model of how x is connected with y this is called the likelihood right the likelihood the likelihood model so if I have a prior belief and I have a likelihood model then Bayes theorem tells me how to update my belief and this is called in fact the posterior belief posterior because it comes post having observed your uh which in this case is an experiment on x good okay so some people can understand the writing that's cheers me okay so this is the kind of the basics of probabilistic modeling and Bayesian inference so probabilistic modeling means setting up models in terms of relationships between random variables so to do probabilistic modeling you have to start with some beliefs about at least some of the variables priors and you have to start with some model that tells you how likely some observations would be started from your prior belief so you need the prior and the likelihood this is a probabilistic model is a model of the probabilistic relationships between different random variables and Bayesian inference is a procedure that allows you to connect the prior beliefs and the likelihood model in order to update your belief on the unobserved variables so a simple example and we'll see actually explicitly how this works suppose you had you know a variable let's say your variable x and you have a very broad belief about this variable and then you observe a variable y okay which is equal to equal to x plus a little bit of noise okay so if x is equal to three then your y would be centered would be distributed like this here let's say this is three so it would be something that can fall around there so now suppose I observe y equal let's say one two three and so on suppose now observed y equals I don't know five so I have an observation here well then after having observed that my belief over x will no longer be spread all over the number the real numbers but it would be much more concentrated it would be something like this so this would be the prior this would be the likelihood so this is y given x equals three and this would be the posterior okay any questions let's have a wee pause so you know the important thing that I've been telling you now in these last 10 minutes or so is really the key concept of the course okay to do probabilistic modeling you have to formulate all your models in terms of random variables you have to formulate the model in terms of probabilistic relationships between random variables and based on inference is important because it is the procedure that allows you to update your belief on unobserved variables based on observations and that's why it's so important and used in practice now how do we get that chat so I see that there is some question why don't you unmute yourself and ask questions now sir I have a question so no probabilistic modeling is possible without some prior belief yeah that is a good question indeed so it's let's say it is possible to formulate likelihood models yeah but if you want to do based on inference then you have to compute combine the likelihood model with the prior so you could have say absolute certainty that your unobserved variable why is equal to I don't know five say and you could still have a stochastic outcome x but then y would not be a random variable okay so you would only have one random variable if you have multiple random variables then you have to have something that acts like a prior and something that acts like a likelihood in order to build a model is this sufficiently clear I think so professor yes please in the denominator you wrote e of x i is that the marginal probability absolutely and this thing has got a lot of names so this is the marginal and it's also called the evidence so it's the probability of the observations regardless of what the unobserved y variable would be so it's the evidence of the observations on the wrong thank you I think there was one more question maybe yeah did I get it right do you say that it doesn't matter what is my prior belief probability if I have the right likelihood my answer would be something specific no you didn't I don't think I said that and it would be wrong so your prior belief is extremely important as you see to some extent prior and likelihood are exactly on the same footing so if I change my prior my posterior would change for example in in this simple example if my prior now I'll try to draw it dashed so that it's not confused if my prior was that the random variable x lies all to the right then what the posterior so alternative prior let's say then the posterior given this observation would also be shifted to the right okay so this observation was in the previous case quite close to the prior mean so what would happen is that the mass the the mode of your distribution would be shifted a little bit with respect to the observation in this case if the prior is very much in disagreement with the observer with the likelihood then the posterior would be a compromise it would be somewhere in between the prior and the likelihood so the prior is absolutely important professor yes can we plot the marginal probability the graph just like we did the rest of the parameters well so the marginal probability this is a graph of x and so here we're looking at probabilities of x so the prior is probability of x it's a marginal probability if you wish itself but it's a prior the marginal probability would be the probability of y so if I have y equals five regardless of what is x I should plot it here and it would be given by the integral of p of y given x times p of x in dx and so here we would it would probably be something rather broad as well I guess thank you I have a question are we going to see in this course how to choose the prior or or even if we don't if we don't see it is there any way in which to know if the prior would choose is good or is it's wrong okay so that's two questions one is related to this course and and the answer to the first question is really not really so we're not going to look at how you choose the prior now I could open a very very big discussion on how to choose priors yeah I won't but I'll tell you a little bit so there are essentially there are two schools of basins yeah there are the so-called objective basins and subjective basins so the subjective basins insist that the prior is something that an expert should specify and should be informative so if I were doing probabilistic modeling as I do actually for a job say in a in a scientific context for example in applications to biology I would choose the prior by either talking to a biologist or for example if I was looking at I don't know structural data on proteins I might use models of protein folding to obtain a prior distribution if you're an objective basin instead you're looking at situations typically where you don't think there is good knowledge available and there is a whole school amongst basins statisticians on how to choose a prior that biases your posterior results as little as possible so the objective basins say look we don't we only want the data to speak we don't want to impose our biases on the results and so we want to select priors that confer convey as little information as possible and the problem is that as should be clear from base theorem itself there isn't such a universal uninformative prior so depending on what the actual likelihood model is an uninformative prior might be different okay and so a lot of research is for these class of models this is an uninformative prior that was the first part of the question so this is all I'm going to tell you about how to choose your priors in general I'll assume that you have someone that can give you good information or you are an expert yourself maybe you are a condensed matter physicist that writes down priors for product folding the second part of your question instead is are there good ways to decide whether a prior was good now I'm also not going to tell you much in this course although I could tell you a little bit more this falls into the so-called model selection set of questions yeah so there are ways to assess whether one model gives a better fit to the data than another model and there are also ways to do unconditional let's say to assess how good the fit number model is to another model so let's let's do that now so the key comparing models okay so you can always view you in your Bayesian setup so we are looking at the probability of a certain variable yi being equal to y being equal to a certain value given that we observed x being equal to xj so that marginal probability that we were talking about is the key to decide whether a model is a good model okay so this is also called the evidence as well as the marginal likelihood and it says essentially answers the question of what is the probability of observing xi regardless so now if I have two different models so two different either because the prior is different or because the likelihood is different or possibly because you know there are even different sets of random variables this is the objective quantity that allows to tell me allows allows me to say you know how probable is this data given the model so if I now have two models prior probabilities over the models p of m1 p of m2 because you may think that the model is more likely than the more probable than the other a priori then the base factor is the probability of the observations and model the observation and model one divided the probability of the observation and model two which is of course the same as the probability of the observation given that we're working in the framework of model one times the prior pretty from model one divided by the probability of the observation given that we're working in the framework of model two and so if this number is greater than one that means that regardless of all the other variables that have marginalized the data is more supportive of model one than of model two so base factors are used for base and decision making so I saw there was a question flashing up so maybe we can have that question I also saw in the in the in the chat that many of you were asking me about references and I can give you a couple of references at the end of this lesson or maybe I can send them to Matteo and you can circulate them by email but let's have the question is it still there the question hi sir like I also have a question yeah yeah so like you're saying the model models don't determine the priors right the priors are determined by the expert yes and the model is like a likelihood model is the model right or is it something else no say that again I didn't get that like you said p of xi given yi in the last slide I mean yeah yeah it is the likelihood model yes and then so that is what we think is the model yeah yeah and what is p of m1 then p of m1 is how you compute p of xi given i mean m1 is how you compute p of xi given yi right no so m1 so now we are thinking that we have two models here so two models a model means a connection of random variables and how they are related yeah of these random variables x is observable okay so in one model we might have that the temperature we consider temperature and pressure and they have a certain relationship in another model we may I don't know consider temperature pressure and volume for example so it's a it's a richer model but we always only observe temperatures okay so in model one we have a certain set of random variables and a certain set of relationships with the random variables and we observe x in model two we have another set of random variables and another set of relationships but we still observe x so what we do we compute the marginal distribution for x in the framework of model one okay and divide by the marginal distribution of x the observation so this is a number it's not it's not a distribution anymore so because we put in the observation it becomes a number and so this becomes the ratio of two numbers and then maybe we might have someone an expert again that tells us look it's the probability that you also need the volume is a third while the probability that you don't need the volume is two stars so we put those numbers there so there was another question appearing also there was a question in the chat asking whether there are any situations where the prior doesn't matter so let me answer that question very briefly because this is a common situation that we will see so what about when you have many observations yeah so maybe you observe x equals to xi once and then you repeat the experiment let's say this is the first time you do the experiment and you repeat the experiment and you see the second time you get xj this index counts the number of times the the the number of the experiment so if these experiments are independent then effectively in any case you have a joint distribution over the probability of all the x1 to xn if you've done experiments and time and there are no variables so suppose you're always doing the experiment in the same condition so that the variable y remains the same okay if the experiments are independent and identically distributed then this becomes a product of n equals one to big n of p of xn given y times p of y so now this is still my prior and this is my likelihood for all the observations but now as you can see essentially the more observations I do the more likelihood I get so the contribution of the observations to the posterior will eventually dwarf the contribution of the prior so it is true to some extent that the prior becomes irrelevant in the limit for the number of observations becomes infinity but even that is not precise because if the prior says that one certain value of y has zero probability then there is no amount of observations that will shift that prior however it is common to say that in the limit for very large numbers of observation the prior becomes less important so when you have lots and lots of observation the prior becomes less relevant for your posterior computation and the likelihood becomes dominant so I sort of was a question flashing please just unmute yourself because I don't know how to do with zoom to find I mean the chat flashes and then goes away I have no idea how to excuse me professor yes please back then when you were talking about the bias factor I mean the odds ratio I have a question about that could the odds ratio give us any intuition I mean any information about the limits of our variables that we choose in our modeling I mean for example when you were talking about having both variables pressure and temperature we for sure in our modeling we could have some limits on each of these variables could the odds ratio give us any intuition about which I mean what we should choose for the limits yeah so okay so one way maybe the base factor could be one way to do that but another thing that one could do and and we're going to see it later on towards the course you could actually consider the limits of your variables as some additional parameters or some additional random variables yeah and you could if you wanted to be fully based and you could get a posteriori what are the distributions over the lower bound and the upper bound or what many people do and it's convenient computationally to do you could take the evidence and say okay this evidence depends on these additional parameters and so I will find the parameters so the lower possible let's say pressure and the higher possible higher possible pressure I'll treat them as parameters that I optimize so I'll optimize the marginal likelihood with respect to these parameters okay thank you and one more question please do we prefer to have less variables in our modeling when we are using the Bayesian theorem or not it doesn't matter at all in I mean okay so there are several answers to this question so the purely theoretical answer is it does not matter in fact one of the one of the great advantages of Bayesian models is that because you are doing this averaging when you're comparing models you're not really at risk so when you're doing you see when you compute the base factor so you're averaging out all the variables that you're not observing okay in model one and in model two and these averaging essentially automatically accounts because the probabilities are normalized automatically accounts for model complexity so it will not allow you to overfit in some sense now obviously though the the other answer to your question is practical and working with lots and lots of variables might be computationally more complicated so and also it could be let's say that from the point of view of understanding what you're doing might be more complex so to some extent there is always a good case for having simple models thank you no problem sorry yeah please um based based on this base factor if r is greater than one then you are saying model one is preferred over model two or are you not making any such comments I'm not making so yeah yeah so if r is greater than one you would prefer model one compared to model two I mean can you explain I mean I don't understand why you say that oh because you see what does this probability means look it's better to look on the right side and then the left side yeah this is a joint probability but on the right side you have okay given the assumptions of model one so the number of random variables the prior distribution the likelihood what is the probability of the observations on the denominator is given the assumptions of model two what is the probability of the observation so the concept is that the observations should be somewhat typical and so if they are more likely under model one then model one is a better explanation for the observations and that's why if r is greater than one I will prefer model one I will say okay model one is more supported by the data okay okay thanks no problem and there were some more questions flashing were there more questions or was there or was that it I don't know there are some questions I don't just unmute yourself and and speak is the easiest thing so if not let me just write a roadmap for the course so today we've kind of covered the the the basic concepts of probability now what we're going to look at are some explicit cases of probability distributions next time and first of all in one dimension and then we will do probability distributions multivariate so in multiple dimensions how do I find the chat well anyway once we have that professor yes if you stop sharing your screen you can easily see the chat yeah I know that but I still want to write so maybe I can stop sharing and then I can I can see yeah so now I can see the chat yeah so okay Matteo was suggesting the Davy Makai book yeah another good book that I and it's freely available another good book that I recommend is Bayesian Reasoning and Machine Learning by David Barber which is also I'm sorry I didn't want to send it privately which is also freely available online yeah so okay so let's let's go back to the roadmap for the rest of the course so we'll do probability distributions in one dimension and then we'll go on to multivariate and this would be mainly well it would be the Gaussian distribution the multivariate Gaussian distribution which is kind of the fundamental tool then we'll spend some time talking about linear models and so this would be probabilistic PCA which is a linear dimensionality reduction method and then we'll talk about Bayesian linear regression and basis function regression and then finally we move on to Gaussian processes all of these models are in some sense linear which means that you can actually perform Bayesian inference analytically and and write down explicitly what the posterior distributions are if we have more time towards the end I will explain briefly some methods to perform Bayesian inference when the models are intractable and by intractable I mean that you can't analytically compute the posterior so the problem which you know there's all sorts of problems in Bayesian inference one of them which we discussed quite a lot is how do we choose the prior but another good more problem is how do you choose the likelihood the toughest problem computationally is now how do you compute the evidence and without the evidence you don't get normalized probability distributions and you can't compute the posterior so most of the difficulties in Bayesian statistics and machine learning research are compute efficiently these evidence because these evidence involves marginalizing out all the unobserved y variables and if you have lots of variables or if they are continuous variables you may have sums with lots of terms or high-dimensional integrals which are not feasible not even with very strong computers so I think that's it I guess I'm just over the one hour and yeah professor can I ask regarding the material so for instance professor Marsili will be providing say lecture notes and videos beforehand will that be the case with your lectures will the notes and or videos be available some for instance maybe not the videos but the notes will they maybe be available somewhere beforehand um well I can discuss so how does it work I'll talk with Matteo about that because I guess like I could pre-record the the lectures and then we could just have some questions during the lecture hour um if if that worked the the notes I've I've got stuff scribbled on my on you know okay on this piece of paper but on this notebook but I don't have them on maybe it'll be maybe it will be uh say more useful maybe yes more useful for the students to kind of have the notes in advance maybe even just to get acquainted with what we'll be discussing about during the lectures to make them more uh followable but I don't know it's uh it's may not be I don't know yeah so I yeah as someone is saying that pre-recorded lecture won't be interactive so yeah I think I would prefer not to record a lecture but I can try to be specific with the the type of so giving you some concrete pointers to the books beforehand Professor okay well if hi um would you be providing us with a syllabus or of course outline detailing the entirety of probability and where you can discussions yeah yeah I can I can write you a more detailed syllabus but you know it would be like you know a page of bullet points or something like that all right thank you okay so I think um we're getting close to the session so you will have uh time in the forthcoming lectures to ask for question and uh so I think uh again uh Guido for um sorry sir I have like just a small question yeah yeah like uh sir so like the base factor is a function of the outcome which you want to kind of study right no so the the base factor is a function of the observations observation sorry not yeah yeah observation yeah okay yeah so typically in the Bayesian world you have some things that you can observe and some things that you can't observe the idea is to use the observations to learn something mathematical about the distribution of what you can't observe the base factor is yeah it's a function of the observations but the observations are not something that can be changed sorry yeah okay thank you yes any other question so okay yes just one more please uh professor uh when uh when we say the probability of x given y uh x and y are two random variables right so that means these variables can be either independent variables or x could be the x could be dependent to y it means in terms of y is our observable our independent variable yeah and x it depends on how we see the y so yeah when when x is a dependent variable and we don't know the formula and we just we we just can't observe y yet we don't know the formula so how can we find the evidence which is p the probability of x exactly that is why the course is called probabilistic modeling because you have to make models so you may not know the formula but what you generally will do is you will have a model of what x will be conditioned on y so all the models assume that there is dependency if there are independent variables then there's not much that can be said it's just you know the prior if if x is independent of y then there is nothing to be learned about y from observing x thank you okay so i think we can close our session here thank you again rido and see you tomorrow at 2 p.m. for tomorrow's lectures excuse me professor marsiri yes so you said that you upload the recorded lectures and we see them in advance or we just look at the material then yes i mean the idea is that you you look them in


Depending on the platform that you're dropshipping on, certain items are going to sell better than others. This is a given fact. Yeah, sure. The perfect example can be Etsy. So while on Etsy, you might get a ton of sales for custom made t-shirts or a handmade bracelet. Chances are you're going to get practically zero sales if you're selling something like headphones or on Amazon, you're going to sell more headphones than you would let's say a custom t-shirt. Now that's not to say you're not going to get any sales of those particular items on these platforms, but you're going to get a lot more of a particular type of item. I get it now. And because of that, this video is going to focus on the top 10 products that you should be dropshipping on eBay.  Come here, because you have to listen to it and I'm going to tell you very low. eBay is a numbers game. Now, don't tell that to anybody else. This will be our little secret. But it's completely true. When it comes to eBay, it's a numbers game. So the more that you list on eBay, the better. Yes, you can start your own brand or your own branded dropshipping store and focus on particular products. That's totally doable, but it's not necessarily the best way to go about selling on a platform like eBay. That's best done on somewhere like Etsy or on Shopify. Now, the biggest reason behind this is primarily because of how eBay got started and what it was originally known for, which was being a platform where anyone can sell anything. So when eBay got started, everyone was selling whatever they needed to sell from their house. It was basically an online garage sale. Nowadays, of course, the eBay marketplace is bigger than it's ever been. And it's actually one of the biggest marketplaces in the world. Now, while you still can sell anything from, let's say, your house, you can also sell literally anything else from anywhere else. And that's where dropshipping comes in. There's a lot of dropshipping on eBay. Now, because there's a lot of dropshipping on eBay, there could be a lot of competition. So make sure you offer a bunch of different variations to products that are already out. Remember, not one thing fits for everyone. So if there's different variations and if you can be unique with your variations, then try and do so. You need to be able to stand out as much as possible to be able to get recognized from the different customers. Again, not everything is gonna work for everyone. Some people might not like the color scheme on a particular product. So maybe yours can have a different one.  Now, like I mentioned, eBay is a numbers game. So the more that you list on there, the better. So when it comes to the 10 items that I'm gonna give you today, instead of picking and choosing maybe two or three of them, just list all of them. Unless you're going for a particular niche on eBay, then you're more than likely gonna have a general store. In which case, you can list as many products and as many niches as you want. So why not list all 10 of these different items? They're already trending. They're proven to be trending. They have a proven track record and people are constantly searching for them. So if you list all 10 of them, you're gonna have 10 times the chances to get a sale. Those are pretty good odds. Now, before we start the countdown, we're gonna have an easy to reference cheat sheet that's gonna give you access to all of the different links for all of these different products. Now, if you want access to that, all you have to do is go ahead and comment down below with the hashtag eBay products. And let me know whether or not you're gonna be adding all of these different products to your store. Are you or are you just gonna pick and choose a few of them? Let me know down in the comments below. And once I see that you wanna add in commented, I'll reply back with a link to the cheat sheet. So kicking off this list, the first item that we have is an anti-slip car mat for phones. So we all know that when it comes to driving, you really shouldn't be looking at your phone too much or even at all. But sometimes you need to so that way you know where you're going. Me personally, I always need to have my GPS turned on and it's extremely annoying having to have my phone in one hand, my other hand on the steering wheel and trying to drive while figuring out where I need to go. It's very distracting and quite frankly, it can get pretty dangerous, especially in a busy street. And that is where this product comes in. This product lets you simply just place your phone on an anti-slip mat and it lets you choose your viewing angle. Best part is it's non-slip. So the non-slip surface, what's gonna happen is it's just gonna be able to be placed on the dashboard and it's just not gonna move. Without having to use any adhesive, that could potentially damage your dashboard. Now that right there can be a huge selling point. A lot of different car mats or anything that has to go on a car dashboard in order to place anything on there, like a phone holder, for the most part is gonna have some sort of adhesive that can damage the dashboard. So you can market this product by simply saying no adhesive needed, no extras needed. All you need to do is simply place your mat on your dashboard, no damage to the dashboard. And trust me, that in itself is gonna get a lot of people clicking on it. Now another really neat feature about this product is the fact that it actually has a swivel. So you can choose your viewing angle and you can set it to your preferred number. So let's say you have the mat placed one way, you can have a different viewing angle for different occasions. You can have a different viewing angle for let's say the passenger seat or the driver seat. The passenger seat might be at viewing angle nine while the driver seat could be at maybe three or four. It's really at this point all personal preference, but this is also another cool feature that you can advertise as well. Next up we have a USB IR remote. Now what this is is simply a remote that can control your phone. Most of the times people use this to take pictures. So what happens with one of these is depending if you have an iPhone or an Android, you're gonna choose one of these two plugs and it's gonna go into the charging port of your phone. Then with the other one, this right here, this little controller, all you have on there is a simple button. So for the most part, this is used to control your phone's camera. But nowadays you can use it to control pretty much anything in your entire house. It's absolutely crazy. So this product actually has a mobile app that you can download that works with this remote control. That's how it's able to control pretty much anything in your house. As long as it's a smart device that can connect to your phone, either through Bluetooth or Wi-Fi, then you can control it using this controller. Next up on our list at number three, we have Bluetooth sleeping headphones. Now let me tell you one quick thing about this particular product. So there's an influencer whose video I was watching and they were talking about their first store. That first store actually happened to be a very similar product to this one. It was an eye mask that I believe played music. I'm not entirely sure. I think it was that it played music and it had a few other extra features on it. But at the end of the day, they started dropshipping this product and they got their advertising down perfectly. So they eventually ended up building a brand around this entire product. Later on down the line, it was doing really well. They sold the company, made a few million dollars and that's it. Of course, there's a lot more that goes into the story, but it's really cool and it's really crazy to know that a product like this can have such an impact and it can be so successful. It's really all about how you market it. Of course, you also need to have a product that has some sort of value and a wall factor, in which this has both. But still, you know, to be able to see a product that has such a backstory, it's absolutely mind blowing and quite frankly, inspirational. Remember, you're one product away pretty much from retirement. But anyway, what this product does in particular is it's simply a mask that you put over your eyes or over your face as you're going to sleep. This can help you get to sleep by playing music or white noise. Up next, our fourth product is gonna be a portable work light. Now this particular light is actually extremely bright.  Trust me, it's not like a regular light. It's not like this right here. If I purchase it and I turn it on in this room, it's gonna illuminate the whole thing as if it's daytime. It's absolutely crazy. But aside from that, I'm pretty sure if you've checked out some of our previous Top 10 products in the past, you've probably seen this item once or twice before. And the reason you've seen it before and you're still seeing it now is because it's still making sales. People are still purchasing it and it's still trending. So if you don't have this in your store right now, I highly suggest that you go ahead and import it right now. Speaking of importing, did you know that the quickest and easiest way to import products to your store is by using AutoDS? When you use AutoDS, you automate your entire drop shipping store. Now, if you don't have access to it just yet, you can get access right now for just $1 for the trial period. So you can see if you like it or not. But trust me, you're gonna test it out.  But if you wanna check it out for yourself, make sure you just check out autods.com and sign up for the trial period. But to quickly show you how efficient it is in certain aspects of the business, such as product importing, check this out. So I'm actually gonna import this item to my store. All I have to do is simply import draft, that's it. Now, if you wanna find more trending products, we have a winning product section here as well. And we also have a marketplace. Now, the marketplace has an insane amount of items from a few different sellers, but our winning product section has our private suppliers in here and our warehouse. Either way, whichever section you choose to go with, you can't go wrong. You're gonna find some pretty good items on here, some of which have some extremely fast shipping that can even rival Amazon. So on here, you can see that this one ships between two to 10 business days, but that's two to 10 business days to get to your customer, not for the supplier to actually ship out the item. So let's say for example, we go into this product right here. So these are wall decals to give your wall a sense of depth.  This gives you what it's being sold for and how much you can potentially make. On top of that, you also have a target audience if you're running Facebook ads. Now the cool thing is that this target audience is similar to what this Facebook ad is targeting. So we also give you a Facebook ad that you can use as inspiration so that way you can structure your Facebook ad similar to this one. And if that's not enough, we also give you a website that's selling the same item. So that way you can get some inspiration to structure the website as well. Now this next product is actually one that's really interesting. I've seen it use a few times and I've seen a few Facebook ads about it and I actually considered purchasing it myself at one point. But now that I have a supplier, I'm just gonna order it from the supplier. But anyway, this next product is a snowflake hex key. So this is pretty much a key chain in the shape of a snowflake. And on each tip is a different tool. So on one, you're gonna have a flathead. On the other one, you're gonna have a Phillips and so on. Now this is pretty cool because this is practically a portable screwdriver with a bunch of different types of heads. This is perfect for people that like to do things outdoors like let's say ride a bike. A lot of times if you're riding a bike, maybe something can go wrong with the bike. You have this tool on you to be able to make any necessary small, of course, repairs or to tighten any screws or anything like that. Up next on number six, we have a collapsible silicone water bottle. Now this is really cool because a lot of the times you have a water bottle and it's super clunky, like let's say those new Stan Lee cuffs, those things are huge. So what this does is simply make your water bottle into a key chain. This is super cool because you can take it with you anywhere and you can refill it anywhere as well. Now of course, if it's filled with water, you're not really gonna be able to compact it into the size of a key chain. But the fact that you have that option to be able to take it around with you anywhere else or to be able to simply put it away once you're done with your water is absolutely fantastic. Next up, we have an automatic multifunction fruit peeler. Now, a lot of the times when people are peeling fruits to make a fruit salad or to simply just eat the fruits outright, what they do is use a knife or use a handheld fruit peeler. Now either or of course it's fine, but me personally, I don't like using either one. For one, when I use a knife, it's really complicated and I end up taking off huge chunks of the fruit itself. Okay, maybe it's not that complicated but I'm just not good at peeling fruits with a knife. And the other option is using a handheld fruit peeler. Now this is perfectly fine as well, but it can potentially get kind of messy.  Mangoes are really juicy, it's gonna make a huge mess and yeah, at the end of the day that's really personal preference. But this makes it a lot easier. So as you can see, all you have to do is stick your fruit on there and just give it a spin. Once you do that, it's gonna automatically peel the entire thing without any mess and without getting you dirty or without getting anything else dirty. Another thing is that if you don't like getting your hands sticky or if your customers don't like getting their hands sticky, this is the perfect tool for them. That's a huge pet peeve of mine.  And when you peel fruits with your hands, your hands are gonna get sticky. So this is the perfect tool for people with those little pet peeves. Next up, we have a sink filter rack. Now I actually absolutely love this and I'm about to purchase this myself because this can really make cleaning dishes a lot easier, especially if you have kids that leave food on their plates and they just throw it in the sink. You know, instead of doing the normal thing of either throwing away any leftovers or simply putting them in the fridge, kids like to just grab their plates and just throw it in the sink. So something like this can make cleaning a lot easier and a lot more hygienic as well.  You can simply just take it directly from the plate and throw it in this and then just empty it out later on when you're done. Now this actually comes with multiple nets, which I think is really cool because you can change the net that's on here to either throw away whatever's inside or to simply hold things like your produce. Let's say you have lettuce and you're rinsing it off, you can just place it on there. So this filter rack is actually multifunction. It's not just for garbage. Next up we have a portable camping light. Honestly, at this point, this is essentially the modern day candle. This is so cool. This is really just a light bulb that you can take around with you that you charge via USB. It's, I don't know, I'm mind blown right now. I think this is absolutely fantastic. This is the first time that I'm seeing this. But honestly, I love this product. I think it's so cool. I think it's a great modern twist. And I really do think that you can market this as a modern day candle essentially. So pretty much what this is is simply a light bulb socket that can be charged through USB. These are also waterproof, which makes them perfect for just what they're advertised for, which is camping. So this is perfect for the outdoorsy type people that like to go out at night. People that like to hang out by the campfire, maybe. People, again, that like to go camping, maybe hiking or fishing. Pretty much the outdoors type people. Now at number nine, we have the Crystal Rose Table Lamp. Now I'm pretty sure, again, this is one of those products that you've seen before. But again, it's because it's still trending. People purchase these for multiple occasions throughout the year. These are the perfect gift. Now, when I say the perfect gift, I don't wanna say that it's the perfect gift to encompass all gifts. What I mean by perfect gift is that this is a gift that people are gonna love. This is a gift. This is something that you give people. A lot of the times, this isn't something that people purchase for themselves, even though they can, but mostly this is something that people are gonna purchase to give to someone else. Now, when you're offering this product, it is important to know that they come in a few different variations. So there's three primary variations to this. The small, the large, and the three colors. So essentially the three colors only changes to three different colors, but the RGB changes to pretty much any color that you see up here. Any color that the colors red, green, and blue can mix into. And last but not least, we have an electric mini waffle set. In this case, it's a heart shape, but there's a lot of others that you can choose from. So they have different ones that actually make breakfast fun. So you have some that are squares, some that are circles, some that even have designs built in so that way when you make the waffle, it comes out with a design on it. These are perfect for parents, especially parents that make breakfast for their kids before they go to school. It really gives the kids a little pick-me-up and it makes them a bit more excited to wake up in the morning and actually go to school. And those are the top products that you should be dropshipping on eBay. Remember what I said, eBay is a numbers game. So don't just choose two or three different products, just import all of them. And if you want access to all of them and all of the different links, make sure you check out our cheat sheet. For access to that, just go ahead and comment down below hashtag eBay products and let me know if you're gonna use all of these products or if you're only gonna use a handful of them. Huge thank you for watching, especially if you made it all the way to the end. As always, it truly means the world to me. Please, again, as always, you made it this far. Might as well smash that like button. Might as well just go ahead and subscribe. If you made it this far, that means you do enjoy our videos. So why not subscribe for more? Also hit that little bell notification so you don't miss out on any future videos and I'll catch you all next time.


Hi everybody. Thank you so much for joining today's Protocol Labs Research Seminar. Today we're joined by me. I'm going to be the one presenting the research today, very excited about it. And we're going to be talking about what influences NFT secondary pricing markets. To give you a bit of background on what I do, I am the marketing lead for the network goods team, which is formerly the Protocol Labs Research Team. And before I joined PL, I was a research associate at Cornell University and also the smart contract research forum, focusing on consumer economics as well as game theory and other things that impact crypto economics. So thank you so much for having me today Protocol Labs. And I'm going to be presenting my research titled Influencing NFT Pricing on Secondary Markets, a case study of vPunks. And I originally presented this at the Crypto Assets and Digital Asset Investment Conference, which was in Wren on April 7th. So a big shout out to Brian Lucy, who was the one who coordinated that conference. And I'm really excited to present this again here today. So with that, this research was independently done by me. This is not done by Protocol Labs. And so any of the NFT projects or companies or anything like that that I bring up in this presentation is solely meant to describe the research. It's not meant to be an endorsement of any type of PL or anybody at Protocol Labs. So quickly, we're going to introduce what we're going on about today. And really, the background and literature review of NFTs is pretty slim at best. So as many of us know, NFTs allow for immutable and permanent proof of origin for digital assets. So this could be, for example, a photo or music or assets in player and video games, for example. And there's a whole lot of other assets that are starting to be used for NFTs. But those are really kind of the bread and butter that we've been seeing recently. And surprisingly, even though this is a huge industry at the moment, there has been hardly any research that has typically been done that looks into what influences actual sales of NFTs or what impacts the prices of NFT sales. Now, this is kind of growing. There's been a few more conferences this year and a few more journals that have been asking for this type of research, but right now it's really limited. So what we wanted to do was more of a basic study, and we're going to get into the details of that later. But to kind of get this into perspective, NFT sales are obviously a growing multi-billion dollar industry. We've seen this for people who have been paying attention that this has grown from a couple million a few years ago, so billions just last summer alone.  But it's still very popular. In an example, what we're seeing are websites like OpenSea, LooksRare, or Rareable, are kind of your top three NFT markets at the moment. Now, remember, NFTs can allow artists more direct access to buyers, and they also can potentially receive more money in secondary sales. So for example, on OpenSea, one of the things that people can do is when they mint NFTs or when an artist mints NFTs, they receive the initial payment when people mint them. But then also when that NFT is sold a second time, like a secondary sale, that artist receives more or less a royalty for every sale. Now, that's different than traditional artwork. Whereas if I went and bought a painting from an artist, and then I wouldn't sold that painting, I'm more than likely going to keep all of the money that I sold that painting for the second time, right? And that artist doesn't get anything. So that's a big benefit or a big potential benefit for artists who want to work with NFTs. And so there have only really been two major studies that have looked at anything related to NFT prices. And that was before I wrote this paper and this presentation. There might be more now. But at the time of writing, there were only two and they're both done by Michael Dowling, one who's looking at the land NFT and DeFi land and found that it was pretty inefficiently priced. But the prices increased rapidly. And that was pretty dependent on how many users were using the game and how many people were playing. And then also NFT prices have been shown to be independent of Bitcoin and Ethereum prices, right? So if Bitcoin goes up or Ethereum goes up, the NFT prices in terms of Ethereum and Bitcoin don't necessarily go up as well. So those are some things to keep in mind. So our research questions we're taking a look at today, there are three main ones. And the TLDR of what we're looking at is what are some of the influences that impact secondary NFTs, okay? But the three main questions we're looking at, number one is, does the ability to sort by active sales and auctions or sorry, does the ability to sort active sales and auctions by price and or attributes impact the NFT's final sale price? Question two is, does the existence of a roadmap impact the secondary sales price of NFTs? And three is, does the size of an NFT specific community impact that NFT's secondary sales price? Okay, so these are three things that artists or community managers or NFT sales platforms, for example, OpenSea looks where all those may want to know to be able to maximize NFT sales moving forward or maximize price or increase the speed of NFT sales. All of these questions may have something to do with those. And for those of you who are impatient and you just want to know the answer, yeah, all three of these things do impact NFT secondary sales prices. So we're going to dive into what that actually means and how they actually impact prices. But to be clear, all three of those questions are resounding. Yes, they do impact prices. What we're using today are called V-Punks. I'm going to explain what those are. We're just using this as a case study. And the reason why we're using it as a case study is because it gave us a very unique opportunity to study a marketplace both before and after auction sorting features, before and after a large community was formed, and before and after a roadmap was published. So this is a very unique case study where a lot of other NFTs don't have that type of opportunity to take a look at those before and after situations. So what are these? V-Punks, more or less, think of crypto-punks that we see on the Ethereum blockchain. This is very popular. It's about 10,000 unique 24 by 24 pixel algorithmically generated images. And the way that these were minted actually was pretty interesting. So the first 1,000 V-Punks only cost 390 vets while our VET, which is the ticker symbol for the V-Chain token. And then the final 9,000 to 10,000 V-Punks that were minted cost 3,900 vets. So it increased for every 1,000 that were minted, the minting price increased as well. So for those who got it really early, the minting price was cheaper compared to those who got it pretty late. Now, V-Punks are stored on what's called the V-Chain Thor blockchain. So for those who don't know, this uses a dual token model. So VET is the ticker symbol for V-Chain. And basically, what you do is you hold the bet in your wallet, and then that generates V-Thor. So that's a V-T-HR, if I'm recalling correctly, for the ticker symbol. And if you think of this in terms of Ethereum, when you're doing transactions on the Ethereum blockchain, you need to burn gas, which you're burning Ethereum for each transaction. Whereas with the V-Chain blockchain, what you're doing is you're doing transactions typically priced in VET. However, you're burning V-Thor. V-Thor is like the gas token.  So just to give you a little bit of background there. Now, the V-Punks had created its own auction house for secondary sales and also just fermenting as well. The reason why is because the NFT marketplaces on the V-Chain blockchain and platform didn't really exist at the time when V-Punks launched. There wasn't like an open sea. There wasn't a looks rare. There wasn't anything like that. But when you want to go inside, okay, I'm going to go buy V-Punks second on a secondary sale on this website. And so the team who made V-Punks actually had to make their own auction house and their own website to be able to do auctions. So to give you some important dates, on July 25th, 2021, the initial minting of V-Punks began. Then on July 30th, the official V-Punk telegram channel was created. So this is interesting, right? Like they didn't have a telegram channel before they started minting. Whereas now what we see with a lot of NFT mints or drops, we're seeing telegrams or discords being formed beforehand. And they're using that to kind of hype up the mint. So this is an interesting case study here that they didn't do that before. And then finally, on August 5th, the 10,000th V-Punk was minted. So it took a couple of days for these to fully mint out. Where we're seeing other PFP NFT projects that have like 10,000 images. Some of them are minting out in the day. Some of them never minted out. This one, they fully minted all 10,000 of them. Took about, I don't know, two-ish weeks or less for the full mint. So really quickly, this is what they look like. The one on the top here on the screen is that's one of the ones that I own personally. So these are kind of what they look like. We've got four examples here. So there are many different character traits they can have. Each V-Punk has a corresponding rarity score. They are aliens, apes, and zombies. Those are the rarest type of like humanoid forms that they represent just over 1% of the total supply. And then they could have up to six attributes for the max and a minimum of one attribute. So they all look different. None of them are the same. And based on a combination of different attributes, they all have different rarity scores. And in theory, and what people postulate for these types of NFTs, is that the higher the rarity score, the more expensive that these NFTs will be on secondary sales. And so we're going to take a look at that as well in a second. But the data methodology that we have today comes from a few different places. So we have the V-Punk secondary sales. So we have on-chain secondary sales provided by the V-Punks team, specifically the community management team of them. And what we got from that data was the cost and vet for every single secondary sale, the rarity score for the V-Punk that was sold at the time, each individual characteristic of each V-Punk that was sold, and then the time of sale. And we also received the number of Telegram users provided by the V-Punk community management team. And so what that means is we took a look at the Telegram once it was launched, about five days after the initial minting period started. And we took a look at the number of Telegram users up until about one minute before every single sale. So if we were looking, for example, for sales today, what's today's date? May 24th. So if we were looking at a sale that happened at 12.13 right now, we'd look at the number of Telegram users up until 12.12 of today. And we would use that for our data point. So a couple of important dates that we need to take a look at here. Remember when I said that the V-Punks team needed to create their own auction house? Well, when they did that, the auction house was very, very bones. It didn't have a whole lot of features that what we're seeing today in some of these more mature auction houses like OpenSea, Luxury, Rarible, et cetera, et cetera, right? So on August 11th, they added a sort by price feature. On August 15th, they added a sort by rarity. And then on August 25th, they published a roadmap. So remember, it started minting on July 25th. And then if someone minted one of the V-Punks in the beginning, they wanted to sell a secondary, they could list it for sale, but no one could sort by price. They couldn't sort it by rarity. And so to do that, what you needed to do was basically click through each individual page to find whatever V-Punk you wanted. You couldn't sort them until August 11th and 15th. So that's a very interesting concept to take a look at and how this auction house evolved and how that potentially impacted the NFP sales later down the line. So what does our data look like? We have 7,272 sales that we take a look at. The start date for our data set is July 29th, 2021. And the reason why we don't start on July 25th, which is when minting started, is because we wanted to take a look at only secondary sales. We're not looking at minting data at all. So if something minted, or if there's a sale that happened for the 29th, we don't take a look at that as only 29th and afterwards. And it goes in from July 29th until September 6th. So about a little over one month more or less. So the average sales prices in vet was 9,328 vet. And unfortunately, I don't remember what that average sales price was in dollars. We can go back and take a look at the historical information on the coin tracker or multiple different websites. And then you can also take, we have 69% sales were post price sorting. So 69% of our 7,272 sales points that we're taking a look at was after August 11th. Then 42% were post the roadmap. So that's after August 25th. 62% were post the rarity sorting. So that's after August 15th. And then at the time at the end of the data set at the end of September 6th, we had 1,616 telegram users in total. So that kind of gives you an idea of what our data looks like. So here's our methodology. I quite frankly hate these types of slides, but I know we always have to talk about them. So these are our two base models that we have. And how we take a look at this is our dependent variable is going to be the log value of the sales price in the vet. And the reason why we don't just simply use the continuous variable of sales price instead of we take the log value of it is that the floor continuously rose throughout our time period throughout our data set. And we need to take that into consideration. So when I mean the floor, I mean the minimum sales price per NFT that was being sold on the secondary market, we take into account that the floor was rising to be able to really take a look at what these individual variables did to these secondary sales prices. So we take the log price of that at the time of sale. And then we have two different models here that we're looking at. And each one of these actually has three different equations. So on the right side of the screen, you'll see one, three, and five, and then two, four, and six. Those are different equations that we're going to take a look at later in the results slide. But basically, we're taking a look at how the sort by price function, the sort by rarity function and the published roadmap impacted sale secondary sales prices. And these are both binary variables. So if it has a one for sort by price, that means the sale took place after sort by price was implemented into the auction house. If it has one for rarity, it means that sale took place after sort by rarity was added to zero means it was done before. For that top equation, we have rarity score, and we also have the number of kilogram users as independent variables. And rarity score again is going to be more or less a combination of the different attributes that each feedback has. And what we wanted to do also is make sure that that rarity score had what we were to expect some correlation between these individual attributes that these view punks could have. So then that's why we had the second equation on the bottom. Again, the beginning is the same. We're using the log sales price event. We're also using the binary variables for sort by price, sort by rarity, and published roadmap. But instead of a rarity score, we instead have a variable that accounts for the number of attributes. Again, it could be between one and six. And then we also have the humanoid form, which means it was either a male, female, zombie, or an ape. And we're comparing those four types of humanoid forms to the alien form, specifically, because that has the lowest number of V punks that are alien human oids. And we also have the number of telegram users. So when we take a look at all this long story short, this should tell us, in theory, if any of these variables have an impact on the secondary sales price in terms of vent. All right. So that's that. Those are the models there. So let's take a look at it. Is there any impact? And if there is, what are these impacts? Now, again, this is one of those other slides for economics that I really hate showing, because there's a bunch of numbers. But it's very important to take a look at anyway. So the big things that we find here are once the auction house had features such as sort by price and sort by rarity, the actual secondary sales price for these NFTs decreased. And you notice those have three little aspects that's next to it. That means it was significant at a 1% level. Very important. We also found that project features and community grow, such as having a roadmap published and having more telegram users, both of those significantly increased the log value of these secondary sales for these V punks. So what does this really mean? Basically, sort by price and sort by rarity, which makes it easier for people to use the auction house actually decreased the secondary sales prices of these V punks, whereas adding a roadmap, publishing a roadmap. And in this one, for example, for V punks, they published the idea of creating what was called the VPU or the V punks utility token. So they actually launched the run cryptocurrency that is meant to be used in future play to earn games that will be featuring V punks. And so once that roadmap was published, the price of any NFT secondary sale in theory increased afterwards. And we can see that at a 1% level. And then also we see more telegram users as well also increase the secondary sales prices for V punks. And something that I don't really have on the left-hand side there, but I do want to address is that the rarity scores, the higher the rarity store, we see the higher the secondary sales price for these V punks. So that's rarity score. That's the fourth one down on the left-hand side. And then we also see for the more attributes that we have for V punks, the higher the secondary sales price. And then we also see that when we compare to aliens, apes, zombies, female and male humanoid forms all have a lower sales price than what aliens would have, comparative, right? And that makes sense because aliens have the smallest supply, followed by apes, followed by zombies, followed by females and males have the highest supply in this specific NFT drop. And so what we see is those have a pretty consistent impact on price where the ones with the lowest supply have the higher prices and the highest supplies have the lowest prices. That's not surprising, but it's something that we did confirm. And it's important to take a look at that as well. At the end of the day, what does this actually mean? And why is this important for anybody? Who's going to use this? First off, there needs to be, and I know this is really fast presentation, but that's okay. So we have time for questions at the end. First off, there needs to be a lot more research done that looks at the behavioral economics of NFT buyers. There's not really anything out there right now that looks at why people make certain decisions, what makes them want to buy one NFT over another, what makes one type of drop more exciting or the type of artwork that people want to buy. There's not a whole lot of research about any of preferences or any type of behavioral economic research for buyers. So that's one thing that we need to take a look at. We also should expand this study to include significantly more different types of NFTs, right? This is just a one NFT drop that we're studying here, and there's a million of them out there with datasets that are relatively easy to get if you just take on-chain data, download it, and you can put it into models. And so we can do that. We can take a look at if these results hold across other NFTs as well, or if this is a one-time thing. And then also, we would like to see more studies that will look at NFT sales across different established platforms, such as OpenSea, Variable, or the newer one looks rare, because, like I said, these sales were all done on an auction house that was brand new and was built specifically for V-Punks, right? Whereas these other larger websites like OpenSea, people know them. People go and buy stuff there all the time. They have huge volumes, and so studying how some sales are done there and velocity of sale and whatnot may be a very interesting thing for future researchers to take a look at. However, what our findings today mean, and who can these findings be used for is also super important, right? So really, this research is meant for people who are looking to enter the NFT market, so artists who maybe want to launch their own NFT line, and also for companies who are using their platforms as a place for secondary sales or auctions to take place for NFTs. And what we found is that community is super important. At the end of the day, when you run the numbers, we found that for every one new Telegram user that joined the V-Pump Telegram that increased the secondary sales price by 0.2%. Now, that sounds relatively small for an individual person, but I mean, if you think about that, five new people, that's 1%, that's compounding, right? So if you're able to get a ton of people, the large community, very excited about it, then you may be able to increase your price as an NFT artist. And that's not necessarily surprising, right? If you have a higher community demand for your product, for your NFT, maybe higher, if demand's higher in theory, we would expect prices to go up as well. Now, what we also found is that roadmaps were important. And I do want to flag that for a discussion at the very, very end here. But what we found in this specific study was that after the roadmap was published, the secondary sales prices of V-Punks actually shot up at roughly 33%, almost immediately. And it was pretty consistent for the days that followed the roadmap publication. So if you're an artist and you're able to build value and show there's a future in the product and the NFT that you're minting and putting out to the world, people may get more excited about it. They may be willing to pay more for the NFT if they believe in that roadmap and they see value down the line. So that's something that artists should consider potentially doing. Also, we found that the higher the rarity score and the lower supply of certain characteristics, the higher the NFT sales price, again, that's not surprising. But what that does mean is that if artists do want to maximize some of these secondary NFT sales, they do want to maximize hype around certain drops, they may want to increase these very hyper rare traits where maybe one of every 1,000 or something like that are a certain color or something like that. And having that higher rarity for certain types of NFT projects could potentially increase these secondary sales prices. And then we find that the sort by rarity and sort by price, when both of those functions were added to the auction house, we found that the prices, the secondary sales prices actually decreased and it was relatively significant. Now, I do want to just give my theory as to why that is. And again, this is just a theory, this is why more research should be done with this. But if you recall, when the auction houses were originally built for the V-Punks community, none of this, none of these sort features were there. So you had to go through and click every single page. And at times, that would be hundreds of pages of options for people to click through. And if they wanted to find a deal, they wanted to find one that was cheap, they had to keep spending a ton of time and just clicking, clicking, clicking, clicking, clicking, clicking. And eventually, they might have gotten higher. They might have gotten lazy and just bought one that they liked. They didn't really care about the price necessarily. So they didn't buy the cheapest one. They just bought one that they liked. But if you add these sort by rarity features or these sort by price features, then in theory, you can just sort by price lowest to highest or highest to lowest. And then buy the cheapest one and you immediately have one of these NFTs. It makes your job much easier as a consumer to just buy the cheapest thing on the market. And this is anecdotal. And I didn't, I don't have any proof on it with the data, but I did run a couple of tests. And we did see that after these sort by price and sort by rarity features were added, what we found was that those NFTs that were kind of selling in between the maximum price and the lowest price, those in between middle priced NFTs were not selling very much. It was typically the ones that were priced the highest or the ones that were priced the lowest that were selling. And those ones priced in between basically had to wait until the NFT price floor increased to get to their price. All right. So those are some of those things of big take home things. And I didn't say I wanted to flag road maps. After this study was conducted, there was another study that was done that looked at multiple different NFT collections on OpenSEA and used huge amounts of data, taking a look at secondary sales there. And what they found is that actually projects without roadmaps typically reached a higher at the height in terms of ETH for secondary sales than projects with roadmaps. So that's a very interesting concept, right? That's a very different finding than what this study found. And something that I think is very important to take into consideration with both of these is that this is a brand new area of research and we need a lot more research done that looks into pricing of NFT sales and how roadmaps impact them. Maybe something that should be looked at more because we are getting some conflicting data or conflicting results, my apologies, with different research papers and people that are publishing them. So that really concludes the main part of it. Here's my information for those who are looking at it. So thank you everybody for coming today. We really appreciate it. My cat me looks really wants to be in the video apparently. And we look forward to seeing you guys next time.