What is going on guys welcome back to the channel Critical Overlord here since I'm about Halloween ends in this video once again We're gonna be talking about the OG scream Queen Jamie Lee Curtis or one of the OG's many out there So Jamie Lee Curtis has wrapped filming on Halloween in she put out a heartfelt post Earlier today where she's titled or where she captioned it a bittersweet in for me on the Halloween movies I've made great friends and have collaborated with wonderful artists on these three movies and today my part in the film has Been completed and with it the end for me of this trilogy that of course being the David Gordon Green timeline That consists of three new trilogy of films that are tied to the original John Carpenter classic from 1978 And she shared this nice image here that you see on your screen Lori Strowed forever Now I have to address the biggest concern that I see a lot of people have They think that this means that Lori Strow dies that on top of the fact that she is in the past stated that the Ending is not gonna sit right with a few people or some people it might make some people upset You know combining that now with this she's already done Filming and she hasn't the production itself isn't done So people think that this is even more proof or more reason to believe she's going to die I will say the the way that they've managed to keep a lot of things Tight-lipped and secretive this time around is very impressive. They definitely learned from the Halloween kills issue But this does not mean that Lori Strowed is going to die You need to remember that there's a lot of instances where movies can be shot out of order If I'm not mistaken, I think that's more common actually where movies are shot out of order This would be to accommodate that the time you have available for certain actors, maybe budget stuff And just other other things that are going on involving the production That would that would resort in the movie being shot out of order So that way you can still have the movie you are desiring to have in the final product when you put it put it all together And put everything in a nice Correct order, even though you shot it out of sequence for the most half So even though Lori is done Jamie Lee Curtis is done with the movie in terms of filming it That doesn't mean Lori Strowed is dying Lori Strowed very well might actually be in the finale I have no doubt that she will be in the finale actually only because of the fact that if I'm not mistaken This is the longest time she spent working on one of these movies since this new trilogy started I think this was the longest time She spent on set out of out of the other two movies that she's done Halloween 2018 and Halloween kills I think she spent the most time with Halloween ends a lot of her scenes probably just have already been shot Such as the fact of her final moments in the film Which I'm certain will be in the finale or leading into the finale if she's not in the whole entire finale aspect That doesn't mean that she's dying though Something could happen where she sideline for the rest of the movie like I've stated before You don't need to have Lori Strowed die for this to happen But I very much so rather have Allison Nelson be the one who has the final say against the shape in the in a manner in which Lori Strowed kind of Kind of is the one who gives the assist to that She does something to Michael Myers, but Michael does something to her which sidelines her But then in the end Allison and Michael are making up a big bulk of what ends up leading to the ultimate demise of Michael Myers Whether that be us actually seeing him die or something else who knows But then Lori could have like one final who raw Where she assists her granddaughter in taking out the shape while she's still sidelined because it's something that was done to her earlier in the movie and Then that's just how the movie could play out. That's that's how I envision it That doesn't require Lori Strowed dying her dying isn't necessary if she does die I'm fine with that too. Just as long at this point I'm only fine with her dying because you've already done it once before it just was terrible if she dies this time around Hopefully it's something that is making up for that lackluster exit She had from the franchise with Halloween resurrection because in a lot of ways just to talk about Jamie Lee Curtis really quick I'm glad that we got these new these new three movies with Lori Strowed coming back and with Jamie Lee Curtis getting to have a Recreation of what she can do with this character and now end it on better terms The Halloween resurrection exit for a character like Lori Strowed after what Jamie Lee Curtis has contributed to the character in previous entries That was completely lackluster. It was very much Very much so in the same vein as what they did with Sally in the latest Texas Chainsaw So I'm glad we have Lori Strowed back for these for these latest three movies And if she does die again, that's that's fine. It's just a manner in which how you do it. That's really gonna Make the difference and whether or not I can accept that or not like Nancy's death in Nightmare Nelm Street 3 It doesn't bother me because she's not she's not being It's not a death that is so disrespectful to the character like Sally's death in the latest Texas Chainsaw That is atrocious. It is it's just atrocious It's a complete slap in the face to the character itself Even though she is paper thin in the original anyway But with the character like Lori Strowed who has proper character development and we've seen her come back multiple times We have different timeline variations of her But they always connect back to that original film where Lori Strowed again is a much more fleshed-out character Even when she's with her group of friends, they're constantly making stuff They're constantly pointing you the audience towards Lori even though there's two other girls with her Those two other girls are always talking about Lori in a way or turning the conversation in a direction that makes it about Lori So then you the audience are again being conditioned to grow attached to Lori And then you spend a big bulk of the movie with Lori as she's babysitting and then now we're here 40 years later Where she's still suffering from the events of what happened 40 years ago in a movie in which you were given enough time spent with a young girl who was just trying to Focus on her studies graduate Be a normal girl and then her life is turned upside down by this maniac and it's been plaguing her for the past 40 years So there's some there's more sentimental value there with Lori And they've been doing a great job at portraying that in this latest in this latest Timeline that we now have Halloween kills. I know she took a she took a step down in in terms of just resting in the hospital But with Halloween is I think she'll be more involved and I think her exit this time around from the franchise will be Something that fans will be in agreement with for the most part as opposed to us being in Disagreement with her exit for the most part with Halloween resurrection. I know those be there'll be people who like she said will have a problem with the Ending of the film that is going to cause some people to be upset But I think for the most part everyone will be fine with her exit this time around whereas before with the resurrection I know a lot of people were mostly against that exit. So We'll see what happens when I'm glad she came back. I'm glad she is done with filming She can focus on other things Maybe like participating in scream six which it's going to be one of the topics I have for my next video But let me know what y'all think about all this down in the comment section below If you haven't already, of course, make sure you subscribe turn on post notifications video in the description I have links in my social media accounts on facebook, twitter and instagram You can message me there, of course, then there's any movies news or views you're having to cover in the future And with all that in mind guys, I will see you in the next video


e of the politicians and then the international organizations that can help in that effort so what I'd like to see can we come to one or two things that we collectively feel we can push for and put that on the horizon so that if we come together in a year from now and some of you mentioned that we want to come together again in a year from now somewhere in Eastern Europe that we can say we made progress right so those are the points that I took away an important other one is that nobody seems to be ahead and already have that framework in place so even in the US even within the EU you see that some are a little bit further we heard from Latvia and Lithuania for instance but most are struggling to get this right so we have to do this together right we have to exchange lessons internationally we have to work with platforms so I think another reason why this conference really was not just at the right time but also with the right international crowd so just to say again we have that opportunity how do we seize it that's my starting point I'd like to hand over to Alexi Gromov from OCE ODIR to give his reflections thank you thank you and again as I told already for ODIR it's very important to participate in such conferences and such discussions and we were very happy to be co-organizers of this important topic as you probably know we are dealing and we are contributing quite essentially to the development of our methodology but methodology also for all other OCE participating states how to tackle tackle the campaign political and campaign finance issues including online now you probably know that quite recently it's actually not recently already some one and a half years released our new guidelines on how to observe campaign campaign and campaign finance online but maybe it's my takeaway and I could feel it also from all categories of our participants that's true everybody considers that not the campaign as such is an issue it's fine it's just a modern tool but maybe it's not even so modern no problem with this but there are two main principles which should be should be kept in mind while the authorities try or want to regulate these these activities and financing of the campaign online but also among the political parties among the voters the transparency and integrity and we have just discussed it with our keynote speaker Barbara Johnstone street about this the political finance and campaign finance online is a two ways to it it is important to set some legal frameworks some benchmarks so that everybody were on the same page of what are the main rules rules of the game actually but it's very important also to be to understand and to engage not only oversight agency who will as marching has said who will raise the red card and say no no no it's not allowed no it's it's not the main aim it's important that civil societies engaged and political parties also understand and take part in this process so that everybody could not see okay am I complying with the rules but to see whether there there is a level level playing field there is an integrity in the whole campaign process and campaign finance as well but as we discussed it today in particular it's very important also to provide certain safeguards or barriers against illicit illicit not campaign I would say but really misuse misuse of the campaign or the electoral process and of the democratic elections as such as an institution against some foreign interference some malicious activities and approaching to some disruptive disruptive elements and this is important to keep in mind and this communication and cooperation between different actors oversight agencies civil society political parties first of all will enhance I hope so increase the accountability accountability of the main players political parties first of all accountability accountability before their population before their voters and people before all of us but also it will create a kind of common common share and common responsibility that we have all of us we have to take care of and really be aware that there could be manipulation but we should be acting as one entity against these misuse and thank you again and as it was already mentioned there should be some more workshops and this joint joint activities organized and co-organized it's very good thank you over to Magnus thank you it's work right okay so I just want to one point that I'm taking away so I don't repeat what's been said in meeting the big challenges that we have ahead of us it's then essential that we find a balance between applying existing regulations and techniques on campaign finance the balance between that and the type of radical changes that that some and Nana was talking about yesterday and in that I think the point that the guitar also made yesterday in her virtual presentation is very important about identifying the principles for campaign finance oversight why are we doing this and the principles that apply to all forms of campaigning and then based on those principles identify the tools and regulations and monitoring techniques that are needed specifically for online campaigning which will be different from monitoring and regulating other forms of campaign finance in moving forward in these discussions this is excellent we need to involve political parties more we need to continue involving tech companies that we did yesterday and today but bearing in mind that the popularity of individual platform changes very quickly so if we invite one company one year doesn't mean that that's necessarily the company which will be my next time on behalf of my first one of the organizations behind this event I want to thank all of you for coming and participating so actively I really want to thank the central election commissioner Moldova for hosting us so expertly and kindly and I want to give a special thanks to Julia Shubilova to be honest if it wasn't for her we wouldn't be here so thank you to round off I want to nominate topic for the next time we meet a topic that's very close to what we've been discussing yesterday and today but we haven't had chance to get into a lot of detail it's the campaigning and the spending of non-contestants what used to be called third parties that is one topic we really need to address further as well and I want to end by sending an appeal to Verkhovna Rada in Ukraine to restart a party financial report it has to assume once the war is won Ukraine needs to win the peace and in doing that ensuring transparency fighting corruption would be essential thank you thank you Magnus over to Martin Engelbi of NDI Moldova thank you thank you Sam I have five conclusions big and small the first one is my conclusion is that we have a really really good invisible team here everything has been flawless I want to extend my thank yous to the NDI office staff Ina Karolina and Istina who have been doing a lot of logistics but everybody else as well interpreters tech team photo even photography I think everything has been completely flawless and now going into the substance one of my most cherished moments that I would remember from this conference is actually the speech of the president I think that was really showing the value of political parties political parties is the essence of democracy citizens coming together to make a better future that's why we have democracy so political parties are not it's not a bad thing it's a good thing and they need to be protected the genuine genuine actors need to be protected and the bad faith actors have been taken out and of course all parties have to combine but parties if we think that parties are in essence a good thing we should help them with compliance we shouldn't think about punishment we should we should help them with compliance and secondly and and another really good takeaway for me was actually today's section seeing the really great research that's being done in civil society and I think that with donors also being able to actually channel more money to civil society sometimes than the channel to government actors it's really important that civil society organizations that are interested in monitoring are given the right resources and that there is a permanent dialogue between the authorities and the civil society so that the civil society can produce research and results that they're actionable for for the for the authorities I think that's very important third one um no fourth one now Sam was saying can we do something before next time and I think we can I think there is a low hanging fruit and I think it's it's a political what they call the Google political ads report I think that there should be political momentum actually to put that on the table certainly by the international partners international idea has an office in in brussels and the eye and I first have offices in in washington dc I think and the eye has a has a tech team who's who's interacting with silicon valley all the time I think that it should not be given the world situation it should not be impossible to to to take that step with google telegram seems more complicated but that would be a major win I think and there are lots of elections coming not maybe in Ukraine we don't know but in mold over we will have three consecutive election years and my last conclusion is that I really look forward to keep working with all of you at both the international partners and the local partners when we meet the formula cc regularly it's very fruitful and I'm very looking forward very much looking forward to this cooperation thank you thank you Martin last but not least the cdc chair miss caravan go ahead thank you dear colleagues dear partners thank you very much from my side for these two amazing days full of discussions days in which we discussed about very important subjects for all of us and now we can make certain conclusions definitely today thanks to informational technologies elections are not as they were at least a few years ago and we talked about the fact that everything has potential and sometimes it can be fun to see what are the tools and we all together through special synergies can actually use the it tools in our favor to try to make the election process to to be a modern one but in the same time transparent and credible we spoke about the influences within the elections and it's a problem but not only a problem but also it's a crime against democracy this is how we see it and this is why we have to be well prepared we need specialists we need expertise we need experts and that is why I believe this exchange of information is crucial in our common path we talked about the fact that monitoring and control of the online space could be designed could be given to a special institution and we understand this very well that alone the election commissions or certain institutions state institutions that are controlling finances they cannot face this challenge alone because it's too complex what it's important is that we already talked these two days about these problems and how we can within our authorities underline the commonalities the problems that are similar for all of us we spoke about the regional context but the challenges identified are the same lack of regular regulatory framework enormous space that needs to be monitored and this is of course a challenge to cover it everything and also there is this financing by third parties there is the danger of financing from outside that can badly influence the result of the elections and as a consequence to denaturate the governance act itself so despite all of this we cannot limit us in using this space because it is already too spread and this way of disseminating information and communication with the society with the community is crucial today another very important thing we spoke about statistical data and the statistics show that the place of the social networks of the online is increasing the role is much higher than a year before and it becomes also more credible we talked also about referral documents and regulations at the european level but it doesn't matter how good those regulations are it could be perfect they can be perfect but they are not a universal solution in every case and we should focus on tools tools that would function in our context context i for example made a few notes about some very useful tools that our colleagues presented and we can implement those starting from today verification register of the eligibility online of those that are promoting themselves online also we have the monitoring system online monitoring system the colleagues from ukraine told about talked about that the analysis of expenses from Lithuania and they also talked about some courses and we are the first in line to enroll in those courses it's it's very interesting for us also we talked about the creation of a database that could identify geographically where a problem appears where a red flag appears where we should focus our attention and all of these tools can help us in our path of monitoring where the problems appear and how we can ensure a fair competitiveness with same types of weapons in if we can say so in a election competition we talked also about the paradigm about transforming the paradigm of transforming the challenge into the opportunity and I think the digital solutions are the ones that can help us collect that information assess that information properly and to compare that information in order to understand better where the problem is so I encourage everyone to transform all the challenges in opportunities and I think these practices must always be guided by some basic principles that my colleague told talked about efficiency and how fast we react is very important and it has to be proportionate and of course efficient efficient predictability we know already about some vulnerable points so we should focus on them we should focus all our resources on them because of course we cannot keep everything under control it's impossible continuity this is an extraordinary idea we were talking about the fact that even when the central election committee that was elected let's not forget that the mandate is a huge responsibility not only for the central election committee but also for each candidate each party that gets to the power let's also mention the cooperation cooperation through different platforms that can give can make available tools but also cooperation between election authorities different state institutions and of course central election committees they cannot cooperate only among them they need to be supported in their cooperation by the civil society but also by the other law enforcement agencies because all together we can enforce and we can apply our expertise in the benefit of the final beneficiaries who are the voters the voters by the way they should help us in this process to fight against misinformation and illicit financing of those parties through political culture through election culture through education education of the citizens about how we choose what information from the ocean of internet to consume and talking about ideas we made an exchange a beautiful exchange of ideas and we also talked about the importance of the post-election campaign of course there's there are always money in politics it's impossible to function without them but I remembered in this context about the paradigm that I liked a lot if I have an apple and if you have an apple then when we exchange those two apples we stay with the same apple one apple yes it's a different one but it's an apple if I have an idea and you have an idea and if we exchange these ideas we actually exchanging are becoming richer we now have two ideas so it doesn't work like in the case of apple so let's just use this treasure of having those amazing ideas shared so I encourage everyone to ensure the continuity of this exchange of good practices and experiences to be ready for the next challenges and to contribute together to the new interactions let's see each other at the next conference that my colleague mentioned already my panel colleague and this being said I would like once again to thank our partners international idea always see the national democratic institute ifs foundation and every single partner that helped us in our path and is are continuing to help us and thank you very much also for choosing us as a host of this event we hope that our host you felt at least a little bit of our hospitality in moldova and we are always open for your new visits to us so thank you very much thank you madame chair uh just by ending I'd like to go back to what the president maya sandhu said yesterday who referred to the political momentum the window for reform given that her government but many governments around this region are currently driven by reform minded people so we have an opportunity that wasn't around it just a few years ago let's seize that opportunity and make sure that the solutions that we bring about here in eastern europe are not just as good as we see in other parts of the world but are even better so that eastern europe starts to lead the way not just for its own region but really globally I truly believe in that and I think we can all help the CECs of this region to get there so thank you for that I think everyone in this panel mentioned the next conference so I think we can always look forward to that next conference I look forward to seeing all of you hopefully next year the jury is still out on the the topic of that conference I think a small campaign on the topic of campaign finance conferences has already started but count on us we will reach out for the next event and I'd like to end by a warm thank you to of course our partner starting with the CEC who was so kind to host us here and all the other partners that made this happen a few people in my own team that I'd like to really thank personally Julia Nanak Senya you did a fantastic job together with all the partners the staff of the CEC thank you so much for making this happen and thank you to all the participants for traveling from far and close to this conference I look forward to seeing you again in the next one thank you messages the translators yes translators of course I don't know how many languages but we will thank all of them all four languages you did a great job we didn't see you but you did a fantastic job it's a very heavy job and we thank you are there any housekeeping messages


Thank you so much, Louise. Very excited to be here. Thanks for the organizers inviting me and thanks for being offered the discussion in advance. So I'm gonna talk about my paper, Drivers of Digital Attention, Evidence from a Social Media Experiment. So for those of you in Toulouse, you might have seen a version of this last year recently revised it hopefully for the better. So looking forward to your comments. So yeah, so the starting point for this project was that people spend a lot of time online and in particular a lot of time on social media services, right? So last year, nearly the average American spent 1,300 hours on social media. In any given month, nearly half of the world's population use one of the most prominent social media applications, Facebook. And because of this increased time and concentration of attention on a select few platforms, there's emerged a broadly global debate kind of thinking about regulation in these markets, right? So you open up the economists, the New York Times kind of pick your favorite newspaper or pick your article. There have been continual articles over the years thinking about various regulatory issues in this space and particular questions of antitrust, right? And so this isn't just something that's been focused on in the press or in academics. The FTC is active monopolization cases against Facebook is actively rethinking a lot of their merger and competition apology strategy in digital markets more broadly. So if you go beneath the surface of kind of, in the FTC complaints in particular, in the Facebook case, a lot of the sort of back and forth between regulators and Facebook kind of go down to simple issues of consumer demand, right? The fact that these services are free to consumers kind of complicates the application of a lot of the standard antitrust toolkit. And so a lot of the back and forth is kind of which set of products are feasible substitutes for a product like Facebook. Now this question of demand measurements further compounded by the issue that's best depicted by pictures like this, right? So people spend vast amounts of time on these services, but there's sort of this notion of people being glued to their phones and kind of a, in a psychological and colloquial sense potentially having some kind of addiction to these applications with unclear implications for antitrust, okay? And so that backdrop, what I wanted to do in this paper was to try to think a little bit carefully about, you know, how can we study the demand for these applications? And I'm gonna try to answer two broad questions in this talk and paper. The first is kind of what are the set of relevant substitutes? How do we think about measuring them? I'm gonna have two kind of reduced for measures of relevant market definition that I'm gonna be able to get at. And second, you know, can we quantify the role that this kind of more broadly inertia plays in usage? And I'm gonna try to be thinking about how important quantifying this is for thinking about substitution patterns as well as these questions of market definition, okay? Now, because these services are free to consumers, you know, it's not obvious kind of what is the relevant unit of demand, right? And so this is a quote from, you know, the CMA Digital Markets Report in 2020, where there's again this kind of back and forth between, you know, Facebook and regulators where Facebook kind of argues that they compete with a product like YouTube, which isn't technically a social media applications primarily because they compete for the user's time and attention similar to YouTube. And so, you know, this isn't without controversy, but I'm going to kind of take this notion of time is being the relevant dimension of competition and think carefully about substitution patterns in this time dimension, okay? So what I'm gonna do in this project is I'm gonna run an experiment where I'm going to try that as the best as I can to kind of comprehensively collect data on how people spend their time, okay? So I'm gonna have people in this study install a print of control software on their phone, a Chrome extension on their computer that's going to give me minute by minute automated monitoring of how they're spending their time, right? And so this is the measure of demand that we're going to be collecting. Now, because we're in this world of collecting our own data, we're gonna use the fact that the way we collect data on the phone is through print of control software and we're going to have two restrictions. One is going to be restrictions of Instagram. So we're going to take away access to Instagram on people's phones. We're gonna take away access for you on YouTube for people's phones. There's gonna be some variation the length of these restrictions, but the sort of core of it is running an experiment. We're gonna collect data on how people spend their time.  Now, the first thing I'm going to do with the resulting data is to try to characterize substitution patterns of time allocation, right? So I'm going to argue that we can think of the sort of substitution that people do during the restriction period as even as kind of a conservative measure for which kinds of activities are people viewing as relevant substitutes, right? And so we're able to observe all the different applications that people use on the phone and surveys that give us some sense of how people are spending time off the phone to try to understand which class of applications are substitutes for these different applications. The second thing that we're gonna do is because the restrictions are relatively long, there's some variation in them and we're tracking people after, we can ask the question of whether or not there are persistent effects of the restrictions, right? So not just on the applications that are restricted per se but also on other applications, okay? Now, using this sort of more qualitative reduced form measures of substitution, I'm gonna use this to kind of guide the assumptions of this demand model that I look at. So I'm going to estimate kind of a demand model of time with a consumer inertia. And broadly the goal here is going to be to answer two questions. The first is to just answer the question how much of the overall usage is actually coming from this inertia channel? And then the second is we wanna get concrete estimates of diversion ratios between these different applications. And so I'm going to compute these with inertia and without inertia and then apply both of them to kind of a very simple, relevant market definition. And sort of the purpose here is to, one, I'm going to give you some argument that these sort of without inertia diversion ratios are sort of more core measures of substitution patterns but we're gonna apply both of them to kind of thinking about, these relevant market definitions, okay? So to sum up running an experiment to collect out on how people spend their time, gonna shut off different applications, measure in the reduced form how they substitute and then uses this variation in this data for a demand model, okay? Now, before going into the details, let's stay still a little bit of a high level to kind of think about what are we actually getting here, right? If you kind of think of the simplest possible market, the typical sort of antitrust approaches, let's just look at what happens when price change, what kinds of applications do people substitute to, right? And so here the kind of core challenges, prices are just fixed at zero in terms of the monetary price that people pay. And so how can we think about measuring relevant substitutes, okay? And so the approach that I take in this paper is that one unique aspect I think of digital goods is that these kinds of experiments are substantially easier to run relative to other markets, for instance, cereal or something that IO economy is typically study. And so here we can think of, you know, this experimentally generated unavailability is going to allow us to look at relevant substitutes by saying how people substitute during the restriction as well as allow us to pick up on sort of more dynamic elements of demand, okay? Now, you can map this back to kind of a standard price-based framework, right? So if you look at some of the theoretical literature on media markets in IO or some of the sort of legal scholarship thinking about these specific kinds of markets, you know, people have posited that we can think of various sort of price on people attention that these firms can set, right? So the sort of most prominent one is thinking about the advertising load, right? So Facebook and, you know, if it has, you know, more market power can set a higher advertising load when people go on the platform. And so at least within the sort of econ theory space this has been the most popular modeling tool. If you look at the legal scholarship that kind of have these attention costs as being broader where we're thinking about product quality, privacy concerns. And so here I'm not going to directly sort of use these measures, but you can kind of think of the product unavailability variation as kind of asking, you know, if you thought this was the relevant price, the experiments kind of simulating the thought experiment where, you know, what happens if we take these to their choke value, right? What if Facebook sort of filled all of their newsfeed with just odds, right? And we're measuring how do people substitute at that kind of extreme part of the demand curve, okay? And what that's going to lead to is to be, we're going to get a relatively conservative measure for kind of how are people substituting, right? Because if we don't observe, for instance, substitution from a Facebook to a YouTube at this choke price, we likely wouldn't expect it from a smaller kind of advertising increase, okay? And again, here I'm going to be studying social media applications. I think we can kind of think of these kind of experiments as being more broadly useful across a large range of digital goods, right? Because again, you have your version of Facebook on your phone, I have my version of Facebook on my phone and you can kind of experimentally remove mine with while keeping yours, okay? Okay, so that's the broad overview. Let me get into the study details unless I don't see any questions in the chat. So I'll keep going. Okay, so what is the experiment? So we're going to recruit roughly 400 people from a number of university lab pools. There's going to be like four people we recruit from Facebook ads, so they're predominantly from these university lab pools. This is going to lead to demographic selection, right? So we're going to have relatively young participants. Now in this kind of environment, there's, it's hard to get kind of the perfect sample. And so I think we don't have a demographically representative sort of population but relative to this demographic, we kind of have an unbiased measure of kind of like the intensity of use of these applications should be broadly representative of people within these demographics. But that limits a bit sort of how much you can interpret the kind of diversion ratios that I get to, for instance, some of these ongoing cases, but I think most of the kind of the more qualitative conclusions for this study don't too strongly depend on this. In terms of incentives, people get a baseline $50 for just completing the study. So kind of keeping the application installed, doing the surveys. There's going to be a secondary restriction, which they can earn additional money for, which is not going to be in the experimental analysis, but kind of helps with the incentives for this participants. I'm going to pull with sort of an earlier data collection round that was a little bit smaller for some of my reduced form results. In terms of what are we going to do? So this software is an Android product control software, which I'll tell you a little bit about in a couple of slides, and a very simple custom Chrome extension, which I'll also tell you about a little bit of some details on in a couple of slides. Like I said, the application restrictions are going to be either YouTube or Instagram, and the timeline for the study is going to be as follows. So the study starts where people set up a calendar or they sort of set up their software, meet us on Zoom, and we kind of go through any questions they have about the study, make sure the software is all set up properly. And this is important for kind of building trust with the participants, because this is a very privacy invasive kind of data collection. So the study period runs for five weeks. There's a baseline period of one week with no restriction. Now importantly, participants when joining the study had no knowledge of which particular applications would be restricted and in particular, whether or not it would be YouTube or Instagram, they just knew that they broadly would have some social media, entertainment application restricted. And so at the end of this week, one hour before the end of this week they're informed via SMS, you're going to have no YouTube for one week or no Instagram for one week or whatever. The restriction period runs from one to two weeks where again, they're either gonna have no YouTube, no Instagram or be in the control group. And there's a, after that there's a post restriction period of two to three weeks without any restrictions, okay? So how does the sort of flow of users through the study look like? So there's a lot of coordination across, you know, lab pools. So we have 553 people which express interest, you know, there's some time lag in between, you know, for some of the lab pools when we originally reach out to people and eventually can actually get this data collection going. So we end up with 410 people who show up on the Zoom and complete the survey. Now I learned that, you know, Android software works very widely, like some Android phones don't work particularly well with our software. So we lose a decent number of people here because they had older versions of Android phones that didn't work with the software. So we end up with kind of the core sample of 389 participants. Now of these 389 participants, remember there's kind of no explicit instructions that you have to use Instagram or have to use YouTube. And so here, you know, we're gonna condition for our main experimental intervention on the set of users that either used Instagram or YouTube. We're going to block randomize based on their baseline usage in Instagram and YouTube, into the Instagram group, the YouTube group and the control group. The block randomization kind of ensures that we have sort of equal usage, at least in the baseline of the restricted applications but also sort of the other core set of social media applications. Following this, we kind of bend people into one or two week groups using uniform sampling within group. And then the final thing to note is, you know, the extent to which there's differential attrition based on treatment status, we don't really observe this is the case, right? So only four people dropped from the Instagram group, two people dropped from the YouTube group, two dropped from the control group. And in most cases, this is from either survey fatigue or a couple of people decided to buy iPhones in the middle of the study, which didn't doesn't work well with the data collection. So most of the kind of attrition here is front loaded. And so this means we don't, I'm not going to worry about it too much in the experimental analysis, okay? Cool, so let me tell you a little bit about the time data collection before we jump into an overview of the results. So for the phones, you know, we're effectively going to piggyback off the fact that there's this market of applications for, you know, parents trying to monitor what their kids are doing on their phones, right? And so we're going to use this print control software called screen time labs for phone time, right? And so what the way these apps work is effectively the kind of one, they only work on Android. So that's why we kind of only recruit Android bargespence because the app basically sits on top of the OS and just pings the screen every few seconds and monitors which app is on. Can I stop you for a second? There's a question clarifying question, which I think is relevant. Do you speak people from using Instagram, YouTube on other devices? Yeah, good question. I did not, I was supposed to say talk about that, but I didn't. So no, so this is part of the reason for the Chrome extension. So if you look at the baseline data, so especially for Instagram, basically no one uses it on the computer. For YouTube, there is a large amount of usage on the computer, but I can precisely measure the substitution patterns toward the computer. And so to be very clear, so one thing that we were very explicit with people in the setup was that we're going to have you install this Chrome extension. There's going to be no restrictions on the computer. So you should not have the need to kind of deviate your usage on the computer during the restriction. If we block YouTube, you should be more than free to use it on other devices. And so we can measure both the substitution during the restriction period towards the other devices, but also there's a tiny survey where we ask people, how much time off the phone do you think that you spent on YouTube or Instagram during that week? And as a sneak preview of those results, so for the survey based measures, we don't really seem to find that people think that they're substituting. For the Chrome extension, we observed some small amount of substitution that's a little bit larger for Instagram, but still dwarfs the amount of time that people spend. Thanks for asking, that's an important question. Okay, so to go back to the software. So this software collects minute by minute phone usage of all applications. There's some interesting scaling challenge of getting this to actually work with 400 people, but you can read about that in the paper. So beyond just collecting data, we're going to have this minute by minute usage of all the applications on the phone. For the reduced form measures, we're going to manually collect the application category that the app is assigned in the Google Play Store. And so every application you go into Google Play Store, there's some categories up top there. So we're going to manually collect those for all the applications that we observe people using. In terms of restrictions, again, this is pretty robust, because it's kind of the reason why these parental control applications exist.  So you can't go to your browser and go to Instagram.com or download some secondary Instagram app. We can kind of restrict any request to the Instagram domain. Because it's parental control software, again, it's really hard for them to get this off their phone without letting us know about it. And so we don't have to worry too much about evasion on the phone, okay? That's where this question, one key concern is, we're only blocking this on the phone. How do we know people aren't substituting to the computer? So we build this really simple Chrome extension that kind of sits in their Chrome web browser. 90% of participants install it. It has some privacy things that only monitors time spent on a core set of websites that we care about and kind of gets a measure of how much time people are spending, okay? There's a bunch of surveys. You can read about them all in the paper if you're interested, but I'll bring them up as relevant during the talk, okay? Are there any questions about the data collection before I go forward? I don't see any other ones in the chat, but no, okay. Let me keep going. So I'm gonna give you a broad overview of kind of what we find in the reduced form experimental results. So the empirical specification chair pretty straightforward. It's a block randomized experiments or effectively I'm gonna be giving you estimates of the average treatment effect kind of controlling for this block and baseline usage. Now, the first thing we wanna look at is during the restriction period broadly, how do people substitute, right? And so we wanted to take seriously this notion that there's some cross category substitution. And so the measure of category that we use is the category in the Google Play Store, right? So we're gonna measure substitution across the most prominent categories during the restriction period. And so the first thing we observe is that in both treatments, there's a substantial reduction on category time, right? So if you look at the bundle of usage on social, that reduces in the Instagram treatment versus Instagram control. If we look at the bundle of usage on entertainment, that's lower in the treatment versus control. Now, perhaps the more interesting question is, what happens to the non-YouTube entertainment time or the non-Instagram social time, right? So in the YouTube case, we don't observe much substitution towards these other applications within the own category. For Instagram, we observe an increase in the usage of other social applications. And I should note here, kind of there's a skewness in the data, so I'm mainly reporting the log specifications, but you can see the whole things in the paper. Now, the more interesting question is kind of, is there any evidence for this cross category substitution, right? This notion that we should think of an application like YouTube as being substitutable for an application like Instagram, okay? And so in the YouTube treatment, we do observe kind of this evidence that there's an increase in time spent on social applications, which does mean at least one direction of this seems to be true. When we look at substitution in the Instagram case, it's a little bit more muted, right? So we observe a positive, but statistically insignificant substitution towards entertainment. We observe some marginal substitution towards communication, but most of the sort of substitution in the Instagram case is within the social category. Now, these sort of seemingly asymmetric substitution patterns may be a little bit puzzling at first, but you can look at this survey measure that I kind of elicited at the beginning of the study, right? So I asked people, how do you perceive that you use these different applications, right? Is it broadly for kind of entertainment purposes? Is it for social purposes? So keeping up with your friends? Is it for direct communication, right? So like directly messaging your friends? Is it for getting information is sort of broadly defined as not just getting news, but potentially some kind of niche informational content like economics papers. And so what you find is that, in the YouTube case, people are predominantly using this for entertainment, but if you look at sort of how people perceive they use these different applications which are in the social category, you actually have that there's wide heterogeneity, right? So TikTok is heavily viewed as entertainment. Instagram is viewed a lot as an entertainment application, but sort of distributed across these different activities and similar with Facebook, right? And so it's not too surprising that these product categories which are effectively based on sort of functional product characteristics don't seem to fully capture the extent of substitution, right? And so this is kind of important when we're thinking about these sort of debates with respect to these social media applications is that getting this kind of revealed preference measure of substitution is really important. We can't just kind of look at the product characteristics because we don't have prices, okay? A final couple of things in this. I'm sorry, can I just ask a clarifying question? When you say it's 0.1% increase, is it 15? So it's the time spent on social which increases by 15.1% but do you have any idea of how much of a, what percentage that makes of a time which people used to spend on YouTube because that would be the relevant issue I think in some sense. Yeah, so that'll be, so in this measure, right? We're just looking at the sort of time percentage increase over the baseline. When I get to the diversion ratios that's effectively exactly what that's going to measure. How much of the time from YouTube is sort of divvied up into these other applications? Okay, thanks. Yep. Okay, and so two final things to note here. You know, again, you could use, you could sort of think about defining a really conservative relevant market where you kind of think about the set of applications that people substitute to and you would get sort of a qualitatively different measure of market concentration using this multi-category definition versus single category. Finally, this was brought up before but just to rehash, you know. So if you look at these numbers, you know these are non-zero but they're not 100%. A large amount of substitution also goes off the phone, right? And so only a small amount of it kind of from the Chrome extension surveys is substitution towards the other applications in terms of what are the non-digital activities that people substitute to? I don't have precise enough measures of that to kind of speak to that but there is a sort of substitution towards a broad number of sort of off devices. Okay. So that's the brief overview of the reduced form restriction period substitution results. Let me give you a broad overview of kind of how people substitute after the restriction period and I'm mainly going to use this to kind of motivate the demand model that I'm going to estimate, okay? So this is just the sort of moving average plot of the time spent on the sort of restricted applications in the Instagram group and the YouTube group, right? So this black line here is the control group. The red line is the one week group in Instagram case and the blue is the sort of two week restriction group similar here for YouTube. You stare at these and you say, wow, the two week restriction group seems to have reduced usage relative to both the control group and the one week restriction group. You can plot this with computer time and you get a roughly similar pattern. And so I put this through the empirical specification and we're going to find that there's a five minute per day reduction in the Instagram two week group. Here I'm looking at levels instead of logs and so the difference between the two kind of comes from the fact that if you measure quantile treatment effects like all this is basically coming from the heaviest user of the application which is kind of consistent with what our intuition would say in this case, right? So if I'm a moderate user of this application doesn't seem like the restriction itself is leading to persistent reductions but if I'm a really heavy user of this application, you know, this two week restriction sort of temporarily leads to a reduction in my usage, you can match this with the qualitative data on how I, like there's an open ended survey response after the study and you can kind of see the people that are very intense users or are affected by this two week restriction. Now the natural question is, okay so I only really observe people for two weeks after that how long does it persist? So I send a survey one month after, sorry, one month after and, you know this is subject to the sort of, you know it's not as pristine data as the one during our collecting the study because kind of it's based on people's time perceptions but at least people perceive that they're spending less time on Instagram even one month after the study relative to before the study, right? So there's some evidence for kind of a persistent reduction usage for this group. Now, the second thing is that, you know for YouTube, you don't observe as strong in effect, right? So you see a negative point estimate but it's statistically insignificant but we can kind of look at, you know whether there's other dimension of persistence here, right? And so one cool thing the data collection enables is that I can actually see, you know when people install new applications, right? So I can actually measure, you know did people actively seek out new applications during the restriction period and persist to use them? And in the case of YouTube I find that both, you know people spend time on newly installed applications during the restriction period and they persist to use them, okay? And so I'm going to take both these results kind of reduce form evidence for kind of broadly consumer inertia that's going to drive the usage of these applications. And so what I want to do now is I want to get sort of concrete estimates of diversion between these applications. I want to sort of quantify the role of this inertia place and then you use these estimates of diversion to kind of do a simple relevant market test to kind of see how broad we should define the market. Okay? So what does the demand model look like? So the way I'm going to think about this is, you know people are making, there's kind of a panel of discrete choices, right? And every time interval I make some choice of which applications to use. So for computational purposes I sort of put this at a five minute interval. And so choice is going to be the application with the plurality of time in the interval. Now I'm going to think broadly about modeling demand and in particular the dynamic aspect of this kind of consistent with the state dependent demand estimation literature that's mainly used in quantitative marketing where I'm going to have roughly two assumptions. The first is that, you know only current period utility is going to matter, right? Effectively people are going to be myopic they're not going to sort of be thinking how does my current usage impact my future usage? But this sort of inertia is going to make a difference, right? Where I'm going to have that when I make my choice today some factor that's going to come from my past usage that's going to factor into my current period utility. Right? And so the typical identification challenge I'm going to face here is how do we kind of disentangle role that past usage from kind of the more intrinsic preferences for the products? Okay. And so here I'm going to restrict focus to kind of a prominent set of social media applications. So Facebook, Instagram, Snapchat, Reddit, Twitter, TikTok and YouTube. And the data I'm going to use is going to be this panel of phone application you should throughout the study. One important thing to note is that I'm going to try to leverage the fact that I actually observe for most of the time the core set of applications that people have installed, right? So I'm going to have kind of a subject specific choice set on any given point in time based on the set of installed application. Okay. And so I'm going to think for a consumer I in application J in market K where a market here is going to be a distinct choice set. So that's going to be the sort of set of installed applications minus any experiment to the restricted applications in some time period T. I'm going to have that first kind of this longer term inertia comes from some continuous stock of usage in the past two weeks. There's going to also be here because of the discrete choice formulation you might think there's some shorter term inertia which I'm going to use more of a nuisance, right? And so the idea here is, you know if I am spending time on YouTube now it might depend on the fact that if I used it five minutes ago I might be more likely to use it now, right? And so we might think that especially in the shorter term there's going to be some diminishing returns. And so I'm going to have this enter both linearly and quadratically to take care of any potential satiation effects. Now, the next thing is, you know the sort of intrinsic preferences that people have for these different applications, right? So I'm going to have the kind of standard product fixed effect here. The one cool thing is that, you know because I'm running this experimental study and I have this sort of more subjective data I can also kind of control for these stated activities, right? So I can make it so that people get different utility using Facebook for entertainment purposes versus for broadly social purposes. And so I'm going to build that directly into the utility function. Finally, I'm going to have from their survey some measure of the self reported number of connections on that application, right? So this is going to be the number of people that they follow, right? Some measure of kind of how much content for instance they can potentially get here. Okay. So we're going to have the sort of standard type one extreme value error. And then you notice these QI subscripts on everything, right? And so the way I'm thinking about preference heterogeneity here is a most concerned that kind of power users of these different applications are going to have different intrinsic preferences for these applications relative to none. And so this clustering procedure effectively allows me to sort of factor out these more intense users. And so I'm going to estimate the model by using the baseline data. I'm going to assign types to participants using k-means. And then for each type, I'm going to estimate a loge of model using maximum likelihood estimation, right? Okay, and the final aspect of the model is if you stare closely at this, there really isn't that much that kind of depends on the time of day that people use this, right? But if you look at, you know, this is a, oh, sorry, a heat map of usage throughout the day, throughout the week, this is the hour of the day, this is the day of the week. You can kind of see here this yellow is the most intense usage, this blue is the least intense usage. And so, you know, you wouldn't think that, you know, this is because, you know, Facebook for instance has more utility during the day versus at night, but rather the outside option for using Facebook at night is rather high because sleep is a very valuable thing. And at lunch, you're not really doing much else. And so, yeah. So I'm going to have this as the outside option is going to be time-varying. Okay. Cool. So that's the demand model, brief discussion of it. A little bit late on time. So there's two types of inertia. I'm mainly interested in this long-term one.  I'll talk about this a little bit more on the, when I go through the counterfactuals. The identification here, I think it's pretty straightforward where I'm going to use this experiment to induce exogenous variation in this habit stock. You know, the sort of identification assumption there is that the experiment really only influences the stock of usage. You know, I'm kind of helped by the fact that I can observe this consideration set. And in terms of incorporating preference heterogeneity, so I do the somewhat normal thing of having type-specific coefficients. And I can control for the subjective usage of these applications. Okay. So in terms of estimates, you know, I find, you know, that this was a sort of positive and statistically significant sort of role of this long-term usage stock for each of the types. I see, you know, a negative coefficient in the quadratic for the short term, which is kind of consistent with these satiation effects. And overall seem to get reasonable estimates. I do some model validation by sort of validating that in the baseline, you know, the model predicts at least the aggregate market shares in the non-restriction period, but also that, you know, during the restriction period, the model predicts the substitution towards the non-restricted applications. Again, the model is estimated over this data, but these are the validation exercises that I use. What do I get from this? So to the question from before, so I'm going to get the sort of second choice diversion ratio between these different applications, right? So I'm going to ask, what would happen if we removed YouTube from the choice sets of people? So what fraction of the time that was spent on YouTube is substituted towards the other applications and the outside option, okay? Yeah, you should try to finish up in five minutes more or less. Okay, yeah, I'm almost done, I think. So, so yeah, so the main counterfactual we ask is kind of what happens if this long-term inertia channel shut down? And so again, I think there's kind of two interpretations that I want to give of this kind of factual. The first one is that a lot of cases here we're kind of thinking about either a merger or we're trying to measure diversion between an application which is kind of a very prominent application and sort of potentially a smaller application. And so if we wanted to really think about kind of the intrinsic substitutability between these applications, we do care about the diversion ratios in the baseline, but kind of by parsing out inertia, we can actually get a more direct measurement of substitutability of these applications that isn't as dependent on market conditions. The kind of second interpretation of this is, one of the behavioral mechanisms are things that are loosely consistent with what we might think of as addiction. And so there's a whole separate debate which I haven't discussed about potentially some design features that applications can actually contribute to this inertia. So you can kind of think of this as the diversion ratios that I measure here kind of a limit case on how these policies aimed at kind of curbing these platform features would kind of shift around the diversion towards these applications, okay? And so what do I find? So first I find that the overall usage of these considered applications drops by nearly 30% when you kind of shut off this inertia channel consistent with at least intuition, TikTok seems to be the most impacted application at least colloquially people perceive this as being kind of the most addictive application. And it's not just diversion towards larger application drops, but it's not a uniform reduction, right? Some applications which are kind of more niche applications within the set of applications I use actually have increased diversion, okay? So now we have these estimates of diversion with inertia without inertia, what do we do with them? So to conclude, I'm gonna apply them to a really simple market, relevant market definition, right? We're basically take these metrics of diversion ratios and then basically we're going to add each of the applications to the sort of relevant market as long as the sort of some of the diversion doesn't go above some threshold talent, okay? And so what are the sort of main takeaways? So my reading of these results is that kind of consistent with these reduced form estimates you can actually see that the relevant market is somewhat broadly defined. And so this is a little bit in contrast to for instance, the FTC's definition of relevant market for Facebook was only Snapchat. Here I kind of see a broader set of substitutes in both cases. The second is that this inertia channel is actually quantitatively large enough to expand this relevant market definition, right? And there's kind of two takeaways from this. The first is that you might interpret this as there's a larger set of substitutes than just using those in the baseline. The second is that you can kind of view this from the regulatory policies that actually target, this inertia channel can actually be pro-competitive, okay? And all the details and stuff for this are fleshed out a lot more in the paper, okay? So let me conclude then. So what do I do today? So I conducted a field, I told you about a field experiment where I collected data on how people spend their time. I used friendly control software to shut off access to popular mobile phone applications. I observed sort of dispersed substitution results. Some reduced form evidence for consumer inertia. I quantified this to show that it's roughly 30% of usage. And what I view as kind of the broader takeaways here. So I think the first is kind of, I hope I've shown that these particular kinds of experiments in digital markets are super useful for thinking about substitution patterns. I think there's a lot of challenges in thinking about issues in these markets, but the kind of data that we collect in the variation that we can generate is super rich, I think relative to other markets. And so this is at least one application, I think of kind of the things you can do in these spaces. The second is in terms of direct policy implications. I do think my results point to at least a suggestion that relevant markets may be broader than those that are currently positive by regulatory authorities. Definitely sort of warrants, I think further investigation based on my results. The third is that, I think at least I haven't seen this discuss too much is that these sort of debates about digital addiction policies may actually be useful from a sort of competition policy perspective in terms of sort of a tool that people have to kind of soften diversion towards these more popular applications, which is a little bit less heavy handed than some of the alternative approaches. Okay, so that's all I have. I know I went through some of the results a bit fast. Hopefully all the details are in the paper and happy to chat more about them now. And I'm looking forward to the discussion from Penner. So. Great, thanks Guy. And so next we'll have a five minute discussion by Penner Yildrem. Penner, floor is yours. All right, so thank you so much for having me. This was a great paper that I've seen in a few conferences before and I'm really excited to, I think I'm really excited about the implementations and applications of this particular paper. And it's fairly well done. So there isn't much yet. So I thought that I would just at a high level point out a couple of things that we might need to think about to make broader generalizations regarding the substitution in digital economy. So again, it's a really interesting well done study in a fairly understudied area. And the focus is as Guy has gone into more detail, how do people substitute time across digital platforms? And this is important for a understanding or defining relevant markets for different applications, different companies for competition and antitrust purposes. And it's also important in terms of understanding the drivers of usage, behavioral drivers, addiction and inertia. And this in this paper, there's a combination of an experimental method with follow-up survey tools that installs the software on individual devices that allows Guy to stop usage of Instagram and YouTube in particular. Then the Chrome extension and follow-up surveys and the parental restrictions or app allows him to be able to track usage of different apps as well as some time use outside of these apps. So as a result, he is able to come up with a few interesting findings. Of course, there's no perfect substitution. Perhaps this is not very surprising and there's a decline in the overall use time either Instagram or YouTube are restricted. But I think what might be fairly more interesting is the asymmetry and the degree of within and cross category substitution for these particular two different particular apps. Diversion from Instagram, for instance, to others, there is some diversion from Instagram to other social apps, but not at least statistically speaking, not to other apps. And there's no diversion from YouTube to other entertainment applications, which are in the same category, but there is a diversion to social apps. And he also does a number of things in the paper that he didn't really go over in terms of looking into diversion to new apps and fairly more prominent apps, apps within the same family, for instance, Facebook meta-owned apps. And he does find some substitution towards the more prominent apps as opposed to maybe the apps that are on the long tail. He also finds some evidence of persistent declines in activity with these interruptions, which actually surprised me given two weeks as a fairly short amount of time for an interruption. And then he tries to disentangle how much of this is due to inertia versus other factors. So just at a high level, I thought that there are three things that we could think about maybe as considerations, factors of considerations in trying to make generalizations from this. And the first thing or at least the first part is going to relate to experimental design. Then I'm going to think about some of the responses from the firms, what might be the responses that are happening at the same time as you start creating these interventions that might make it harder to interpret those substitution ratios. And at the same time, we might also think about possibly what to make up the results, the interpretation will be the last thing. So in the experimental group, it seems like in the paper, guys spanning a lot of time trying to make sure or address the issue of selection into the experiment with the concern that maybe there are people who are generally concerned about their time use and their self-selecting. That's something that seems to be at least not a concern, that doesn't seem to be a concern, but I'm a little more concerned about recruitment of college subjects for two particular reasons that don't seem to get too much attention in the paper. One is that students in general, compared to the other rest of the population, they seem to have better or more connected outside options. If I'm a student, I probably live with my, all the people that I communicate over social media, I probably live with them or I go to class with them. So I think that the outside options, especially for social apps, might be a lot stronger for a student population compared to the rest of the population. And then these younger cohorts may also substitute less to paid advertising. So this is going to come in, for instance, when we do not find within category substitution moving from YouTube to say Netflix. Netflix is a paid app. And of course that creates a different level of friction in terms of substitution for a younger cohort, maybe compared to an older cohort. So I wanted to, first of all, just bring up this issue of how much can we generalize from this group because over again, they're the level of connectivity that they have as well as the paid alternatives. A second important thing is that many of these goods that we are looking at at least the digital goods, they tend to be network goods. And when I say network goods, I mean that in a number of different ways that brings challenges and interpretation. What does it exactly mean to limit a single user's access? You can't really just think of this intervention in term as I'm blocking your access. The moment I block your access, these user-generated content platforms, it also implies that some of the content on the platform is also changing. I'm reducing the content, I'm changing the content, I'm changing other people's communication as well. So A, multiple things are changing. B, maybe not so much of a problem, perhaps if these individuals were randomly participating from a campus, but in a larger experiment where people tend to communicate with each other, there might be spill overs to the control group in terms of the intervention. And three, if you think about network effects that might also come up with maybe a bigger experiment of similar type, how would we interpret those effects? Again, a number of things that are unique to digital goods with network effects or a network structure as well as content generation platforms that make it hard to interpret just these access, shutting off access type of experiments. And since Guy is making some interpretations or at least suggesting that there could be some elements that we can take away from this study for those bigger exercises, I wanted to also highlight that. Another thing, I didn't see much of a discussion of this on the paper, maybe I missed it, but of course the moment I decide to go off Facebook or I don't visit, let's say Facebook for five days, the firm starts to respond to that, right? They might be sending emails to me back saying, hey, we miss you or there's this friend of yours who just posted something, is there anything else that's changing in the way that these firms are trying to communicate and trying to lure back users into the platforms? Or maybe they are just simply, this may be also means that firms are blocking the regular communications that they have. I wanted to get more clarity, but also try to understand how those effects might change the diversion for the groups whose access was limited. Then finally, I know I'm taking a lot of time, but what exactly are we observing? So this is related to maybe the interpretations coming from these diversion ratios. One is, we are of course, possibly talking about substitution across devices and Guy is doing, he's trying to put a lot of effort to try to capture that through surveys and the Chrome extension and everything, but I in a venue look into, for instance, table one in the paper, it's looking into some of the summaries of how people use different apps on different devices. For instance, Snapchat, there's a zero mean. That means that nobody is using Snapchat on a desktop device. 9.35 minutes on average for mobile. I look at TikTok the same way, 50 minutes on mobile devices, but one minute perhaps on desktop. Whereas when you look into other apps, I think YouTube, Buzz One, Instagram, somewhat also similar, they're much easier to use perhaps or much regularly used across devices. So when you talk about, for instance, some of the substitution, you're not only thinking about substitution for a particular app to another app, it's in the same device, but you're thinking about also adding maybe an additional friction or additional cost of having to change devices that you don't normally regularly use. So I wanted to understand also how much does the substitution across devices add to the diversion?  Okay, last minute, last thing. And thinking about time or attention as a relevant dimension of substitution. Two minutes on YouTube in terms of advertising value is not the same as two minutes on another platform, Instagram or Facebook. How do we interpret these relevant to especially advertising markets and advertising revenue? And finally, the one thing of course, we don't know just because people are switching across these platforms. We don't necessarily, we cannot say much about welfare effects. That's the one last thing. The great paper, I really enjoyed reading it and I look forward to seeing more of the results. Thank you.