and welcome to the 5.30 p.m. to 6 p.m. session of the 2019 Open Simulator Community Conference. In this session, we're happy to introduce a presentation called Art Gallery, a plug-and-play OER immersive activity. Our speaker is Bethany Winslow. Bethany Winslow is a volunteer with Community Virtual Library and an instructional designer at San Jose State University. In her years in virtual world, she has found that much more is needed to help educators understand how vibrant is the history of virtual worlds in education. Today, she'll talk to us about creating an immersive virtual event. And please check out the website found at conference.opensimulator.org for speaker bios, details of sessions, and the full schedule of events. This session is being live-streamed and recorded, so if you have any questions or comments during the session, you may send tweets to appopensimcc with the hashtag OSCC19. Welcome, everyone, and let's begin the session. Well, thank you very much, Scott, for a very nice introduction, and thank you, everybody, for attending this presentation. I appreciate it. I want to tell you a little bit about some of the content I've created in the hopes that it'll help other people to be able to host other fun, immersive events. Creating an in-world event, no matter how large or small, is a really challenging endeavor, I think, for most of us. But for busy educators in particular, the prospect of having something else that needs to, you know, something else that needs to be created, yet another explanation or tutorial that needs to be developed, these are just one more little hurdle that gets in the way of wider adoption of virtual worlds in teaching. And so, as an instructional designer working in higher education, I know firsthand how easily instructors can feel discouraged by those little hurdles. But I also know from experience how helping facilitate a little bit of success can really help encourage and help people to persevere. And I think we know the same thing for our students, too. Indeed, the same can be said, I think, for anybody. The Community Virtual Libraries Hypergrid Resources Project is something that I took on because I wanted to support educators to navigate and explore the wider metaverse of OpenSim. We want to help educators find their tribe in OpenSim, and we have hypergrid portals that connect with persistent virtual worlds, and information about those worlds and the people behind them. And our Director Valerie Hill often remarks that we've really got to stop reinventing the wheel and working alone in our little silos. That's the default mode I see in the physical universities. And I think that solitary habit kind of dies hard in in world, too. An educator who's new to a virtual world and new to the concept of hypergridding. This whole metaverse concept and open world can be very intimidating, and people can be not knowing where to start. Not everybody has the ability or the inclination right away to build in their own world. And one of the first questions that faculty have for me when I talk about any new concept or show them a new technology is to ask me for an example. So one of the things that I wanted to start providing at the Hypergrid Resources Library are a few things that sort of scaffold the use of the virtual environment with some pre-built areas and, you know, instructional content and immersive lesson examples. My latest project is, you know, an immersive art activity, but that's really just one piece of a larger picture that I want to kind of sketch out for you before I tell you about the immersive art activity and the content that I have for you guys. So, you know, I got the idea for this activity after I attended an enrolled event where students had presented their work with slides, just like we're doing here, which is awesome. But it occurred to me with some tutorials. It might not be that hard to go once that further and have students do something beyond maybe exploring a region. And I know that doing something creative is highly engaging and that embarking on that first step of making and tinkering really supports the community of practice that we've been talking about. And I really think this activity can help your participants to get invested in what they're doing on your region. But again, faculty shouldn't have to reinvent the wheel. I wanted to create content that anyone could use because this is the sort of support that faculty need. And this isn't just my own observation. You know, research has identified that the lack of institutional support and training is one of the reasons virtual worlds didn't stick with educators despite a lot of early adoption. And I've also heard this directly from educators I've met in world and in world events. In fact, Greg Perrier from Northern Virginia Community College has shared extensive thoughts and ideas on how to address this challenge with his presentation materials that are available at the Science Circles website. And in fact, I really want to continue to develop resources at the library that will help bring Greg's vision for educators orientation to teach in virtual worlds kind of come to life at the Hypergrid Resources Center. So that's a bigger sort of long-term plan that I've got. And it really kind of ties in nicely with what open educational resources that we're kind of talking about here. And ultimately, we want to have that Hypergrid Resources Library be that hub that helps coordinate an educator-specific orientation of virtual worlds and the lessons that we learn here. And so I'd like to support educators who want to have a safe space to do their work and be able to practice presenting in worlds before trying it out with their students. And so that's a sort of big picture of the direction I see for the Hypergrid Resources Libraries. And so creating content that's an open educational resource and using OpenSend to do it, for me it's really about two things here. It's leverage and sustainability. And at CVL, we really promote collaboration, sharing, because crowdsourcing, the development of virtual environments for teaching and learning, it just seems to make good sense. I think it's a worthy long-term sustainable activity. And the alternative is that we might be just at the mercy of others who are selling us solutions. And for educators just getting started who might not be ready to have a region of their own, where else would they go first? That has been a library. So we do have portals that connect to other people's worlds. And we have public spaces where educators can meet with a colleague to try out their hand at presenting in worlds. At our kindly location, we have some additional beautiful meeting or discussion areas and settings that I think really are conducive to learning. And we really want to go beyond providing attractive areas where they can meet with students or colleagues or practice delivering a presentation. So I want to offer additional resources that show an educator what an immersive activity might look like in a world and actually give them content they can copy and put into their learning management system too. So what I've been working on are several different examples. And all of these are marked clearly as shared via Creative Commons Licensing. So anybody can copy and use what they find at our location here. And perhaps they'll just give somebody an idea or two. But visitors to the HyperGrid Resources Library on Kitely Oradicon will find an example lesson that integrates hypergrid exploration. So I have on the screenshot here some example lessons, a lesson of a hypergrid with instructions. There's directions on how to hypergrid do a local teleport hot for the students with some screenshots. And I have inquiry questions that are pre-written. So an instructor could actually look at this and see how this could be useful. And they could modify this for a real lesson. The note card giver provides links to Google Docs that an instructor can use as is or modify. And finally, one of the long-term goals of CBL I kind of think should become maybe a GitHub type of repository of various open educational resources as well as maybe OAR files. Now I don't know the technical side of how to do that, but it seems to me to make a whole lot of sense that this is very much a library and archival kind of project. But I want to tell you about the immersive art activity. So this immersive art activity, I call it the art gallery contribution. After completing a survey about an educational event, when it asked for a recommendation to make it the next one even better, I wondered if I might be able to help walk new participants through a process of taking a photo or a selfie somewhere in the region and then meeting at the sandbox and showing them how to add their photo to a simple prim. And I actually volunteered to help do this. And then you know it dawned on me I should just write up instructions, create some generic signage, make a couple of tutorial videos, and then just share this with anybody who wants it. I think the activity could be adapted to a number of different kind of different ways. So the event where we're going to test this out for us is our own event where is going to be a friends of the community virtual library wall. So at the event we'll be asking people to take a selfie anywhere on our region 52. And someone could you know adapt this to some artistic or landscape photography art gallery at their own event or whatnot. I think there's a few things though you need to do to make this work. First of all, you've got to provide orientation to your specific location and you know how to access the tutorials. But my generic content, tutorial content should really help. But I'd still recommend meeting your participants in world. I would recommend to instructors to encourage collaborative exploration, maybe in teams or small groups. And I'd even provide, you know, recommended teleport locations on Google Doc or something. And you could put that sort of thing in world on a note card, or it could be something you post in your learning management system. But my instructions are generic, but there's a tutorial video on how to use the search and your mini map and the world map and how to just click to teleport and explore. Ideally, I think the event would take place over a long enough period of time to allow people to you know, to log in and out multiple times over several days. Because you'll want to meet at the stand box to help students with the resin of a prim and helping out there. But I really tried to make the tutorials be enough to help your students on their own. And finally, you'll want to have some kind of little concluding activity like a mixer where you add their artwork to a gallery wall. And I'd also recommend a prize for participation. That's a fun incentive. And it's not essential, but giving away some goody is another engagement device that helps participants feel invested in the activity. So if I were an instructor, I'd probably also give extra credit for this kind of thing too. So I'll show you a couple other photos of what you'll find at the Hypergrid Resources Library. So first of all, I have to tell you guys, I've learned everything I've learned here, I've learned by doing and it has not always been easy. This kind of thing has been part of my own journey as a learner and building in worlds. For example, I had a terrible moment of panic this week when I went to Kitely with my OSCC avatar and found my horror that I could not build in the 52 sandbox. I know you guys can appreciate that. But that kind of panic, it worked with another avatar from Avocons. I couldn't figure it out right away why it wasn't working. And I went to the welcome zone and then noticed the sign welcoming me to join Communication Arts Group, and then I had an epiphany. And indeed joining the group is what enabled me to use the sandbox and have a building permission. So of course, I completely amended my instructions. But seriously, this is a kind of hurdle that can derail educators. So I don't mind going through this frustration and trying to document everything I'm learning because it can become a resource for anyone else in the future. So I've created some signs with embedded note cards for students. And these are at the exhibition booth 10. And these are full permissions. So you should be able to, and I tested it, you should be able to take copies. Let's see here, I'll show you. In terms of the tutorial, I am using Google Drive and Google Doc so that anybody can take a copy and sort of edit themselves. But the idea here is making something that's generically useful enough for many people so that anyone can participate, even asynchronously, if they can't come to your live event, that they can still participate. I've created some generic signage with directions on how to find and get for sandbox. And the signs have some note cards that have links to the tutorials and screenshots. They're on the Google Docs and video tutorials that we have on our YouTube was YouTube. So I don't think I really have time to show you a video, but you can get copies of these resources and check out the videos and tutorials at expo booth 10. And I would love your feedback. For example, if you see any problems or have any suggestions, I can integrate your suggestions right onto the Google Docs. It's really nice to be able to make this kind of edit and it automatically means that everyone using this resource has the latest and best instruction. So, hooray for crowdsourcing and for leverage, right? I did intend to group all this content together and into a single folder, but I haven't done that yet because, honestly, I figure out stuff as I go and I haven't figured that out yet. So we do encourage you to please join us to help us test the useful moves of these tutorials and become a friend of the Community Virtual Library. We have a holiday mixer on Friday the 20th from 3 to 4.30. And if you can't make it at that time, we hope you'll still come another time. And because you can follow the instructions there, share your photos with me and I'll add it to the Friends of the Library wall. What we're going to do is meet at the Hypergroup Resources Library at Cookie 2 on the Kite and Grid. And to kick it off, we're going to, I'll recommend some places to visit on our region. And you can use this or the teleporters on your own and kind of look around. We'll take some selfies or photos and gather at the Cookie 2 sandbox and make our objects and then go back to the gallery, which is right next door to the main library building. And we'll be adding people's portraits to the Friends of the Library wall. So, you know, I think that's about it. Thank you very much to everyone who's come to this presentation. I really do hope that you find some of this content helpful or that you can see the value for an educator who doesn't have the expertise that I think most of this audience actually has. And I think we might have a minute or two for any questions or comments. But if your educational location is not connected to our portals, please contact me. I really do want to connect to your location. And like Eileen, I have a Google Doc. I think this was a great idea, the Google Doc. I totally did this. Let me copy this in here. So if you guys have any suggestions, just post it on that Google Doc because I can't keep up with all the virtual worlds that there are. And of course, if I've ever linked to your hyperverted location and you don't want it, please let me know that. I'll take the portal down. But oh, thank you, Eileen. I wish I had more faculty at San Jose State who are doing stuff in world. And that's why I'm delighted to help you or your students or your faculty. It's great. So if there's any questions, let me know. Thank you very much, Bethany. Yes, if anyone has any questions, please put them into the chat stream and I will read them out to Bethany and we'll take a few more. Is there any attempt to do more than put a picture on a prim, like build something? Well, that's a great question. Like I said, I learned by doing, I'm not a great builder. It was a revelation for me to be, and now I think it's easy to put a picture on a prim. But I have very little other skill. I wouldn't know how to make a script. So I am definitely looking to do more in terms of creating tutorials or help for other faculty who want to teach in world. And like I said, I want to build on Greg Perrier's vision for an educator-specific orientation. So I have some definite ideas with that. And the biggest challenge you see for new users, well, that's a great question. Eileen can speak to this at length as well. There's a learning curve. It's hard. And I really believe that it should be. And my pitch right now is I go to a lot of physical world conferences. And I, there's lots and lots of interest in VR. It's everywhere. But a lot of people I know, I meet all over the country. They've never been in a virtual environment. And I encourage them to make the virtual worlds their professional hobby, as a form of preparation. Yeah, you guys have videos going over the basics. Yeah, I should add that to the library forums. I'm going to go ahead and copy that right now. Thank you. We do need to kind of like, yeah, have things not be scattered all over the metaverse and be recreating the wheel. You know, we're reinventing the wheel all the time. Alan asks, have you asked folks if they spend a few hours a week helping teach building to teachers? No. I think there's different kind of groups kind of doing things. Everybody's kind of doing stuff on their own. And it looks like you want to ask another question, like, would I be willing to do that? I would. In fact, I've thought that I could maybe start hosting like weekly Zoom sessions. Maybe I can't do this weekly, maybe monthly Zoom sessions to get started. But I need faculty who are interested. I only know a few people at San Jose State even doing anything in the world. And they're all at Bacara and CDL. They're all my peeps in the world right now. Well, again, I appreciate you guys maybe taking a look at what I have created and what I've shared out. And then again, give me feedback. If you say, you know what, this video is too complex or it's too simple or can you make one like such and such? I'm all ears. See, I'd like to, I need to know where that curriculum is. If you, Rhiannon, if you could put that link on that Google doc, I want to kind of start being able to connect to all of that content instead of like make a curriculum you've got one. But then start kind of providing a, hey, start here first, go to this curriculum, do this for two weeks, and then come over to here. Nice. So anything you guys want to put on that Google doc? Like, you know, Eileen has a Google doc. I think we got to kind of try and work more collaboratively together. I cannot agree more. I'm definitely, I bookmarked your doc myself and I'm going to go do a deep dive. All right. Thanks, everyone. Thank you, Bethany. That was a terrific presentation and there's a total need for everything that you're doing. Now, as a reminder to our audience, you can see what's coming up at the conference schedule at conference.opensimulator.org, following this session. The next session will begin at 6pm and it's entitled Music at OSCC 2019.


Hello, my name is Richard Urban. I'm a doctoral student at the Graduate School of Library and Information Science at the University of Illinois. I'm going to talk today about the DLF-DCC Beta Sprint and Linked Open Data. Currently, many of the collections included in IMLS digital collections and content project are disconnected from the larger Linked Open Data community, even though many institutions share OAI records through the Open Archives Initiative Protocol for Metadata Harvesting. As part of our sprint, we looked at the feasibility of integrating IMLS-DCC and its partners into this larger world of Linked Data, in particular by looking at how we might reconcile some of our metadata with available Linked Data sources. I'm going to use the five criteria put forth by Tim Berners-Lee as a way to talk about the opportunities and challenges that we face. For example, all of this material is available on the web, but we know very little about what the licenses are for this metadata, including metadata that we create for the IMLS-DCC project itself. According to a 2006 study, only about 20% of OAI repositories included something called a set description, where a repository might specify what its license is. It's likely that an even smaller percentage of those indicated a license, an even smaller percentage used an open license. This seems to be a big sticking point in moving the community towards Linked Data. However, I believe that IMLS-DCC is well placed and to provide some leadership in this area, given our past guidance on metadata quality and other forms of aggregation. Currently, material is available as machine-readable structured data in a non-proprietary format, namely Dublin Core XML, either in a simple variety or as a more advanced IMLS-DCC collection description, which is based on the Dublin Core Collection Application Profile. We also republish about one million records as simple and qualified Dublin Core that have been harvested from participating repositories. For each of these records, there's an OAI ID that we've also used as an identifier in the URL pattern for our DPLA prototype. So if you know an OAI ID, you can construct a URL for items and collections. Identifiers for images that come from partners depend on the kinds of systems that they've implemented. So for example, if they've used Content DM, the resource identifier will depend on how Content DM has specified it. We also have another little tool called ThumbGrabber that will grab a thumbnail if an image is available. We've used the OAI ID to also construct the URL for those thumbnails. We're also exploring methods for sharing RDF serializations of this OAI data, either as a Sparkle endpoint or as RDFA that is built into the interface itself. A big challenge, of course, was linking our data to current sources. One of the things I found early on is that there's not much information about institutions themselves. Someone has kindly uploaded the public library survey to Freebase, so if I have information about a public library in our collection, it was pretty easy to reconcile that with available linked data information. However, other kinds of archives, libraries, and museums were not as well represented. One of the things we might do as a DPLA community is at least make sure that there's a linked data representation for each of these institutions. I was also able to extract controlled vocabulary terms from collection-level metadata and reconcile it with linked data sources. For example, our item type uses this RS of graphic materials terms, and I was able to reconcile that with the TALIS repository from the Library of Congress. Using Google Refine, I was able to align other terms through Freebase, for example, a mirror of the Library of Congress subject headings, or aligning it with non-library vocabularies like Freebase locations. One of the things, of course, that we've been saying all along is that quality metadata is important. It becomes particularly important in this process for well-controlled and well-defined terms, reconciling to linked open data is pretty trivial. But at the item level, we have a lot of uncontrolled terms and various formats of metadata that's going to make it much more challenging. More information about this is going to be in our full report that you can read as part of our submission.


We're very excited to have Greg Lewis present this really cool work on the welfare effect of consumer reviews. Why don't you take it away, Greg? Thank you.  And this is work we've been working on for a very long time. I'm excited to present it, but also it reminds me that we really should write up a paper and send it somewhere. So this is good motivation. Thank you very much for giving us that mental prompt. And so what I'm presenting is going to be a mix of older stuff, the newest stuff that we've done on this paper, and all the results that we have kind of seem to line up across many different versions, but they're from different specifications at different points because we haven't collected them to a paper. So nothing should be taken as literally like this is our final answer on a topic. That makes sense. Okay, so let me tell you what the goal of the paper is. The goal of paper is to measure how crowdsourced ratings have affected consumer demand, firm pricing, but most importantly, the distribution of welfare in the hotel industry. And the reason we are interested in that question is because we think that consumer reviews have played an important role in many markets and also illustrate this later. And we have an idea that they affect demand because there have been many papers on this topic and again, I'll refer to those papers later, but Mike Lucas worked, for example, in Yale. I'm sure many of us know. But we don't know whether this is actually doing very much to improve consumer welfare. So just to fix ideas, what are we talking about precisely? We're talking about the star ratings that you'll see, for example, on Expedia or on TripAdvisor. So there's 4.3 out of 5 next to the Shelburne, right? We now know that the Shelburne seems to be a little bit worse than the Courtyard, which is 4.4 out of 5. Does knowing that information really allow consumers to make better decisions and in so doing, improve consumer welfare? Okay, so the motivation here is that reviews are widely used in e-commerce. I started to think of all the places where I think of reviews being used, Amazon Yelp TripAdvisor, Etsy, Expedia, eBay, and it turns out, I checked this out yesterday last night. These are some of the most visited sites on the internet. So this is where people are spending a lot of the time that they spend consuming content. Obviously they spend a lot more time on Netflix, but it's only when I think about shopping, this is something they're doing. And so people are sitting there reading reviews to some extent and we really want to know is this particularly helpful? There's a lot of related literature, which I'm going to briefly mention, but for a long time people spend a lot of time thinking about how the internet was going to be important for price information. So thinking about when this would increase price competition, there was a lot of interest in the study of the so-called law of one price, that the internet would drive prices down to marginal costs. And if the answer to that was just generally no in many ways. And then people started to move on to the question of what about non-price information? I think one of the classic papers there being Jen and Leslie in 2003 had hygiene grade cards being given to restaurants and seeing does that change demand? That's not an internet example, but it was certainly sort of an important paper there. And then papers on book reviews, Mike Luka's work and other people's work on Yelp. And then more recently, Rhymes and Wolf will think about Amazon book reviews, trying to think about the effect of professional reviews of books versus amateur crowdsourced reviews of books and trying to disentangle the differential impact of those two channels. What's going to be distinct about this paper is that we're going to really think hard about wealth there. So many of the ratings papers, the Yelp papers are thinking about revenue. So basically if you get more stars, do you make more money? But here we're going to be very much interested in what this does to consumers. And in particular, we're also interested in a market in which there are going to be meaningful supply-side pricing effects. The hotel market is not a big market. Well, it's not, you know, there are many hotels in the city, but there are not, you know, a hundred or a thousand of them. And so it's going to be the case that people are going to actually adjust their prices in response to ratings. And this potentially is a channel through which welfare effects could become muted. And I'll show you that when we get to the theory. OK, so what's the thesis of the paper? This is the basic story we're going to try to tell. So hotel quality is going to evolve over time. And we're going to have a long panel to measure this. So it's going to be important to think about that. But it's partially unknown to consumers. So consumers have some information about quality, but they don't really know it for sure. Hotel ratings are going to be a signal of quality. And therefore they get effect demand. When people consume this content, read these ratings and reviews, they're going to be able to update. And so this is taking a pretty strong start actually on the role of ratings. The ratings are purely an informative channel here. Nobody's persuaded. Nobody gets high utility as a result of reading a hotel review or rating. They're just informed about which are better and worse ratings. Hotels. Third, equilibrium prices are a function of demand. And therefore there are a function of ratings. And so it's going to be important to think about supply side responses to the change in demand and use by ratings. And therefore, because there is going to be a supply side response, effects on the distribution of wealth are ambiguous. It's not just the case that consumers are all uniformly better off because of this extra information, because it's a game. And as we know in game theory, providing information can lead to everybody being worse off. And so therefore it must be measured using a structural model. There are many things that are not in this paper. And so it's worth spending maybe a bit of time highlighting them. The first is that consumers are making a single purchase decision. And firms are optimizing prices for static profits. And so we're doing everything as though it's static. And in fact, we know that consumers visit hotels repeatedly. And as they revisit them repeatedly, they may become more informed and no longer rely on ratings. We're not going to model that. And firms, of course, maybe think a little bit about the dynamic problem of, if I know that I might have repeat customers, how do I think about charging prices? We won't be doing that either. Maybe more important, I think, is sort of the sense in which we're oversimplifying the whole review process. We're really thinking about these ratings, four out of five stars. But in fact, consumers are going to read text. They're going to read photos. They may be able to improve their welfare by matching on horizontal characteristics. Tripadvisor will tell you what a romantic hotel looks like, which is the best romantic hotels. Maybe that's what you need right now. And that's a meaningful source of value, not the four to half out of five. We're not going to be looking at that either, primarily to simplify our lives. But it suggests that we're probably going to understate the welfare benefits. I mean, the third thing is missing dynamics. So we're not looking at firm side entry, investment, and exit decisions. And one of the channels by which you think reviews might matter and people have written papers about this is by giving customer voice, basically. So by giving them an opportunity to say, I'm unhappy with something, and then having firms respond to it. And this may have meaningful effects on the market. We're not going to really look at that. Entry next in the hotel market, it turns out, is not that important, perhaps, because there's not a lot of it. It's these giant capital investments. And so they're not going to be very responsive to reviews. But investment might be. And we're not going to see that at all. OK, so the modified goal, yeah. There's one question from Jack. I don't know if you want to ask it. Yeah. Can you really study the effect of the ratings without taking into account the way they enter into the recommendation algorithm of the platforms? Yeah, that's a really good question. I mean, I think you can, because what we're going to do is look at the effect on the all-up market of the major review sources. So OK, a couple of responses. So one is that much of demand is mediated through these platforms.  But it is also the true that many people will consult TripAdvisor for information and then go book somewhere else completely. And at least from this point of view of the data we're looking at, one of the advantages we have is that the data we're looking at is from STR at Smith Travel Research. And so we're looking at basically a very, very large chunk of the hotel market through all booking sources, not just through the platforms themselves. And so it is true that at least some of the consumers are being mediated through the way that the algorithm reacts to those ratings. But some of them are not. So that's one response. And I think the other thing to say to the extent that these are coexisting, we're just not going to be able to separate them. So if a large fraction of demand is being mediated through the way that TripAdvisor ranks people, we're just going to be looking at the all up effect of how a ratings point translates through the TripAdvisor algorithm all the way through to the demand. Yeah. And yeah, I think you're right. You're also stating an important concern, which is the algorithms themselves, the ranking algorithms are in many ways much more important than the ratings in terms of how consumer attention is directed in general. Okay, so what do we find? We find that hotel reviews have substantial effects on demand and consequently on prices. When we get to prices, you will see that the price effects are not that big and that's because the hotel market is not that concentrated. So there are some price effects that are not as substantial as maybe we thought Exante there would be. Okay, we find the demand effects are big. We find a rating point increases demand by about 27.8% in the final year of our data, which is 2014. I'll show you that there's an upward trend throughout the data, throughout the sample. So one of the nice things about having a panel is we can do this year by year and we can show that the effect of having higher online ratings becomes more and more pronounced year on year. There's substantial heterogeneity. So the effects are bigger for high-end hotels. The effects are bigger for independent hotels. And that sort of makes a lot of sense. You'd expect that where people care a lot about quality, as in when they're looking at five-star hotels, they really care about going to the very best such five-star hotel. And so the effects are substantial. We find that without allowing prices to adjust, consumers are about $2 a room night worse off without ratings, which is about 2% roughly in our data. And when we allow prices to adjust, consumers about 1% or $1 room night worse off without ratings. Although this is distributed over the population. So in our model, we allow for consumers to have differential preferences for quality. And the people who really want high quality are going to be much worse off. They're going to be 6% worse off in the absence of ratings because they're not going to be able to find these best hotels anymore. And yet, so these, I don't know how to think about these numbers. These numbers seem small, but realistic to me. So sort of 2%. I remember having this slightly disparating experience of giving this talk at Princeton once and enlisting people's priors before the talk as to what they thought the number would be. And they were like, yeah, about 2%. And I was like, oh, that's disappointing. It's like, okay, this is not surprising. This is sort of exactly what people think. But hopefully that means it's somewhat realistic. So why haven't we written this paper up and sent it somewhere? Well, we spent a lot of time doing two things that we probably didn't need to do, at least the first one, the second one seems important. So the first things we decided to estimate BLP using maximum likelihood and without additional instruments. And we'll probably switch to GMM pretty soon. But we did that and that took us on a long, long side journey through many, many Jacobians and Hessians. The second thing is we decided to think about capacity constraints seriously on the supply side.  And so Chris Conlon and Julie Mortimer have worked on this before, but the models that they have are not very computationally tractable. And so we've tried to work on some different ways of looking at that that are a little bit more straightforward. And so that's something I'll show you a little bit at the end. Okay, so I've got about 30 minutes, 25 minutes maybe. Oh gosh, I'm gonna say 25-ish to go through the data descriptives. I'll tell you a little bit about the causal effect of reviews. Theory instructional model and counterfactuals. I'm gonna skip more than I've said on these slides. I can see I'm gonna run out of time. Okay, so the data is from Smith Travel Research. We have a complete census of about 6,000 hotels in the Western United States. It's about 10% of the US hotel market and about 45% of the hotels report financial performance to STR. So we know all the hotels in the market. In addition, we have this financial performance data which includes revenue, prices, occupancy rates for every hotel year, month. So we're gonna be looking at the Sheraton Palo Alto in January 2014, which is 1.2 billion room nights. And so we have a lot of data here. We know that many observations but we have hundreds of thousands of observations. And we observe a set of attributes for every hotel. And we augment this data set with a panel of consumer reviews from tripadvisorxpediaandhotels.com which were the main three websites during that period for finding reviews. For some subset of the hotels in the years we have daily data on revenue and prices. So at a higher frequency. And that's gonna be important when we talk about the possibility that a hotel might sell out and trying to measure demand in the presence of capacity constraints. And then finally, last data issue. For the structural model, we're gonna use imputation procedures sometimes to complete our data set. So we're gonna match non-reporting hotels with reporting ones to get data on the full set of hotels in the market. That's important because if we only observe half the market obviously competing with 10 other firms looks very different than competing with 20 other firms. We don't wanna use our incomplete data and make completely wrong assumptions about market power. So we use the full census and then do some work to try and fill things out. Okay, so the first time I'm gonna show you is something that I might generally spend a lot of time on but I'm not gonna spend very much time on today. I'm gonna try to show you that there's a causal effect of reviews that reviews are going to matter and then I wanna move very quickly on to the structural model. Great, can I just interrupt you for there was a clarifying question from the audience. I'll just say it, which is one, are you concerned that these platforms might not display all the ratings? They might like remove some of the reviews from their platform for whatever reason. And then two, which is I think a different point, it's been shown that there is like review inflation in certain markets where reviews are going up over time kind of in a secular trend and how do you think about that? So I just- Yeah, so we're in some ways we have this very kind of agnostic and somewhat, I don't know, I wanna say ladies, I don't think ladies are the right word but approach this, which is to say, yes, there's probably a review fraud, there's probably some review inflation, there's probably some review manipulation and all of these are going on and yet consumers will show you are trusting these things more and more. So the market is generally paying a lot of attention to reviews. So I guess one thing you could be concerned about and we'll come to this right at the end is that we're gonna assume the consumers are an average rate. So when they see a 4.4 instead of a 4.1, they have some taste for the 0.3 and additional ratings that they can give out to utils. And on average, they're getting that right. They're not being systematically deceived when they think that they're gonna get more utility from a 4.4 than a 4.1. But we need that to hold on average. So individual review frauds, so small manipulations with edges are not gonna matter. And another way of interpreting the welfare stuff is the coefficients that they place in front of ratings that the sort of preferences for ratings are basically built on how much trust they have in the marketplace. What we're gonna show you is that there seems to be a fair amount of trust in this data even though it may not be perfect if that makes sense. Okay, so what are we gonna do? I'm just gonna show you that reviews matter. And I think the easiest way for me to show that to you is just to show you a bunch of OLS regressions. If I have a little bit of time I might talk about the instrumental variables approach. I won't talk at all about the natural experiment. We also have data from a natural experiment that sort of is designed to prove causality. So the month here is going to be basically a glorified loge at least until I get to the structural model. So people are going to have, there's gonna be demand, which is essentially a market share ratio. So it's the log ratio of my share as hotel J and time T to the outside good in time T where a measure of the outside good is essentially the maximum size of the market in any given month across our entire dataset. We stick in, so that dependent variable is a function of ratings. It's a function of whether there's missing ratings which we then dummy out. We add in hotel fixed effects and we add in market time period fixed effects. So San Francisco in February is a fixed effect. And so really what's going on is if we're trying to identify why ratings matter here, we're asking what is the identifying variation is the within markets ratings fluctuations, taking out a hotel fixed effect. So the same hotel we're really looking at how their ratings vary over time and how demand and co-gurries with it. And they were also taking out a bunch of seasonality at a very fine-grained level because we're looking to take out fixed effects at the hotel at the market month level. You can still of course be worried that the specification, you know, it has fixed effects but that could be endogeneity. It could be the case that ratings are correlated with marketing efforts, the contemporaneous marketing is really driving people to the hotel and then the ratings follows that. So I market aggressively that this is a great hotel. People go there and later they then tell you yes, this was a great hotel. And then we have an endogeneity problem. For the moment, let's just ignore that and I'll come back to you in a second. So what we do is we run this regression and we run it in a bunch of different ways. So we run a straight regression which just looks over the entire data set and asks what is the value of a rating point for essentially a percentage change in demand? It's about 6%. We then cut by management type and we find that chain management hotels don't get quite as much of an effect from ratings as franchises do, which in turn get less of an effect than independent hotels. This makes sense to us because independent hotels don't have the sort of the brand effect behind them. They need more information. We also do this by how fancy the hotel is and STR ranks them from economy through luxury and the coefficients line up in that order. So basically as a hotel gets more upscale it becomes more and more important that it has high ratings for demand and that again makes sense to us. And then finally we have this sort of pattern of coefficients going from 2005 through 2014 and those are again increasing over time to eventually quite a high level, sort of 25%.  A more realistic deviation is about 0.1 or 0.2 rating points but if you could raise your rating by an entire, your hotel rating by a full rating point that would be a 25% jump in demand. Okay, so that's very substantial. There are some sort of non-minitonisties here we see something to do with financial crisis that was a bit of a dip but in general the pattern is that people paying more and more attention to these ratings over time. Okay, so let me talk briefly. Yeah, sure, very briefly about other ways you could get at this. So we have this idea of using an IV. We have reviews from three different distinct platforms, TripAdvisor, Hotels and Expedia and their importance changes over time. In fact, the main thing that happens is the TripAdvisor becomes much more important in the last half of our sample. It really gets way more traffic. When I looked at the most important websites, TripAdvisor is the top 20 website, expedionhotels.com or not anymore. And so the TripAdvisor's really got a lot more traffic. And so what we can do is we can go ahead and measure engagement through Google Trends as looking at the market platform combination. So TripAdvisor in San Francisco, we go and look at engagement as measured by Google Trends with that phrase over time. And then what we do is we use these two new measures of ratings. So the first is an engagement-weighted platform average. So we're gonna go look across these three platforms. We're gonna use as weights the engagement from these various platforms. So as engagement as measured by Google for these various platforms. And so now your rating at any given time period is gonna be a function of how much people are currently engaging with that platform. So it's important to be doing very well on TripAdvisor at the end of the sample where that's the dominant platform.  So now it's gonna be the case that the instrument says, if you're doing better on TripAdvisor at a time when TripAdvisor is dominant, the instrument is gonna be positive. If you're doing well on TripAdvisor at a time when TripAdvisor is not that important, the instrument's gonna be negative. And the idea here is going to be that the admitted demand shocks, whatever they may be, are gonna be assumed to be independent of the aggregate change in platform tastes. And so as long as I'm willing to believe that whatever this one hotel is doing in terms of its marketing efforts is unrelated to the global changes and preferences of these different platforms, the instrument is gonna be valid and excluded. And so we can then carry through the same scheme but do it as an IV. And what we find is that the IV is considerably less stable. So you'll see that the standard errors are just much, much, much bigger. The first stage is not nearly as good as we'd like, but you get the same general trend in platform engagement over time. The coefficients are slightly attenuated sort of throughout. So we're getting slightly smaller effects from the IV than we get from the LLS. But still big and still on the order of sort of 25% by 2014. Oh yeah, so I moved through that very quickly. Hopefully that was enough to give you sort of the vague sense of what the IV strategy is. Let me talk quickly about the theory and then the structural model. So I'm trying to decide how much the theory did do. Okay, so let's think quickly about this in terms of a hoteling model. So imagine that you're in a world like 2005 where maybe people were not that well informed about hotels. So here I've got two hotels, A and B. Hotels can be high or low quality, but people don't know which switch. And they have priors that it's equally likely that A is better than B, then B is better than A. And the assumption is that one of them is good and one of them is bad. So that world is a completely symmetric hoteling model. The hotels are gonna split the market. They're gonna charge the same prices. The prices are gonna be equal to the transportation cost. And that's sort of a baseline. So okay, so this is one way of thinking about the world without information. Now let's ask the question, what happens when a platform comes to town and tells you which is the better hotel? So once we know the hotel is, say, the better hotel than B, suddenly a hotel is gonna grab, A is gonna grab a large share of the market. And if prices didn't adjust, the threshold type would be given this by this expression here and A would get a lot of the market. All the consumers would weakly benefit, but in fact, only the ones who switch hotels are gonna strictly benefit. So the people who would have bought from A beforehand don't benefit. The information is nothing for them. Likewise, the people who are very close to B and don't switch their decision, but the people between 0.5 and this data bar do benefit. They switch and they now buy from the better hotel. What's gonna happen next? There's gonna be price adjustment. And with price adjustment, hotel A is gonna raise their price because it's commonly known that they are better. Hotel B is gonna lower their price because it's commonly known that they are worse. The threshold type is gonna move back towards hotel A and hotel A is gonna charge more and hotel B less. And so now what happens to consumers? Well, there's a mix actually and it's interesting the distribution of welfare. The people who are near B, who are very near B are actually better off after the information is released that the hotel is bad because hotel B drops their price.  People who are even better off are the people who are near the point who switched to the better hotel. They've chosen to switch despite the high prices. We know that they're doing better and they're benefiting from the improved quality. But there are people who are losing in this world and they're the people who are near hotel A who now have to pay more for the same hotel that they would have chosen without their views. And you could think of this as like what happens when it's your neighborhood restaurant and it's beautiful and nobody ever goes there and it has fantastically good quality and then somebody writes up a review in the times and suddenly everybody goes there, right? This is not ideal.  And so that's sort of what we're interested in here. Okay, so how are we gonna get at this when we try to do this using data as opposed to theory? The first step is to estimate demand and that's really gonna be important. We wanna know how it is, we have a hotel model here but we know the preferences are gonna be more complicated. What's the relationship between tastes and prices and ratings? That's the main thing to figure out. Once we know that, we're gonna try and simulate demand when ratings are not known. But in order to get the prices correct, we're gonna have to write down supply side first order conditions and then suppress the information and allow prices to re-equilibrate and see what the market would look like in the absence of this information. So what would it be like if in 2014 we somehow took away the ratings again and people were uninformed like they were in 2005 and then measure the change in consumer surplus across these scenarios. Okay, so this is easy to write down. As I'm sure everybody knows it's hard to do, what are we going to do here? We're gonna have people forming beliefs about quality that they don't know. Their beliefs are gonna be based on rating. Their beliefs are gonna be based on things that are fixed about hotels, like that it has a gym or that it's a four seasons which we know has a good brand. And then characteristics unobserved us which have some notation side. And so essentially what we ended up writing down is a model in which there's a true utility. There are beliefs about that utility and therefore there's expected utility. An expected utility is gonna have a random coefficient on ratings. So there's fixed effects of various forms. There's a larger taste shock. There's some disutility of price but most importantly there's a random coefficient on ratings. And so this is gonna look like a BLP model of demand. Greg, can I ask a quick question from the audience? Yeah, so the same hotel has many types of rooms and they cost a different amount of money. And so how do you think about that aggregation? We, again, we collapse that, right? So we have revenue and we have occupancy and we just go ahead and we said that's the average price. And so essentially what we're thinking about is average rooms that are being sold. And yeah, I mean, hopefully that doesn't bias things too much. It doesn't seem sort of first order again. And then I have, I guess, an additional question. Maybe I missed this. So it seems like you have a random coefficient on ratings but not on price. But your theory model had, well, I guess no. I guess maybe your theory model didn't have a different price sensitivity, but one would think that there are distributional implications here of the price change. I sort of agree with that. Yeah, so I agree. And I sort of think that we didn't do it because we wanted to make our lives simpler, but I could imagine somebody reasonably asking us to do that and we would do it, but maybe not now. Oh yeah, so we assume away the indigeneity of ratings. But we can fix this when we do GMM because we have IVs, so we'll fix that at some point. The main thing that we're worried about is price indigeneity here. And in the hotel market, price indigeneity turns out to be really quite difficult to deal with. The BLP instruments turn out to be pretty weak. The Hausmann instruments are not very good either because it's a geographic market. And so what we do instead is we add a supply side, which we need anyway for our structural contractuals, and we add a particular independence assumption. And let me quickly show you what that is. So in BLP, typically you have two equations that are important. You have an equation for mean utility, which is this delta JM on the left-hand side. And that has an error. This is the unobserved demand shocks. And you have a cost equation, and that has an unobserved cost shock. And it turns out it's sufficient for identification to assume that those two objects are independent of each other. So if we're willing to assume that the cost shock and the demand shock are independent, you're in okay shape. And here we have a lot of fixed effects so we don't feel that bad about it. So, you know, I would generally tend to think that the four seasons is a good hotel. It costs more because it provides better amenities. It's high utility. If I didn't have a fixed effect in there, I would think that those two things were very correlated, but having at least conditioned on the fact that it is the four seasons, maybe I'm willing to believe that shocks to demand and shocks to costs are not correlated. And so that's the approach we've taken here. And I'm not gonna argue why that's true because I don't have that much time. But in any case, you could do this, and we have done this, and it goes badly, at least it did the first time. And the reason it goes badly is because of capacity constraints. So let me talk quickly about capacity constraints because I think this is actually quite important. So if you have three hotels on a market, say, you have very simple loge of demand. They're equally attractive, and they have capacities one, two, and three, and there's a market size of four. Since they're equally attractive, the quote, unquote, first three people to the market are gonna split the hotels evenly. But then hotel one sells out. And the next person who comes in, person four, is gonna have to be split evenly between two and three. Okay? Observed sales are one 1.5 and 1.5.  But in fact, it's just the hotel one is small, right? They can't take that many people. And so you have to basically correct for this if you're going to estimate the one correctly. Here's also a good point to point out where what we're gonna do is different from what Chris and Julie do in their paper. So what they do when they think about vending machines and capacity constraints is they're not willing to make this continuum assumption that I can hypothetically have a person subdivide themselves into a continuum and evenly split themselves among the hotels. They're gonna actually have literally people walking up to a vending machine and buying a Snickers and then the Snickers sell out.  And so here we're gonna make this a simplifying assumption that people as they arrive are kind of like behaving like smooth loge of continuum consumers one by one. Okay, so that doesn't work. What can you do instead? You could add data. And so we have data that has daily data. And what we can prove is that you can do an inversion if you know more information. So what are we gonna use specifically? We're gonna take market shares at the monthly level, which we have, but also daily market size measures. So how many people book hotels every day? And there's a lot of variation by day of the week, okay?  And it turns out that there's a unique inversion from the vector of capacity constraints, market sizes, market shares and daily demands back to the mean utilities. And we go ahead and we do that. And then this is a more complicated procedure than the usual BLP inversion. And you'll get to a ton of clever programming to make it work, but you can recover mean utilities. And I can sort of prove that you can do that, but I'm not gonna do that today. And then what I can do is write down an appropriate capacity supply side as well. And the supply side first order condition is going to have to reflect the fact that sometimes raising your price is costless because for some days of the week, say, say Saturdays, if you're a small hotel, you're likely to sell out. So it makes sense that you'd wanna charge a high price. One thing to say about the supply side is we're assuming here that hotels set one price for the whole month in the same way that we think about BLP and car pricing. Of course, hotels are smarter than this. They do revenue management. And so this is an obelisk implication, but it's one that allows us to sort of fit into the stand-in sort of empirical IO playbook. Okay, and then I could show you that if you ignored this, it would make very, very big differences. In particular for hotels that sell out a lot, the extent to which we would get their first order condition wrong could become quite, quite substantial if you ignore capacity constraint. Greg, just a question from the audience, which is how often do these sell-outs happen? And then a follow-up question from me, which is like in the data, it's not always, it's a kind of a judgment call when the sell-out occurs. So how did you choose that judgment call? Yeah, both good questions. So hotel sell-outs are not, so first of all, why do you have to answer your question? So we'll often say that the sell-out to 90% is a sell-out rate, it could be 95, right? And then there's a question of like, okay, and so how often does it happen? And we're finding, I actually can't remember the number of my top of my head, but I think it's sort of like on the order of like 5%, maybe it doesn't happen super often that data. But importantly, we never actually measure sell-outs. What we do instead is do this all through the structure. So because we're just using the daily data, total hotel bookings, not individual sell-out rates, we never have to make this judgment call. We just have to know how big the market fluctuations are to rationalize our monthly market shares. But we've also done some separate work in trying to figure out sell-outs, and then it's different. Okay, we estimate the model by simulated maximum likelihood. We run monocalls in a different paper that showed this is statistically efficient. It is, it's great, it's just really a pain. And then we plot the distributions random coefficients. And so what I'm showing you here is that over time, it's more and more the case that we're seeing increasing tastes for ratings, people care more about them, and also sort of a dispersion is increasing. So how much people care about them is changing over time, or that might be a little bit of functional form. Okay, finally, let me talk for the next like two minutes about kind of actuals. So what are we gonna do? We estimate all of this, we estimate demand, it takes a long time. We get a pricey list to stay estimate of about minus two. And then we compute kind of actual national prices. Here, as I mentioned before, we have to deal with sample selection. So we do some imputation. And then what do we do in the actual kind of factual? We say, suppose that instead of knowing the true rating, instead people had, they thought that all the hotels in the given market had the average rating of hotels in that market. So collapsing the 4.8 and 4.2 of the two hotels in the market said a 4.5. And then saying, what would people do when they just thought it was 4.5? And we're gonna do a couple of scenarios. We're gonna do one where hotels keep the prices the same and another where they set them in complete information national equilibrium. As I said before, the change in consumer surplus is about $2 per room night if they stay the same. If they set them in random, in complete information national equilibrium, it's gonna be the case that the really low rated hotels are gonna charge more. The really high rated hotels are gonna charge less. The effects we're finding are in fact almost too small. And so that means maybe since the structural model doesn't fit perfectly, they're not big though. They're on the order of $6 up at the top and $6 down at the bottom, which maybe says something about how much market power individual firms have. Nonetheless, this scenario has very big effects on revenue. So hotels at the bottom would make, we estimate 50% more revenue if the fact that they were bad hotels was suppressed. Hotels at the top and there are way more hotels at the top would be losing sort of 11% in revenue. So it would be a substantial hitch to their revenue. So these ratings are important, but how much they change where people stay and therefore the distribution consumer surplus is much, much smaller than the individual impacts on firms. And so just to conclude there, we find that in a kind of actual world without ratings, the average above average hotels, four-stop plus, they're gonna lose a market share. They're gonna decrease prices. Well above average, below average hotels are gonna gain market share and increase prices. And the effect of consumer surplus is about one to 2%. It's much bigger for firms. It doesn't appear to be to do massive amounts to markups. That is to say the ratings themselves are not quite, the ratings changes are not quite big enough dramatically change markups for the firms. And then finally, heterogeneity is important. So consumer surplus effects are big for the kinds of people who care very much about quality. So those go from sort of one to 2% up to 5, 6%, which makes sense. Okay, so that's it. I wrapped up the last button about three minutes, right? Thanks for that paper. And I think now Kiara is discussing. Yep, I hope you can hear me well. Thumbs up, down, yeah, okay, perfect. So thanks for giving me the opportunity to discuss this paper. I really like this paper for a few reasons. The first one is that this is really the first paper that takes seriously the estimation of welfare effects from online reviews. And yes, there's been some follow-up papers in K. I think Rhymers is in the call today. She and Joel will follow up a paper on book reviews. But I would say that these are these hotel paper proceeded by a few years. The graph was very upfront about the fact that the model is static. They are thinking about maximizing static profits, setting prices statically and having consumers making one-off decisions. And the fact that the underlying quality is sort of cannot be altered by hotels. The other piece that I really like, given our common work, is the fact that they take capacity constraints very seriously. And because these has implications on sort of mean utilities that get estimated for hotels that sell out often and hotels that don't sell out but get remaining residual demand because of the sell out of the other hotels. And in terms of results, I think they make a lot of sense because ratings lead to better matching between travelers and hotels. Travelers are better off on average. In particular, the benefits accrue to travelers who care about quality. And just like a very simple back of the envelope calculation and I'm not sure it's actually completely correct, hotels sort of capture about half of these extra surplus and travelers capture the other half of these extra surplus. Okay, and the other thing is that ratings are more influential for independent hotels for which consumers won't have strong priors. So also these makes a lot of sense. I have a couple of questions. The first one is really about what are we capturing and whether we can separate the role of reviews from the role of the internet and everything else that these online travel agencies are doing. So if I think about online travel agencies, I'm pretty sure that demand for hotels would have changed given OTAs even in the absence of consumer ratings. And that's just because consumers can find hotels online that they didn't have otherwise planned. Okay, there was a reduction in search cost. And so is there any way to separate really the role of online ratings? And these has to do with the type of counterfactuals that you want to run. The current counterfactual has hotels having the same average rating in the city and month, I believe, but maybe we should make the average rating be dependent on some observables that consumers have even in the absence of that five star review. So for example, there's an expectation for ratings for chain hotels that is different for the expectation of ratings for independent hotels or for different locations within a city. The second thing that I wanted to focus on was the fact that benefits are overwhelmingly concentrated among consumers who really care about quality. And if I've done my simple math correctly, if the top quantile, and I think it's quartile, enjoys the most benefits, that means that the other 75% of consumers have negative welfare effects. And so it's super important to really look at the distribution of the losses and gains that these ratings have affected. And here's why I actually think, and it's something that Jacques mentioned at the very beginning, is it reasonable to think that 75% of consumers get negative benefits from ratings? Probably not. And here's why the ratings are bunched up all the way between 4.9 and five stars. And so I think the ratings, as stated, don't signal the heterogeneity that actually the heterogeneity of information that consumers can receive from the internet. And so that's why I think in a sense, you're underestimating the benefits of ratings because of these rating bias. Okay, and the last thing is, if you can talk a little bit more about the imputation or how do you think the imputation is gonna affect especially your counterfactuals on independent hotels? Because I recall reading that independent hotels are much less likely to share information with Smith's travel research. And so are you worried that these imputations may sort of bias upward the benefits to independent hotels because maybe only independent hotels who are successful are disposing information to SDR? And with this, I'm done and overall, great. Shall I respond, Andrew? Yeah, okay, great. So thanks very much, Kira. Those are fantastic comments. I mean, I think so your first thing was about what's really being measured here. And I think my reaction to that is maybe that you're saying there are these important secular trends that we're kind of not accounting for, like maybe independent hotels are becoming important. Perhaps we should consider trying to think about how we could take account of those in the demand system and see what's left over as residual variation that correlates with ratings. So basically, rather than allowing ratings to just be this thing that moves over time. I mean, in some sense, our defense would be, look, within independent hotels we have variation ratings in part were identified with that, but would it be more convincing if we were able to sort of say, okay, allowing for a trend in the direction of independent hotels say, what's happening there? And so I think that's one thing we could do for sure. Your second question was about the distribution of gains. And I think that goes back to the same thing I mentioned at the beginning, which is there's so much more information in reviews than is being captured by the rating, which I think we're saying the same thing. It's sort of like, it's gonna be an underestimate, it's gonna be an underestimate because we're being too coarse with how do we think about the world? And so, yeah, I think one, it's a great suggestion for us to say, this is what we find, 75% lose, 25% win. This is, and then to say, but maybe we're getting it wrong. And I think both those things are important. I think the decomposition of who's winning and losing is very important. And then also to say, okay, but there are these caveats. So I would add a hundred percent agree with that. And then the final things you mentioned, the imputation. So, I mean, yeah, we're doing pure selection of observables, right? So to the extent that it's their specific independent hotels that show up in the day to do better, they are gonna get overweighted. I'm not sure what I can do about that unless I can find some more intelligent imputation procedure or something that uses an IV to check. But it's worth it to be thinking about or trying to do some contest on. I don't know what that right solution is there, but that's something for us to think about.


Hey guys this episode we're gonna see if this bed edger right here will run stay tuned All right guys, this is what I got this week This is a bed edger We're gonna see if it's gonna run this It don't have the recoil, but I can put a rope on here and wrap it around and pull it and start or Use my air gun to spin it over with that nut right there, but this is a bed edger. I also have This thing this is a kerber a curbing machine to make curbs With concrete. That's what this is and I got all the towels for it And I got this and the bed edger. I got all for 50 bucks off my boss so Looks like down under there, but we're gonna get this unloaded get you set up. We're gonna see if it's got spark and Shoot I'll have to send her down after a little gas and we're gonna put some gas in it See if we can get it to fire up here lately. I've been doing pretty good on everything I get I don't have to work on it fires right up or it works already So but it'll give us a little bit of tinkering. I think it's gonna need a tire there So see that that's not gonna hold it or yet. There you go But that's no biggie. I don't know if I'm gonna use this engine to build a little go-kart Or if I'm just going to you know get it running fix it up. Maybe a couple new tires and sell it I don't know what I'm going to do yet I've got something else I bought, but it's not here yet because it's too heavy and too big to put in the back of the truck and It is a miller 303 phase welder, so that's going to be interesting and I've got it cheap too I gave 300 bucks for it So I'm gonna get this unloaded send her after some gas and we're gonna go from there and I'll be right back with you guys Hang tight. Okay guys. We're back first thing we're going to do is check this for spark and I'm going to use this little tester here. I've got what you do is you put this in Right here on to the spark plug and you put your spark plug wire on this in and It will tell you if you have spark if it lights up when you crank on it, so I'm gonna put this in down here on to the plug and this end Onto the plug wire. You see that right there.  You get the camera out and all of a sudden. It's a damnations day parade I'm not promoting that anyway, by the way, so Well, I mean jeesh Alright, I'm just gonna pull with this rope here and let's see if we've got sparks if it'll light up No light. Let's turn the switch on now. That's right Huh? Oh What I'm not trying to start it. I'm checking for spark honey.  It's lighting up. So we have spark guys. I'll do it again Just in case they didn't see it. Let me zoom in on here a little bit Watch right here. Yeah, watch right here Look kind of a little bit weak, but I think maybe enough to fire once I get some gas in here See if it lights up again Yep, okay, so we've got spark so now we're gonna wait on monkey to go get us some fuel and And We're also going to try this Ryobi here that I've got this blower behind me that I bought and We're gonna see if it'll run but I'm thinking it probably does but it's been sitting a long time. So she's gonna get us a fuel and We're gonna put some fuel in here. The tank is very clean It looks like brand new on the inside of this and it is empty. So I'm thinking whoever parked this That my boss got from Probably emptied the fuel out. I'm not sure but we'll see so carburetor. Maybe I will know if it runs It does have some good compression. So I'm feeling pretty confident about that. I'm not sure what size it is What size engine it is but we'll find out and I'll get back with you on that but right now I'm going to jump off here let her get some fuel and Go get my game plan on see If it don't run, which is plan a We're going to go to plan C Because plan B will be to figure out what we're going to do on plan C. You guys follow me on that list And I'll be right back with you guys hang on all right guys I've got the likes on okay I've got the switch on I've got I've already poured the gas in it I've got the choke on and I believe the gas is on And I let it set for a minute Make sure this is in neutral and Let's give it a shot It's supposedly this thing sat well, I know of a year of a year at the shop behind the shop and My boss Eddie He said it sat for several years Before he got it so Let's see what happens guys Freakin believable a little bit of gas leaking out of the bottom of the carburetor right there by the wheel So but it is running Let me shut this off. Okay, right down there By the wheel. I see some gas. I don't know if it's because I had it towed it up Let's zoom in on that right there by that wheel. That's the bottom of the carburetor right there so Little leak there There's a little o-ring right up here on top of these Right there. That's where it's leaking from So no big deal. It's like a 10 cent fix actually it's free for me. I've got o-rings at work So guys, there's that Honestly that is the first time I did try to start this thing and And I do know that it has set Because I've been where I'm at a little over a year and I know For a fact that it's been sitting there for a year and it hasn't been started And I forgot to mention that This this tool here. I showed you a second ago This will tell you if you've got spark from the coil now It's not going to tell you if you got spark from your spark plug in order to to do that You got a Touch in the head or some kind of metal and pull it and if it sparks are good, but this yeah, this will tell me You know that of course It's getting sparked from the coil. So that's that well guys there we are so I got I got the weed eater or the leaf blower See here we go now. We've got the Hell's Angels Anyway, I got the leaf blower here. It's a Ryobi. I bought it for 25 bucks or something. I talked to guy down He said it should run He hasn't heard it run, but he was told it run. I'm told it runs I don't know spark plugs kind of buried up in there So I didn't even check it for sparks of what I'm gonna do. It's a mix. It's a 50 to 1 mix So it's a two-stroke. So what I'm gonna do is mix up a little gas We're gonna put some in there and see what that does so so far I haven't really got to work on anything other than this, you know, we'll on the on the edge or we will Just put some air in the tires I may take it out here inside the house and see if it digs up a little dirt it should One tire. I know like I said, I showed you it's got a hole in it So it's not gonna air up, but that's no big deal So I don't know that that inch that sounds really good didn't smoke or anything does have a slate fuel issue, but That's not a problem. I think it's half inch Maybe 12 millimeter nut or a bolt on the bottom of the bowl And you take that off and the bowl comes off You just put your new seal around there o-ring whatever the case is on that they're usually o-rings now But it may have the gasket But it's no big deal. So or it could just be dry from sentin So it may even stop leaking. It's not one grudge night anyway. So and then we've got the Kerber over here You know, we can plug it in and check it. So I Mean really, I don't know what to tell us to do But let's mix up some gas and let's see what the Ryobi does. Okay. All right guys I'll be right back with you in just a second hanging there All right, I'm back What I did was this takes a 50 to 1 mix and I mixed it in our little boat thing that came with our Boat motor that we bought last year That'll be another one. We'll take it out and and try it out I've got about an hour on it and it takes like 18 hours I'm a whole crap to break it in. I don't know why you don't have to do it to a weed eater And it's essentially a weed eater.  They're just rubber so That's just a couple of dollar fix and just a few minutes So all right guys, let's let's see if this thing will run Put it on a choke There's Joe already primed it and I just primed it again. There is gas coming up out of here it says ten times guys, but If when you're pushing this bulb and you can see fuel in there You're good to go. It's it's prime But if you get a little leak here that may cause it not to run right and whatnot But that may be another Another video. Let's see. It says to open the throttle up Got throttle open.  I hope my mic's working. All right guys There we go, so It runs Not well when you open the choke all the way up It seems like it doesn't want to idle but when you go to rev it up it bogs down. You heard that boo That's telling me it's not getting enough fuel so when you put the choke on and It's running and it runs good you about half choke. It runs really good.  That's telling me that It's not getting enough fuel I'm going to check the air air filter and stuff like that. So that's something we could do take this carburetor part here one of these days soon and put it up here on my desk, which is my workbench now, but we'll do that and see what we can do and See if we can get it get it fixed and run them properly like I said, I'll check the filter first But it could have something to do with that parmer bulb you know because it's it's It could be you know sucking air because you know, it's essentially a leak Because that primer bulb is cracked and gas is coming up out of there when you when you push the buttons So that being said, I'm thinking maybe you know, it's sucking too much air That's why it wants to run when you cut the arrow down on the choke Crows I hate crows and scared of them so We'll check that out, but I've got a runner. I got 75 dollars in this stuff Couldn't be more happier. So we'll see what we're going to do with everything. But again, thanks for watching guys Don't forget check out monkey 1000 she is editing a Video as we speak so she'll have one up here She just put one up me talking to keys in the car with car run and I had done a lot with a with a coat hanger So but that was funny. So anyway guys again, thanks for watching appreciate it Enjoy the rest of weekend and if you see this tomorrow, then please have a great week next weekend We are going to new Smyrna Beach and we'll get some video for you guys of some of that So that being said guys, thanks again Please comment rate subscribe share these videos and With that I'm going to say that she bear I'm gone. Remember the myth the man the legend she bear 1000 Take care guys. See you. Bye. Bye