Hi. My name is Jonathan Citrin. I teach here at the Berkman Klein Center for Internet and Society. Sounds more normal every time I say it. And first, in administrative announcement, we are being recorded, not just by the usual panoply of suspects, but by intention. And we are live webcasting. So be aware that anything you say will be recorded forever and might be used against you. I always like to bring in the Miranda warning. It's my pleasure to introduce Kendra Albert for today's talk. I'm so pleased to have a chance to introduce my colleague, friend, sometime mentor, Kendra. When I first met Kendra, they have been studying the Dalbert case and its relationship to the Frye standard of evidence in the federal courts, which was a little surprising because Kendra was undergraduate and not a law student or law graduate. And Kendra carried on that tradition through a summer at the Berkman Center program working not only on our OpenNet initiative, but later our H2O project for open casebooks. And they took not only to helping to format the cases and such and doing some basic research, but started editing the cases and then reading all of the cases for a course on torts. For those of you who are law students, you will realize that voluntarily reading all the cases for a course on torts is an extraordinarily unusual thing for somebody to be motivated to do. And I think that did inevitably, despite Kendra's efforts at times to avoid it, point Kendra towards law school, which to our great fortune Kendra attended here. And while here, for those of you again, thinking about how to be involved in cyber-related topics, Kendra punched a ticket in so many different wonderful places. The public citizen litigation group, the Electronic Frontier Foundation, and an unlikely third in that triumvirate, Cloudflare, the organization that will help you avoid getting DDoS'd when you least expect it. It will tell you you're going to get DDoS'd and then you're DDoS'd. It's also a good opportunity to thank Cloudflare for their pro bono support of our PERMA project, Cloudflare, we love you. And it's interesting that while working there, Kendra took up an examination of some of the boiler plate legalese passed around like Christmas fruitcakes from one company to another without attribution. Is that still a good reference that the fruitcakes that just get passed on because nobody actually wants to eat it, but they're still an appropriate gift, and they never expire? And the same is true of legal boiler plate. But then Kendra actually decided to read it and think about how it might apply to not just terms of service of the usual sort about if you do something to the company, then your warranties or void or whatever it usually says, but rather what is the role of the company in trying to enforce basic humane behavior among users of a service for which otherwise the company might prefer to be invisible? And to what extent can that be governed by boiler plate to the customer, to the company itself, inheriting that boiler plate from somewhere else, and to what extent is it a challenge to every company to actually sit back and think about, as Terry Fisher would say, what are the prerequisites for human flourishing, and how should we embody that in our terms of service? And I think not to anticipate too much Kendra's talk, the answer may lie disarmingly closer to the second answer than to the first. And that, of course, poses some really tough questions for the companies and for all of us as we try to put companies into a role of policing some of the most profound questions that face us in the digital space as people communicate with one another, get into conflicts, troll one another. How to mediate that is just an unanswered question. Nobody has figured this out yet. And I'm hoping by the end of this talk we will have done so thanks to Kendra. So with that modest introduction, I turn it over to Kendra for her valedictory lap back at the home team after she had gone to the Zeitgeist Law Firm, a technology-oriented law firm that's thinking about this stuff. Kendra, take it away. Thank you. Thank you. And Jonathan, that was an amazing introduction. And I'm not sure the talk lives up to the promise of the introduction, but maybe it does something different, and hopefully that's how all good talks work, I suppose. But before I get started, I want to just take a moment to express solidarity with Unite Here Local 26, who are the folks who are protesting outside. I think it's important even as we problematize and deconstruct notions of free speech to recognize that it has an important role in some of the things that are most valuable, which is ensuring people who, including ensuring people who represent workers can strike and be heard. So I'm about to do a lot of things beating up free speech, but first I want to recognize its role and express solidarity with them. So when I use the phrase legal talisman, what I mean to suggest, as Jonathan alluded to, is that I'm talking about a legal term of art, a legal word that's out of place. And it's invoked to make or justify substantive decisions that don't involve formal legal process. Decisions that are made in a way that does not involve the court system, does not involve legislatures, does not involve administrative procedures. It's a legal term that shows up in sort of the places you would least expect it, the arguments that you would least think that it's justifiable. And I started looking into this, as Jonathan said, because I was spending a lot of time thinking about online abuse. And often what you see in online abuse is legal talismans everywhere. The most sort of comment of these was pointed out by the work of Sarah Zhang and others in deconstructing the role of speech arguments is free speech or free speech, as it is sometimes known. And you can see that both this comic is sort of a funny joke about the role of free speech online. But this idea of nuanced free or self-righteous pain comparisons really points to this idea that free speech has come to mean something in online discourse that is separated from the First Amendment reality or from the law, that it's invoked in places that we might think of as rather unlikely for sort of a First Amendment rhetoric. And so if we imagine why that happens, if we imagine the magical powers that the free speech talisman can embody, it sort of wards off regulation, responsibility, and liability. It's ideal for use in the United States. And it's crafted to be especially effective to responding to claims of rampant on-platform abuse.  Well, it allows us to shift to the conversation in a way that's really meaningful and in a way that changes our relationship to regulation and liability. Of course, I'm not the first person to point out that the use of the term free speech is not really legal. I think the now obvious counter-argument is this XKCD, which is actually not new, which invokes the legalistic responses to free speech. Well, it's not the government doing the censorship or the platforms are not government speakers. People don't have to listen to you. That's all the sorts of things that come along with these invocations. And it's not sort of just confined to trolls online. This is our friend, Ted Cruz, who was talking about the ICANN transition, which was the process by which the US was ceasing its contract with the International Assigned Corporation of Names and Numbers. And I probably got that wrong, sorry. But anyway, what he says is that the internet is the internet. It's an oasis of freedom today because of the First Amendment, which is a sort of strange claim. Because I think most folks who are actually involved in ICANN would argue that although First Amendment principles may be involved in decisions that ICANN made or underlying platform or technical thoughts, the First Amendment was not super involved in the ICANN contract. And before I go any further, especially since Jonathan mentioned Cloudflare, I want to say this is just me talking, not my employers or my clients. So some other places we see free speech are in corporate responses. So Apple to the FBI, forcing us to unlock the iPhone, violates free speech, Cloudflare and free speech, which is a blog post that Matthew Prince, the CEO of Cloudflare, wrote about how they dealt with Islamic terrorism or radical Islam and jihadi posts on the platform. Verizon, net neutrality violates our free speech rights, which was one of their responses to the FCC's net neutrality rules. And then perhaps maybe the most famous quote of these four is from Twitter's Tony Wing, where he said, we are the free speech wing of the free speech party, something I'm sure he regretted for pretty much the rest of what came after. And some of these are actually about legal proceedings. The Apple versus to FBI and Verizon are about court proceedings where they were making a First Amendment argument about what they had the potential to do. But Cloudflare and free speech and Twitter's free speech position are not, this is not in reference to this particular court proceeding. This is not because they're putting forward an argument about their particular liability or lack of liability or how they should necessarily be regulated. In fact, it's just a shorthand. It's a reference to a body of knowledge that's stored elsewhere. And it's a shorthand for a bigger, more complicated set of ideas, a set of rehearsed arguments about free speech and the marketplace of ideas and allowing everyone to speak and the hecklers veto and this whole long set of things that you might hear if you sat in on a First Amendment class or run the First Amendment case. And it's often a way to make things feel a little less arbitrary. If Twitter had said, we take down whatever we feel like, that maybe would present a less compelling statement of their position on the subject, even if it was just as true as we are the free speech wing of the free speech party. Legal talismans also avoid definitional problems. They allow for us to look at this body of law as a way of understanding what exactly is going on. But with it comes its own problems. So this may be familiar to most of you as the text of part of the First Amendment, which says Congress shall make no law abridging the freedom of speech. Of course, the reality is that Congress makes law abridging the freedom of speech all the time. And so when we look at what this means in practice, I actually stole this outline from a law student. But I think it's actually a very powerful illustration of all the things we have defined as not speech in order to abridge them. All of the things that we claim are lower value speech, like commercial speech, or employee speech. All of the things we claim are not speech at all, defamation, true threats, incitement, and how really this is kind of much more what the First Amendment looks like. There's an asterisk after every word with which comes a very long set of definitions of what that means. Like even Congress really also means and the states because of the 14th Amendment. So when we unpack the meaning of these words, we understand that they actually have all of this context and all of these things that come with them that aren't necessarily obvious from the face of the amendment. Moreover, and to reference the XKCD again, Leo Tausman spoke the power and obligations of the state. And I feel kind of silly saying this because it's so obvious. Like when we talk about law, we're talking about the state. But I mean, perhaps obviously, platforms don't have the same obligations to their users that the state has to its citizens. Terms of service are not constitutions. And for the most part, platforms don't grant users positive rights. And so when we think about the legal frameworks that take into account state power and state constraints, like that's what all of those asterisks in the First Amendment are about, like state power, state constraint. How do we think about the relationship of the state or dynamic with people? That's why we have a First Amendment in the first place. We have to understand that law exists in a world where the actor that's enforcing it is the state. And sometimes users might want those state-like restrictions in the context of platforms. If you believe that your use of Twitter is so fundamental to your identity that actually you want it to be treated like a utility and you want due process for getting blocked, then maybe you actually would love all of the sort of restrictions and the costs of the First Amendment that come along with the use of these words. But sometimes users definitely don't want all of the same things that we want from our government when we're thinking about our online services or all of the same protections that force particular people that we want from the government. So that's how I'm defining a legal term of art out of place, invoke to make or justify a substantive decision that do not involve formal legal process. And I've said a lot of things about the First Amendment because I think for many people it's the most tangible, like easy to think of way in which this manifests. We're used to the idea that these legal obligations, that people talk about the First Amendment and free speech independent of the actual legal obligations or legal content of those words. But what I'm about to do is to apply critical legal studies principles to a more specific legal term in the regulation of online spaces. So critical legal studies for those who aren't aware was a legal movement that concentrated on sort of unpacking some of the baggage that these words come with. Most famously one of the practitioners was Duncan Kennedy who taught here for a very, very long time. But other people from Martha Chimalis to Derek Bell have all used the tools of critical legal studies for feminist pursuits, for queer pursuits, for critical race theory pursuits. In order to defamiliarize the familiar, to take the words that we think we understand and to show us how we really don't understand them at all. And so when I talk about something like defamation, I think it's really worth defamiliarizing. Because to most people, defamation means saying nasty untrue things about people is illegal. Like if we want to talk about the most basic definition you could possibly give, that's the best one I got. I've tried to make it shorter and I think I need all of those words or about a person even. But it shows up everywhere, right? This is, the first one of those is the terms of service of slack. So if you have ever said anything defamatory about anybody on a private slack, that's against their terms. The second one of those is drop box. So no defamatory material in drop box. And then the third one of those, perhaps maybe even most ironically, is Yicac. So you're not allowed to defame people on Yicac, which as folks may know is a sort of service for people to anonymously sort of comment on things that was primarily used by college students. I'm sure Ray Hinko is probably turning over like just like had the force moment in Indiana as I characterize it that way. But the legal standard for defamation to do my best at putting it slightly more precisely is that the defendant who is the person who you're suing published a statement. The statement is about the plaintiff. The statement harms the reputation of the plaintiff. The statement was published with some level of fault, which depends on who the plaintiff is, and the statement was published without some privilege. Now even that, a slightly more precise definition is hard. So this is William Prosser in Prosser and Keaton on the law of torts. And what he says, which you do not need to read, you just need to take my word for it that an old white dude who'd spent a lot of time thinking about torts also thinks defamation is really complicated and difficult and that we're really bad at it. But defamation's gorgeous traditional design hides deeply sexist, racist, and homophobic, which means anti-sex worker connotations. The reality is defamation is a talisman, and I mean it as a talisman because the actual analysis is so legally specific as to be unactionable in a practical matter during a content review. It is incredibly difficult for the sort of content reviewer that we talk to think about Facebook or Twitter having, the person, the click worker in Malaysia who's like trying to figure out whether something is like okay or not okay to figure out if something's defamation under US law. That's just like not a thing they could actually possibly do, in part because truth is an absolute defense so you have to know whether the thing's true or not to know if it's defamatory. But the history of defamation is pretty damn sexist. In England, under the slander of women active in 1891, female plaintiffs were a better leg slander for words that impute unchastity or adultery and you need not show others reputational harms. And that law was repealed in 2014, which like yay for that I guess, but 2014. And this historical context is not just like unique to England, defamation law in general is deeply invested with notions of binary gender and appropriate roles for women. In fact, many of the first defamation statutes in the US specifically noted that imputing on a women's chastity falsely was per se defamatory.  It's not just sexist in practice or in, it's not just, it's sexist in practice and sexist in theory because the reality is that even if the statutes are not sexist on their face, when libel and slander actions are brought by female plaintiffs, they are more likely to succeed if they are suggesting that talking about things that are in the private sphere, like accusations of being unchaste as opposed to talking about their business role. So when we see defamation law in practice, it enforces a sort of sex segregation of roles for women. It also, oh, by the way, I didn't add this to my list originally, but it's also really homophobic. In 2012, New York finally overturned the law on its books that suggested that being called a gay was per se defamatory. And so when we look at sort of histories of how this plays out, you can see this is Virginia de Acuna, which is an Argentinian entertainer and she won a lower court ruling against Google for search results for her names that led to porn websites. We can see that it actually does play out as a defamation law having a focus on women's chastity as a successful model for legal intervention. And so the moral of the sexism story is that defamation can work for you as a woman so long as you're concerned about your chastity but much less so if you're concerned about your business reputation or any of these things that involve sort of public sphere roles. As if that wasn't enough unpacking or defamiliarizing, we can also talk about the ways in which defamation law is racist and was racist and is racist. So defamation per se by racial misidentification which is if you call a white person black that is per se defamatory and harmful to their character was the law in many parts of the US up until like really, really recently which is the sad part in the tort was analyzed by John C. Watson in his landmark article Defamation by Racial Misidentification, a study of the social tort and he talks about many things but this case really stuck out to me from 1957 in South Carolina which is is it libelist per se to publish the imprint of a white person that she is a negro that such a publication is libelist per se is supported by the very great way of authority. So facial racism, facial sexism, facial phorphopia. Also you get all the inherent structural problems of the rest of civil litigation, right? Just because a law is not discriminatory on its actual face doesn't mean that there's it doesn't end up being discriminatory in the way it's practiced. So what we know about the structure of civil litigation is that it favors the rich over the poor, that it favors, in the words of Mark Golancer, repeat players over one time lit against people who are familiar with the legal system and who understand how it works and it favors frankly people who are considered more likable by juries, right? Defamation, as I said before, really packed specific. You end up going into, if you end up going to a jury trial, right? Can we understanding that like sexism and racism are still things within the United States?  So people from historically marginalized groups or people who are seen as reaching above their station are generally not necessarily likable by juries. So I wanna come back to this because like, it's sort of funny in one way that these terms prohibit this kind of material, but at the same time, do we, I'd like to think that if Slack understood what the history of the term libelist was, that like maybe that's not exactly how they would choose to frame their terms of service. But the reality is that this isn't just about defamation. Legal talismans were whitewashed real problems in all sorts of different ways. And when we invoke these legal words, we can't invoke them without the baggage. In the words of Inigo Montoya, you keep using that word. I don't think it means what you think it means. And the law is not neutral. The law is not some neutral force that we reach out to and we can pull in, pull into our terms of service or into our online spaces without bringing with it its long, long history of being, of oppressing the oppressed, right? The law is both something that shapes the world. For example, we know what reputational harm looks like because it is defamatory and it's shaped by it. We know what defamation is because we know what reputational harm looks like. And so when we talk about these laws, there are also contested sites, right? The question of whether material is defamatory, the questions of whether these laws apply at all is also a thing that we need to do some serious reckoning with. So although these talismans may appear to be ready to wear, perfect with every outfit, their invocation suggests a bright line rule that in practice, it's really messy, both in the law and it's in its enforcement. And the power of the words is what makes them dangerous. The fact that they feel neutral is what means that we should give them a second look. So I wanna turn back to our headlines from earlier. Have anybody seen the movie They Live? Oh, okay, all right, at least one person. Well, then this joke is worth it. So In They Live, which is a cult movie that everyone, I'm not sure everyone should go see, but everyone should go watch this clip. John Carpenter, the lead character, puts on these sunglasses, which allow him to discover, and I quote Wikipedia, the ruling class are in fact aliens conceding their appearance and manipulating people to spend money, breed, and accept the status quo with some women low messages and mass media. So I'm gonna ask us to put on our They Live glasses and turn back to these headlines. So Apple to FBI, forcing us to unlock iPhone, suddenly politicizes engineering decisions that we framed as technical. Cloudflare and how our business model is protected by a non-interventionist position. Verizon, that neutrality violates our vision of how to make money. To be fair, that's actually pretty obvious from the face of the rest of the argument.  And I both mean these as jokes and not as jokes at all because I sympathize deeply and I actually share the positions of many of the people I'm making fun of. That I understand that for a company like Cloudflare or a company like Apple, that free speech argument may be what they have, using the tools that they have in order to make the argument that they feel is actually just like normatively for the best outcome. But once we understand that legal talismans are protective invocations, we have to be critical of them, even the ones we like, because they come from this history of using these terms in ways that actually don't create, the shorthand is not comprehensible to users and the shorthand is not comprehensible to people more generally. So when legal talismans import difficult concepts, we see the distributional effects of offloading legal responsibility. We see that there are real consequences to, for Twitter who has lawyers, to push off the decisions about what constitutes defamatory to people who use Twitter who generally don't have lawyers. And that legal talismans can both be overbroad and yet too narrow. There are lots of things, I don't feel like I have to say this too much, but there are lots of things that are perfectly legal that are still harmful and that's something that comes up in the online abuse space a lot. And indeed, framing abuse around what's legally permissible and impermissible is deeply ill advised. The insertion of legal language creates the idea that law, not ethics or empathy is what governs. And it substitutes for real discussions of the values that define community spaces. I'm not saying that you can't come to the exact same conclusion about what is appropriate on your platform or what you should do that you would if you invoked the idea of free speech, but I'm saying that the shorthand of saying, well, we're making all the same arguments they made in all of the free speech cases that most people haven't read is not comprehensible to users and ignores things like the real questions I feel like we should ask, like what legal structures does this policy invoke and what values do those legal structures take for granted? When we talk about defamation, it's pretty much impossible to separate defamation historically from fear of sex workers. Is that what you meant? Like, is that the value that your legal structure, you want your legal structure to take? And whose interests do those legal structures serve? And again, we may come to the same answers, but just going through this process of understanding our own normative values in a context that's other than legal argumentation allows us to ask why? And why is harder? Why no defamation? Why free speech? Why these processes? Why these distributional outcomes? But, and it's much harder than this. Than this. Because the talismans are so pretty, right? They're shiny and they're distracting and we get caught up in whether they're the right talisman or whether like, there's some, you know, some ugly thing on them. They're really deeply appealing, but if we want our online spaces to avoid replicating the mistakes that our legal system has already made, we have to move past them. Thank you. So I was asked to end with a question because this is meant to provoke a discussion and I have all the faith that this would provoke a discussion independent of whether I ended with a question or not, but I do have a question, the one that I'm really struggling with now, having spent a lot of time thinking about the baggage of legal systems, which is, are the alternatives worse? And so thank you so much for your time and attention. And I think I'm gonna call on people and then someone will come around with a microphone. So are there questions or if you're gonna do a comment like tweet length, please? Otherwise, other folks won't necessarily get to speak. Well, I guess I've convinced everybody. I hope you don't take this the wrong way, but do you wanna address the fact that most people don't even read this information to begin with and what, I don't know where you're at. Sure, I'd be happy to. I mean, I think it reinforces my point about like offloading consequences onto users, right? The sort of fiction that the terms are binding, whether you've read them or not, is really convenient from a sort of contractual business perspective, right? Because if businesses actually, if companies or people had to make sure someone understands a contract before it was binding, A, it'd be a lot harder for courts, right? Like, because that's like a deeply factual question. And it would be a lot harder for us to make the kinds of contractual transactions that many argue are what make capitalist systems work. So I don't think, like, so I think it just reinforces my point that people don't read this stuff because I don't, like, it doesn't not bind them because they don't read them and it just offloads their responsibility for sort of, like, the enforcement can happen, like, sort of independent of you, whether you need the thing or not. Hey, so following up on that, if enforcement is, let's say, independent of whether the user is really cognizant of what's in the toss, then does it matter that the toss is speaking in legal terms instead of in terms that the user might be more familiar with? And the other point I kind of wanted to bring in was that, I think it's easy for non-lawyers to think of lawyering as kind of rarefied and obscure. And so I wonder if you can talk about the value of including the broader community in the normative discussion that necessarily underlies all of those legal concepts. Yes, sorry, I'm gonna address the second part first and then talk about the toss. I think that obviously there's some irony to giving this talk as a lawyer because, like, my value proposition is, like, hey, I know these words, you should hire me. But I do think that, like, really good lawyering is not around legal terms. And I think good terms of service are also not don't use these legal terms or don't just use these legal terms. Like, sometimes you kind of can't get away with not because actually you really need all the baggage. There is no other set of words that sort of comes out to mean the same thing. But I think that community involvement is, as you said, really important because that can help people figure out what the pain points are in terms of people not understanding what their obligations are. I think, so I was trying to think of some other examples just in case somebody quizzed me before this. And one of the things I kept coming back to is the sort of new copyright infringement intended that you see kind of often in fandom, which is, like, sort of totally talismanic and it's like, please don't sue me. I didn't mean it, which, like, unfortunately doesn't really matter a ton for most copyright infringement lawsuits. But yeah, so I think, like, when you see, like, that kind of community output where people are clearly trying to engage on the legal substance or trying to, like, come up with a better way to explain what they mean, that's, like, a place where, like, lawyering can be really helpful in translating, like, what the community wants into words that are cognizable to the legal system because that's kind of what lawyering's about. In terms of the terms of service and, like, enforcement. So I admit I actually originally did structure this talk around things in the terms of service, but then I sort of figured I would make it a little more broad because terms of service are in some ways actually legal documents, they're contracts, right? And although they're not usually enforced by the courts, they are written as usually written as enforceable, right? So I think what actually worries me more is the presence of these words in spaces where they're, like, not even terms of service, right? Like, in Yikak, it's in the content guidelines, which are incorporated by reference into the terms of service. But are sort of meant to be the cognizable, easily understandable, like, you can follow these things. And I'm sorry, I think I'm not sure I answered your question directly, I maybe talked around it for a while. But if you wanna re-ask it, I will try my best to answer it directly or I can just move on. So speaking of users not reading terms of service, let's say that you read it, but you disagree with it. Right now there's not currently a mechanism where you can say, actually I don't agree to this part. So it's like, okay, well then, you don't like our terms of service at Slack, then don't use Slack. And as somebody who's not involved in the company or not a lawyer, like, what are my, what recourse do I have? Is that something that companies are interested? How would I, as an engaged community member, go about making some kind of change? So, I mean, the sort of legal answer is you can't don't use the service, the sort of actual answer is that I do think there are some companies and some groups who actually really care deeply what their users think about their terms of service and are responsive to user feedback. I've been on, in conversations where folks were like, well, somebody really said they're not joining our service because of our arbitration clause. Can we get rid of our arbitration clause? Like that is an actual conversation that I've seen happen. So it's not just like a figment of my imagination.  But so users do have the recourse of reaching out and being like, hey, I don't like this thing that you're doing. Realistically, part of the problem is like, all of the stuff we talked about with respect to defamation, some of that same stuff holds during contract law, right? Like the history of contracts of ahesion, which is contracts that are non-negotiable and sort of the way that that's come to govern in terms of service is sort of kind of, in some ways not particularly reconcilable with a model that emphasizes user agency, right? Like you can't, you know, you could try to like send them back from the versions of their terms of service with like parts crossed out like good luck. But the, yeah, we like how our body of contract law is just not well-equipped to deal or either is not well-equipped if you think the goal should be agency by third parties or is very well-equipped if you think the goal should be having companies buying people as easily as possible to contracts. So unfortunately there aren't a ton of answers, but I do think you can, especially if you know the company is one that maybe cares about users or think the company is one that cares about users, like reaching out to them and being like, hey, I really wanna use your service, but you have this thing like, you know, sorry. So this is a half-daked idea, so I may regret bringing it up. I'm wondering if your critique of legal talismans could be extended to the way that courts themselves apply legal terms to conclusions. And I'm thinking of the way that the Supreme Court will decide what is obscenity or not, what conduct is speech or not. But is that also a cultural perspective that is having a label slapped on it at the end sort of concealing what's really going on? Yeah, the secret problem with this argument is it applies to all law. I mean, or it's not problem at all with this argument, that like putting things into categories that are complicated that people don't understand and then applying arbitrary rules to them is kind of what judges do and what lawyers do all the time. So that like this problem is not unique to online platforms. And so like some people would consider that a flaw in my argument, but I consider it a strength. So, and what I mean by that is that like, yes it turns out like all of these systems have this baggage and have these roots that like if we dig deep enough we're gonna find the creepy crawlies that this metaphor got away from me that are under the lock, right? That like the baggage is not unique to one area of law. And I think often actually when we talk about the internet we're sort of, we all can come up from it, come to it from this perspective that things like everything is new and nothing has happened before. And I think that the argument here is that we should take the lessons that we've learned from all of this other stuff about how we don't like it when judges misunderstand technology and then come to the wrong decisions. Well, it turns out like they've been judges have been misunderstanding things since time immemorial, right? And the problems are not unique to technology but we can learn a lot from the scholarship that has already criticized these phenomenons and that's part of why I invoke Doug and Kennedy. So. Seems like to avoid falling into the trap you described if I was to write a set of terms of service or just guidelines for something I run, I would have to avoid any reference to any legal terminology whatsoever.  Even if I go to a really, you're like fourth grade playing English, it seems like I can't really avoid these terms. So what do I do? I mean, I do think you can avoid these terms. I actually think since I've started thinking about this, I actually think Twitter and Facebook have made huge progress on their content guidelines about removing legal terms. And part of that is because I think they've realized that the legal terms were not actually serving a good purpose for what they wanted to accomplish. But if you look at some of the Facebook's conduct guidelines, Facebook's conduct guidelines do not contain the word defame. Their terms of service might but their content guidelines don't. The word defame, they don't talk about defamation. Say harassment. Yeah, so but like harassment, like you usually need a word in front of harassment before you like get a very specific set of legal technical terms in a way that's different than defamation or libel or slander. So I do think it's possible to move away from this. And I actually think that like, speaking of half-baked ideas, forcing people to write their terms of service in like, like explain it to me like I'm five style might actually not be a bad idea for like forcing people to actually reckon with what they're putting in there. Because the reality is that people often are doing this in such a way that they're not spending a ton of time that as Jonathan suggested, it's sort of the fruit cake, right? It's like, you know, they grabbed it, they copied it from some others, say modified the parts that obviously should be modified and then like turned it over. And I know none of the lawyers in this room, this room would ever do that, but I've heard that it happens. And so, yeah, so I think that like, the process of actually having to sit down and think about what you mean is like really important. And being, you know, being intentional, like about the process by which you come up with the terms that govern your community and the behavior that's acceptable in your community, forces your instincts into tension with the sort of realistic focus of what you're doing. So I do think it's actually possible. It may be hard, but like, you know, so are a lot of things that are worth doing. Crystal. Thank you, this is a great talk. I find it really fascinating that I didn't really know anything about the, basically the more problematic meanings of the phrases like defamation and other types of things that you brought up. And so applying critical legal studies seems to be a really key part of this argument and realizing that legal talismans are an issue. So I'm wondering how you bridge the gap between people kind of glibly or not glibly using legal terms casually, whether or not it's in news reports or in terms of service or in other capacities online and get into the part where you actually start to see that it's problematic just from, not just from a legal sense in the fact that laws are complicated, legal terms are complicated, but they also have even more underlying issues beyond that. Yeah, I think there are some folks who write, like, it's hard because I think most critical legal scholarship would not necessarily characterize itself in the terms I have characterized it, the sort of defamiliarize the familiar or with the express purpose of sort of like complicating these terms. Although I think that that is one of the very powerful things it brings to the table is the sort of historical understanding of where these words come from. And it's actually something I think that people should learn more in law school. Thank you, John Hansen. So I do think like law school is actually one really good place for people to learn the history of these terms and it's sort of something that doesn't really happen in the traditional doctrinal subjects where everyone's so worried about making sure that you understand what strict scrutiny is, which is a First Amendment test that they like forget to talk about like, you know, why we have incitement law or like what defamation means. In fact, I'm pretty sure, my First Amendment course did not cover defamation very much at all. But so I think in terms of their, it is possible to write persuasively for the public on this, it's definitely hard because it can often seem like the, this historical perspective is really like, oh well that was the past and things are better now, right? Like when we're talking about like the history of defamation, oh like in 2014, they got rid of that part about women's chastity and like 2012 it's like homosexuality is no longer per se libelous, hooray! So it can seem sort of historical but I think, you know, to be trite, those who don't know history are doomed to repeat it, right? Like that we, you know, we think of these new terms and we come up with these new legal concepts and we make the same mistakes that we often have made before.  Hi, thanks. So this was super interesting. I really like, I work a lot with like critical legal studies and I have a question regarding the content that legal concepts carry which is sort of what like you're unpacking and like creates unpack and sort of like the more generalized content that people have about those talismans and how they use them to communicate and achieve things under daily lives which doesn't necessarily match the unpacked content which you're doing. And I wonder how you see the interaction of this like unpacked content which usually comes from the past and there's like constant construction of concepts that happen in their life that doesn't really have to be related and that actually leads to many legal outcomes that strategically people like in the headlines that you showed, I don't know, Apple or Twitter use and how you see like how they relate and how to match them up. That's a great question and I think one thing to be really careful of is not talk about the legal definition as the right one, right? Like I think especially lawyers have this idea that like we have found the one true meaning of this term and it comes from this history and I apologize if that's what I was sort of suggesting because that is one true meaning of the term but it is not the one true meaning of the term to sort of rule them all, right? Like cultural understandings of defamation actually carry huge amounts of weight in how we understand how these things work, right? Like we can't just ignore them because we like the legal one better or and this happens like straight up constantly in the content in discussions of free speech because people who talk about free speech are often using it as a shorthand for like I feel like my voice isn't being heard or I'm being denied due process by Twitter or whatever, right? Like and whether you agree with them factually or not, the sort of facile, well the government's not here so what are you complaining about is tense I think is just not necessarily constructive as a response. So I think it's really important to think about like popular definitions and legal definitions as both valid for different things and that like the legal definition is probably gonna win in court, right? That like that's you know the virtue of it is that courts apply legal definitions and knowledge of those legal definitions can be used very effectively as a source of authority in ways that can sort of silence more popular cultural constructions, right? That's the no copyright intended thing, no copyright infringement intended which is sort of a tag that people put on like fan works like fan fiction where they're using like other people's copyrighted characters that may or may not be copyrighted in ways that may or may not be fair use. Like we you know it is tempting as a lawyer to be like hey I don't have to pay attention to that cause that's not real copyright but cultural constructions matter and like how people think about the law matters in some cases more than what the law is. So I also think that like I think it's Jessica Selby who's been doing a lot of work on like what people mean when they talk about copyright and what creators mean when they talk about copyright and I think that is like that sort of more ethnographic work is a really powerful way to sort of create a to look at cultural consensus around what legal terms mean in a way that because of the constructions of academic knowledge gives it authority that's maybe comparable to legal authority. So I think that it's just to sort of sum up sort of rambling answer. It's just really important not to talk about one term as correct because one term might be enforceable with within the law and the cultural understandings of these terms are in many ways more impactful on people's daily life than the sort of reality of like what may be on the books.  Hi, thanks for that. That was, I got a lot of thoughts going through my head because I actually spent the morning looking at the community standards of three of the biggest social media platforms. Oh, well, awesome. Well, for me. And one of the things that actually I wanted to draw out and is what you began with by talking about the companies aren't providing us with a positive right. And one of the things that actually stood out for me is when you look at their community guidelines of the Twitter rules, they really invoke all this very flowery language about we are providing you with the opportunity to share and exchange in this bounty of goodness. And it seems like they would argue right that that is a positive right, just the ability to be able to go forth and prosper. And we can roll our eyes and we know it's a bit ridiculous, but I'm wondering if that has any kind of standing in terms of how that makes us think about this notion of rights, if that is, and if that has any kind of legal framing in that question. So one of the sort of, one of the things we talk, law school talks a lot about it is there's no right without a remedy, right? This idea that like without some sort of consequence for the violation of a right that like the right might as well not exist. And so although often like you're totally, you're obviously having just read them, but like you're totally correct. There's all this kind of flowery language about using our information service to spread, global peace and flowers and chocolate world wide and like pets and kittens or whatever, right? Like the, I mean the reality, and this is sort of where the cultural construction versus like legal reality kind of like butt heads, right? Is that if you look at the terms and not the community guidelines, what they're gonna say is that we can terminate your service at any time. And like if we do, your remedy is we'll pay you any money. You, we like, we'll give you the money we might owe you back, which like in the context of a free service is like, okay, great, so I get nothing, right? So without remedies, positive rights are not necessarily meaningful. And so, and this is something that comes up in all kinds of different forms of jurisprudence, but when like if we talk about these documents as sort of trying to give people a sense of foundational rights, then the question is actually like, what times like do they provide due process? Like, or like things that we could attempt to understand as due process rights, or like do they provide remedies for the sort of, for not being able to access the service anymore? And I'm gonna pretty much guess that the answer is probably no. So in the absence of a remedy, what is the right? Thanks Kendra so much for a lot of this. It's certainly your last question got me thinking a little bit about the alternatives. And it kind of led me to a place which I guess is going to be a question for you. And that's really how much is the role of the terms of service to sort of set like community values and standards and communicate. This is what we do here and give notice versus how much is simply like, oh, this is a shield in case angry person wants to litigate against us because we like arbitrarily booted him off, or hey, and it's kind of like what role, like should there be two separate things or something like that? Yeah, I think it's a great question. And I think actually many platforms have sort of gone to two separate things which sort of I've been maybe alluding to somewhat but community guidelines versus terms of service, right? And like how those differ. And I do think that I, I mean, I think terms of service are fundamentally legal documents and they're often drafted not always, like certainly not always, and like I don't mean to impugn the good, the good names of many of the lawyers to draft terms of service of which I am one. So the, like they are often drafted with an eye towards shielding from liability and not with the sort of foundational document kind of in mind. And I think that the, I have not seen, although I'm welcome like suggestions on where to look, people who have done the sort of foundational document thing in a way that did not, that passed the SNF test. Like Facebook sort of event at like global governance where like 1% of all Facebook users would have to vote on something, which is just like an enormous amount of people that was unrealistic in every way to think that they would engage with this issue, with the issues Facebook was pushing. So I do think like, yeah, there's sort of this two, this idea of splitting them off and these have two separate audiences and this is our legally binding one. But I mean, like if your community guidelines are all puppies and rainbows and your legally binding document is all like, you know, we reserve the right to terminate at any time. Like this is like, we can kick you off for any reason. You know, I think you may be trying to say one thing out of one side of your map and another thing out of the other side of your map and maybe some context that's totally appropriate and totally okay, but like you have to question whether that, whether that represents the values that you wanna try to promote. Like it's through like the intentional process of actually thinking about like, what is it we're trying to do here, right? And that there are costs to being intentional and thinking about values, right?  But like, if you think arbitration clauses are not, are bad, then like, that's maybe a decision you make. There are all of these decisions involved trade-offs. So I think that individual companies do need to reckon with like sort of the reality that we've constructed. And why I ask if the alternatives are worse is that like, in some sense, it's like, well, like, am I now asking companies to do the kind of soul searching that our legal government has, our democratically elected government has failed at, right? And I am, and I realize that that's like a pretty hard position for me to take on a lot of days and a lot of places because I don't necessarily trust companies, but at the moment, they're doing it, they're making value statements anyway, right? Whether I like it or not, and I'd rather they were more intentional about their values. So that's, I'm answering my own, are the alternatives worse question. I think we may be at time, or I don't know if do we have, so, and seeing no people with their hands strenuously up, I wanna thank you all so much for coming and I'm happy to talk to folks after.


Two men have been charged with first degree burglary in connection to a break-in and assault that occurred on March 9th. 21-year-old Francisco Cicina of Bemidji and 18-year-old Tristan Lee Fox of Bagley were arrested March 10th a day after another man, 24-year-old Joseph Devin Wood of Bemidji was arrested in connection to the same burglary. According to the criminal complaints against the three men, the victim spoke with Beltrami County Sheriff's deputies in the early morning of March 9th. The victim told deputies that he had been awoken by a loud bang near the front door of his home at the Pine Valley Mobile Home Park and saw three men pointing a strobe light and a red laser at him. The complaint says the victim was hit in the head with a blunt metal object, and then the victim's hands and feet were then bound with duct tape. The three men went through the home and allegedly took $2,000 cash in a jar of marijuana. Cicina and Fox were then located and arrested. Both men are scheduled to appear in Beltrami County Court on Monday. If you've enjoyed this segment of Lakeland News, please consider making a tax-deductible contribution to Lakeland Public Television.


When our audience looks at our figure for the very first time, one of the first things that they notice is the relative position of different data points on the X and Y axis. And so what data that we represent on that X and Y axis is critically important. In recent episodes of Code Club, I've been building out a series of versions of a slow plot, where on the X axis we put categorical data, two different months say, and on the Y axis, continuous data say the percent of people willing to receive a future COVID-19 vaccine. Today, what I want to experiment with is instead of putting those categorical or discrete months on the X axis, what happens if we put one response, so say responses in August of 2020 of whether or not people would receive a future vaccine, and on the Y axis put another continuous variable, say people's willingness to receive the COVID-19 vaccine as they reported it in October, two months after their initial being surveyed. So that's exactly what we're going to do today. Now, because a scatter plot is a little bit boring, we're going to kind of spice it up a bit, and we're going to put country labels next to the data so that we can see what country each point corresponds to. Let's dig into our studio and we'll get going on creating a labeled scatter plot in this episode of Code Club. You'll notice in my Git tab that I've already gone back into my commit history and created a special branch off of my highlight branch called labeled scatter. This is taking a commit from after we added the labels and cleaned up our X and Y axes a bit. You'll notice that I've also gone ahead and renamed my R script and TIFF file to remove the underscore slope because we're making scatter plots rather than slope plots, and I guess I couldn't call it underscore scatter, but I want to try to keep things as clean as possible. So we're loading three libraries right off the bat, tidyverse showtext and ggtext. Tidyverse gets us things like Deplier and ggplot2. Showtext allows me to customize the fonts that I'm using in my figure. I've really grown to like seeing patril1 for the title and Montserrat for all the other text labeling in the figure. Ggtext allows us to also incorporate markdown or HTML and CSS into the text of our figure. We'll maybe talk about that later. And then we read in the data, we do some cleaning up, making it tidy. And then here in the data chunk where we're generating the figure, we then output the slope plot using geomline. So one of the first things we'll do is go ahead and remove that. And then we output it as a TIFF file. And so here again, I need to remove that underscore slope. Again, this is the figure that we're starting out with. One of the challenges that we've had is how do you represent two time points from 15 different countries? And as I said, with this slope plot, we're going to convert this to a scatter plot so that we'll have August on the x-axis and October on the y-axis. And then each country will be a different point. If we look at data, the data frame that's being fed into the pipeline to build out the plot, you'll notice that it's tidy for the purposes of building a slope plot, right? Where we have country representing each different line and each color, month being the position on the x-axis and percent being the position on the y-axis. Now, in contrast, what I'd rather have is a column for August and a column for October, rather than separate columns for month and percent. And I can achieve that by running everything but the pivot longer line. So now if I remove that pivot longer line, you'll notice I have a column for country, August and October. And so I could easily map August to x, October to y, and then country or whatever to color or some other variable. So I'm going to go ahead now and remove that pivot longer and we'll load data and confirm that data looks the way we expect it to. So that's great. Turning to my ggplot pipeline, I now need to make a series of modifications to turn my slope plot into a scatter plot. So x, I'm going to put August, y, I'll put October. And I don't need the group because I'm not using lines. I'll go ahead and leave the color mapping to country, country to color. And I need to change the gmline now from line to gmpoint. So I get that scatter plot. I also then need to change my x and y access labels. So I need to say percent willing to receive the vaccine in, and again this is y, so it's going to be October 2020. And I'm going to use the same title for my x and this will be August. And because it's a really long title, I'm going to go ahead and put in a line break between receive and vaccine using the br tag from html. And so what that reminds me then is that down here in my theming, I need to go ahead and do access dot title dot x equals element markdown and access dot title dot y equals element markdown. So this looks good as a start, but one of the things that I'm struggling with is trying to come up with context for where the points lie. And I'm noticing that the y limits are a bit different than the x limits, right? And so, you know, 60-60 is above the diagonal that I might intuitively draw across this plotting panel. And so what I'd like to do is have the x limits and the y limits be the same. And so then we could also begin to think about drawing a diagonal line so that we know if points are below that line, then the intention has dropped between August and October. And if it's above that line, then intention has increased. To start improving the appearance of this figure, I'm going to go ahead and remove the color equals country. That way, all of my points will be black. Also to get my x and y axes to have the same range, I'm going to use chord Cartesian. And we'll do x limb from 50 to 100. And y limb from 50 to 100 as well. So we no longer have the legend because all of our points are the same color. Our x and y axes go from 50 to 100. But one of the things you'll notice is that this plotting window is not square. And I can kind of see this in my preview software here, where if I highlight the whole thing with a rectangular selection, that the x direction is about 1278 pixels, and the y is 935. I would like this to be say 935 by 935. So it's square. And that way then the distance between 50 and 100 and 50 and 100 for the y axis and x axis would be the same. I can achieve that by modifying my chord Cartesian to instead be chord fixed. And what chord fixed does is it allows you to basically set the aspect ratio of your x and y axes. The default is an aspect ratio of one. So now we see that our plotting window is square like we wanted. But unfortunately, the formatting of our title got kind of screwed up, so that we don't have enough margin between globally and the plotting window. And the top of the title, the first the top half of the first line is getting truncated. So we'll go ahead and add some margin. We might also make the font a little bit bigger or smaller. So I'm going to go ahead and make the bottom 20, the top let's do 10. And maybe I'll make the size of my font 25. Before we start digging around and playing with the labeling of our points, I want to go ahead and make our plotting window a little bit cleaner. I want to put in a line for the y axis and the x axis, probably want to go ahead and remove those tick marks. I'm going to get rid of that grid lined background. Again, we can do all that here in the theme section of our figure. And we'll do panel dot background equals element rect, fill equals white. So that's an hexadecimal six f's. We'll also do axis dot ticks equals element blank. And then axis dot line, element line. So I think this looks a lot simpler and attractive than what we had before with the default theme, which I think the default theme is theme gray. I want to go ahead and put in a diagonal line from 50 50 up to 100, so that my audience can get a better sense of if the points are below that diagonal or above the diagonal, I can do that by coming way back up to the top of my gg plot chunk. And actually before geom point, I'm going to add geom ab line. And geom ab line draws a line with a slope and an intercept. So my slope will be one intercept equals zero. And then color, I'm going to make it a light gray, probably with six a is here. I want it to be light gray. And I also want it to be a thin line. So I'll do size equals 0.25, because I don't want it to detract from the points that are there. I want the points to really stick out. And now we can see that we have this thin gray line. And so hopefully the audience can see that if it's below the line, then it's decreasing intention. And if it's above the line up into the left, then those are countries that are increasing in their intention to receive the vaccine. You know what, I might want to go ahead and add a little legend in here to indicate that if you're above and to the left of the line, you're increasing. And if you're below and to the right of the line, then you're decreasing an intention. How can we do that? Well, we can do that by creating a separate data frame that I will call legend. And this will be a table where I'll have x coordinates, a vector of x positions, a vector of y positions. And then I'll have my label, and then I'll have two labels, which will be increasing intention. And I'll also have decreasing intention. And I'll go ahead and put a line break in here. I'll use the backslash n as another way to demonstrate how you can add line breaks. So for my x and y positions, for the x position for increasing, again, that's to the top and to the right. So for my x position for increasing intention, I'll put 90. The y position I'll put as 100. And then my x position for decreasing intention, I'll put that as 100. And then we'll do 90 for the y position there. We'll go ahead and load that as legend. And then down here at the bottom, I'm going to go ahead and add a geom text. So we'll say data equals legend. And we'll also then have mapping equals as x will equal x, y will equal y label will equal label. And we will also then do inherit.as equals false. So those labels are there. But they've got issues, right? So we need to decrease the size of the label, decrease the line height, perhaps make them left justified and go ahead and make them the same light gray color as our line. So for my color, I can go ahead and add in that hexadecimal for the line color, that same light gray, my h just, I'll do zero for being left justified. And my line height, I can make one. And my size, I can make let's start with three. I'm noticing my decreasing intention is getting chopped off by the right side of the plot. It's also down a little too far. So I think I want to bring that up. And I'm going to turn off the clipping in our chord fixed, because that clipping is removing the right half of that label. So again, we can come back up here to legend, and the y position for decreasing should be 95. And then chord fixed, we'll go ahead and do clip equals off. So I think our legend does a nice job of clarifying for our audience, what points on either side of the line represent, you know, perhaps it's not quite as easy to see as those slope plots. But you know, it's another way of looking at the data. The next thing that I want to do is I want to go ahead and add labels to each of these 15 points to indicate what country they represent. Now these points are kind of on top of each other. So the typical GM label, I suspect is going to look like a bit of a mess. But we'll start there. And then we'll see if we can't do better. I'm going to come right back up to GM point. And after GM point, I'll go ahead and add GM label. And I will add to my GG plot, I'll say label equals country. So yeah, that's a bit of a disaster. What I want to do is I want to move those labels off of the points and have an arrow or a line pointing back to the data. And if you've been watching previous episodes, you know that we learned about a really cool function called GM label repel. GM label repel will move the labels away from the data and away from each other so that we can then have lines pointing back to the data so that we can clearly label what each point represents. To use GM label repel, we need to add another library, which is library GG repel. So this isn't perfect, but it's looking better than what we could do with GM label. One thing that's not immediately clear is say these two points here, which is for Italy, which is for USA, the Canada is kind of on top of the point. We need to clean this up a bit. Okay. So the first thing that I want to do is I want to come back up to a GM label repel, and we're going to start adding some arguments to make it look more attractive. The first that I'm going to do is min segment length, and I'll set it equal to zero. And so that we'll put a segment length between every label and every point. The other argument I'm going to add is max overlaps. And I will say INF for infinite so that I don't want to lose any of my labels if the labels happened overlap with each other. So now we have a much more clear indication of what point corresponds to each country in here in the middle between kind of like UK, Japan, Canada, it's a little bit jumbled. So it would be nice if those were pulled apart a little bit. And so one thing I might do is decrease the size of the font. I'm up here in GM label repel. And again, because I'm going to start getting a lot of arguments, I'm going to break these apart on different lines. So I'll start with say size equals three, also do family equals Montserrat, to make sure I've got my special font there. So that definitely made the size of the country labels a bit smaller. We still have overlapping labels here between Japan, Canada and the UK, which I'm not a big fan of. One thing that we could think about doing is going back to GM text repel. But I kind of like having the white background. We've seen this before. Because what if one of these labels say like India here falls on the line, I don't want that line to run through India.  So I think what I'll do is I'll go ahead and remove the border around the name. I'll make the padding smaller. And we'll go from there. So again, up here in GM label repel, and we'll do label dot size. So that's the size of the border. I'll make that zero. And then label dot padding. I'll make that. Let's do that with say zero as well. And then label dot R, which is the radius. This doesn't really matter. I'll make that zero as well. And so now what we see is that we have much clear labeling of our points. And you can definitely see that that line no longer goes through India, right? And so that's nice. But what I notice is that the line goes kind of right into the letter. And so I would like to have a little bit of padding, I think, between the name and the start of the line. So maybe instead of zero for padding, let's do 0.1. So I think this looks a lot cleaner without the border for our individual labels. And it's clear now what country corresponds to each of the points. And I think this of all the options that we've played with, I think this really does a nice job of connecting each country with the actual data in a fairly clean and presentable manner. I think this looks pretty good. There might be a little tweaks here and there that we might play with to try to make it look a little bit nicer. But I'm pretty happy with this. One final thing that I'm going to do just because it annoys me is that I forgot to change the font family for increasing intention and decreasing intention. Again, we can come back up here into geome text and do family equals Montserrat. So again, I feel like this is a pretty attractive version of our data. You'll notice it's all black and white. I don't feel the need to add any color here. And I think, you know, we could add color to perhaps say increasing intention and label label those red and decreasing blue again. But I feel like, you know, it's pretty clean. And I think it's pretty simple and attractive and conveys the message. Now, is this better than a slope plot or a dumbbell chart? I don't know. You tell me. I feel like we're at kind of a happy middle ground in terms of number of points and countries. And that if we had many more, say 100 countries, then this would get really busy. And then we'd probably want to go back to some of those ideas from the highlighting episode where we'd really want to focus on individual countries, perhaps we might have 100 countries. And then we'd want to highlight, you know, the, you know, most wealthy or the most poor countries to kind of make some story about those countries and whether they're intending to receive the COVID vaccine, right? So again, the appropriate visual really depends on the data on the question on the story that you're trying to tell. Anyway, I hope you keep practicing with this data. And by all means, if you're interested in how you can be following along using the branching strategy that I'm using, feel free to check out this episode that I've got linked over here. And we'll see you next time for another episode of Code Club.


I will now continue looking at a goodness of fit test statistic for Poisson regression. So we would like to test if a particular model that we are assuming really does fit the data or if you may want to extend our model and include maybe more covariates in our model. We first of all look at another example, so I will first of all talk that through and then I will be discussing goodness of fit test statistics, so the Pearson-Keyer square test and the log likelihood ratio test. So we've got the example of recall of stressful events, so example three, and basically we would like to first of all start with something simple again, a simple Poisson model without any covariates, but then extend this to an example with covariates. So basically participants from a randomized study were asked if they can recall any stressful events over the last, in fact, 18 months, and if yes, in which months, when did this particular stressful event happen? And we then wanted to look at the number of stressful events that people were able to recall and look at their distribution according to these 18 months. So we had 147 stressful events recorded in total. So the H0 hypothesis is that first of all we start with something very simple, start with some vague sort of conservative attitude, conservative assumption that these events are uniformly distributed over time. That basically means that H0 follows the equi-probable model, that all these probabilities across months are the same. So we have an event occurrence of 1 over 18, so a particular event can occur in any given month. So it would be a percentage of 0.055 or in terms of percentage terms, it's a 5.5%. So we would expect about just over 5% of all events to happen per month. Looking at the actual count data, at the actual data that was recorded, so we've got first of all months, ranging from 1 to 18, and then we've got the actual count data, so the actual number of events that were recorded per month and then the percentage that relates to the actual count variable. So we can already see that some counts are significantly or are actually higher than 5.5%, and some are actually a bit lower than 5.5%. So just looking at the data, we may already conclude that there is some divergence between or discrepancy between the observed values and what we would expect to see based on our equi-probable model. Let's look at the evaluation of the Poisson model doing this more formally. So we would be using the Pearson-Kyles-Square test and the deviants or also called the log likelihood ratio test for Poisson regression. Basically both are goodness-of-fit test statistics and they basically compare two models. One for the current model, the model that we have attained, so in this particular case the equi-probable model, and then we compare this model with this so-called saturated model, i.e. the model that is larger, that is the saturated model, that is the model that fits the data perfectly and that explains all of the viability. That basically means we are comparing observed and expected frequencies. Looking at the Pearson and the log likelihood ratio test statistics, basically we say that if H0 is true, that means that if the equi-probable model actually holds, then we would expect or roll a distribution of 147 times 1 over 18 so that we would expect an expected frequency of 8 points, just over 8 per month basically. So we basically have one parameter model that we would like to estimate and that would be just over 8 events per month. So we can compare the observed counts with the expected count per month and we can see that obviously some counts are quite a bit higher than maybe 8 and some counts per month are quite a bit lower than 8 than what we would expect. Looking at the Pearson-Kal square test, it allows us basically to compare the observed and the expected frequencies and you may have come across the Pearson-Kal square test when testing associations between two categorical variables. So it's the same principle effectively here. We are trying to compare observed and expected frequencies and basically it allows us to look at the sum of the standardized residuals in squared terms and we can calculate this particular statistics for our example. So basically you just have to plug in the numbers for each cell basically so we've got 18 cells in total and for this particular example recall of stressful events we will have the KAL square test statistic of 45.4 and we now need to compare this to the value from a KAL square distribution so basically the assumption is that if H-node is true if indeed the actual probable model holds then this test statistic will follow the KAL square distribution so we can compare it with the distribution from the KAL square tables for example. So for that we need to have the degrees of freedom which is defined as the number of cells minus the number of model parameters which is the number of cells is 18 so capital C minus the model parameters here in this particular case for the simple model is just 1 because we have got only alpha that we need to estimate so we've got the KAL square test statistic of 45.4 with basically 17 degrees of freedom so 18 minus 1 at the 5% significance level looking at the KAL square table the value we've got 27.6 from the table and also we have got that basically associates with the p-value of really rather small 0.001 so that means the value we would then reject H-node based on those characteristics. Conclusion is that there is strong evidence that the equivalent model does not fit the data so just looking at the actual data from our table we already looked at observed and expected frequencies and we already saw some discrepancy but exactly how significant the discrepancy is the discrepancy between observed and expected values we can then formally test this for example here with the Pearson KAL square test and we concluded that the difference is rather large so it is not just due to chance and that the data does not follow the equi-probable model so we have to do probably something else to improve this particular model likewise looking at the log likelihood ratio test statistics for Poisson regression we can now also use this test statistics to again compare observed and expected frequencies so basically the formula here gives the log likelihood ratio test statistic and you can plug in the numbers the observed and the expected frequencies and that is again a measure of the fit of the model so the goodness of fit test statistics and again similarly to before if H-node is true then actually this particular log likelihood test statistics would follow a KAL square distribution so again we need to define the degrees of freedom which again is the number of cells minus the number of model parameters so again it's 17 for our example and the log likelihood ratio test statistics for our example is 50.8 if you plug in the numbers on 17 degrees of freedom so you've got a p-value of less than 0.001 again and you would again reject H-node so basically in the same way as with the Pearson-KAL square test so the conclusion is again the same there is strong evidence that the equi-probable model does not fit the data so we basically now need to use this information to take this forward and to think about possibly fitting a more sophisticated model maybe including another covariate to allow for differences between number of months just a couple of remarks about the Pearson-KAL square test and the log likelihood ratio test they are basically asymptotically equivalent so basically they are relying on a large sample and you would expect them to be giving you basically the same or very similar results if they're not similar this could just simply be an indication that the actual approximation doesn't actually hold and also just to note that for fixed degrees of freedom so when n increases for larger samples that the distribution of the Pearson-KAL square test usually converges to the actual KAL square distribution and also it does it more quickly than the log likelihood ratio test and also to note that the KAL square approximation is usually relatively poor or not appropriate if any of the expected cells are less than 5


What's up everyone? Take down here. Welcome back to another video. Today we're gonna be doing a Toys R Us haul. Let's get right into this Now before I get any of those comments saying Toys R Us doesn't exist anymore It does here in Canada, and I am extremely happy for that Recently we did stop in at Toys R Us and we noticed that the fungal pops and the soda pops are all on Clearance for some reason. I'm not sure if they're getting different shipments in or they want to clear out some of the old stock and Eventually get in new stock. They didn't really tell us but whenever we seen stuff on clearance We decided to go through and see what we could add to our collection So first let's start off with the Funko soda figures Now I did see that they did have a lot but a lot of stuff that was just not of interest to me The figures are regular $20 and I believe all of them were on sale for $13 now two of them We were charged full price, but we did go back in with the receipt and get the difference back So the first one here is the Phoenix figure And I didn't notice until we got home that this is the Funko exclusive 2022 Fall Convention limited edition So that's actually pretty cool that we found that at the Toys R Us But we didn't get to chase but the figure itself does look pretty cool of Phoenix So that is a nice addition to our collection Next I did pick up another Marvel one for myself and that is black Panther Now this one here Unfortunately, we still did not get a chase, but this figure does look amazing check that out That looks really cool the detail just looks amazing So those are the two that I did get for myself my girlfriend Ashley did get to as well and the first one here is the holiday stitch She's has a huge obsession with stitch and we're looking for a lot more Funko's and collectibles for her stitch collection. So finding this at a clearance price was also amazing and Again, it wasn't a chase, but that does look amazing. Just stitch in a Santa costume. That is just adorable So another nice figure for her shelf and then she also is a huge fan of the Grinch so we've seen this and obviously we did have to get this one as well and It looks cute too So I do like these soda figures. They're just different than the Funko figures But they also are different so not only do you get the collectible can but you also get the collectible figure inside and These just looked amazing. So at a clearance price Definitely worth getting now. We did get some regular Funko's as well now. Just gonna open up the bag here and pull it the first one So this one here is a Disney one for my girlfriend Ashley. She absolutely loves Disney She wants to grow her Disney Funko collection and she does also like horror So this one here is just right up her alley and it is the Jack Skeleton. I'm not sure what you would say This edition is somebody comment down below I'm not familiar with it, but she did see this and did have to get it And I believe the Funko's were regular if I remember correctly some were anywhere from $16 to $19 This one here I think most of the Funko's were around $11 on clearance, which still is a great price So she did pick up this one here. The next one is one that I picked up and I was actually extremely happy I did and that is super shredder This one here is the special edition and surprisingly it has a lot of weight to it Now I was deciding on if I really wanted this one because eventually I do want to get all Of the Ninja Turtles as you can see on the back here. I do want them all So seeing this one here for I think this one was actually on clearance for $13, but it was regular $25 or around $25 so finding this one here at Toys R Us on clearance was definitely a great find for my personal collection Next is another one for Ashley as I mentioned a minute ago. She does like horror related stuff She does have a bunch of horror Funko's and this is one for her which is Hannibal so I think this one here was actually on clearance for again $11 which is a great price for Funko's especially if they're ones you want to add to your collection and You we do get a lot of Funko's so it definitely is great to get them on clearance as well So this one here was also for Ashley and it was actually the cheapest one that was on clearance And that is machine gun Kelly here with the guitar. It just looks really cool very punk very machine gun Kelly This one here was actually $10.50 on clearance, which was a great price She wanted it so it was definitely good to get it on clearance and the last one here is for myself now I was actually going to buy a whole bunch of these ones to sell because I am a reseller as well But I decided not to I decided just to get one for my personal collection and that is president's low key It does have the alligator in there. That is just amazing I didn't realize until I got home is a 2022 Winter convention limited edition doesn't have the sticker here. These ones here are regular at Toys R Us around $20 they do sell on eBay for around $25 we paid $11 for this so that was a great price Which is why I wanted to buy the five other ones that they had to resell on my eBay But I decided just to get one for the personal collection just because I don't really need inventory right now But this was definitely a great find for my personal collection So this was everything that we recently picked up from Toys R Us and it was awesome Just to go in and just check things out and notice something on clearance something that not only do we enjoy But we also collect and finding some really great ones They did have a ton of different ones in their shelves because toilets toys are us has a ton of Funkos usually just like Piles and piles of them so just to go through and just see what they have was awesome And we did pick up quite a bit for a personal collection So in my opinion it was a win-win and everything was on clearance Most of the stuff was around $20 regular price We paid between $11 to $13 each for the items, so I'm gonna leave this video here. Hope you guys enjoyed I'll see you guys in the next video. Please take care peace