talk is technology did not create misinformation and it's not gonna solve it either. Basically, yep, there we go. Try not to waste your time. There's a lot of research to be done. A lot of people are doing a lot of research. A lot of it is fairly awkward.  There's a lot of wasted money. So I'm here to talk through what we can actually do, where hackers can help, and how to stop putting good money after bad. Okay, so, who am I? As mentioned before, I'm the lead technologist at the Duke University Reporters Lab in the Sanford School of Public Policy because academics like longer titles and apparently that makes us more important. I'm a journalist and computer scientist. I've worked as a reporter and photojournalist around the world. I live in New York City. And I've been working on misinformation since about 2015. Previously, I lived and worked over here in Bosnia with a couple of the folks up front. And so I've been dealing with this from a lot of different origins and a lot of different places. Okay, first thing we're gonna do is go over some nomenclature. Words matter.  They get in the way. People get really pissy about them. However, a couple of them are really important, including misinformation. So what this is, is it's false information and it's spread without malice. This is your aunt. This is your uncle.  This is you. It's me sometimes. I've definitely done it. Spreading misinformation on accident. You think it's real. You think what you're sharing is important. And then what you end up doing is pushing it out on accident. Second one is gonna be disinformation. This is the same information, but its intention is different. So instead of accidental, these are the people who are creating it who know it's false. This is the disinformation that we're discussing. And when it comes to fact checking, there's two variations. So this usually goes over with journalists, but I'm just gonna make it, just go over it real quickly for you guys so that if you hear it in a couple different contexts, it makes a little bit of sense. The first type of fact checking that really existed within journalism in general is internal fact checking. And what that is, is you do an investigation, you write a piece, you spend weeks, you spend months, you spend years on it, and then somebody else in your organization at your publication, what have you, goes through and fact by fact tries to reconfirm a lot of this, right? So you can call people, you can make sure that the stats add up, et cetera. The journalist hands over their sources and you run through it. Sometime in the mid 90s, people started considering what external fact checking is. So taking those techniques, but flipping them out and using them on people who have made claims in public, politicians, scientists, what have you. And that's the type of fact checking I'm gonna be talking about today. It's the type you've been hearing about for the last decade. It's the type that's really been increasing in size around the world. There's over 300 fact checkers these days. So, can technology solve misinformation? No, it cannot. There is no algorithm, there is no AI model, there is no anything that is going to catch all misinformation and make truth valid. And what is truth anyways, but let's not get into the universal philosophies here. But it's not. Anybody that says this is going to fix misinformation on Facebook, including Facebook by the way, is wrong. Now, is misinformation caused by technology? This is the problem, it is not. So that's a good thing, right? You can't, you don't have to take full responsibility for creating misinformation with technology because it's always been around. People have lied since the beginning of time, the person standing in this, running a stall in Athens says to the customer that the person next to him is gonna cheat him and their bread is better. However, did technology make misinformation much, much worse? Yes, there is a burden on technologists, on programmers, on people who increase this spread. Unfortunately, this is a problem. It sucks, I know. But what can we do? Let's talk about the types of misinformation. Let's talk about why technology is not gonna solve this stuff. It's subtle. Okay, first, what you can get is really obvious and really bad misinformation. This is the stuff that sometimes you can catch. This is stuff everybody pretty much ignores except the true believers in the first place. So what you have is something like this. The food run is out. New York Times is accused of normalizing cannibalism. This was created by Sputnik, the propaganda piece from Russia that is banned in Europe. I found out this morning when I tried to visit their website, I forgot about that. It has been spread by info wars within the US.  Like almost nobody. This is the easy stuff, here's another one. Joe Biden obviously did not resign from the White House. This was spread on Facebook. It was published by Politifact. Third one, public health of Scotland will no longer publish COVID-19 data because somehow it spreads AIDS? I, again, this is just random shit, right? This is coming out. And so this stuff isn't what we have to worry about when we talk about misinformation and fact checking in general. This stuff is easy. This stuff is, you know, whatever. You can dismiss this. It's all mostly spread on fringe organizations anyways. So that's not a problem. This is where we start to get in trouble. These are also fairly obvious, but they're way more dangerous, right? Turkish observers, US and EU hindering Russia-Turkey efforts to solve food crisis and mediate peace. This is also from Sputnik. This is trying to actively screw stuff up, right? This is trying to actively mess with peace processes, with food.  Sorry, I use profanity a lot.  All right, this is another one. This is actively spreading LGBT hate across the US. This came from, I forget exactly what publication, but it was a fairly well-known one in the US, on the far right.  But it sounds like it could be real. This is another one. In the same vein, this is from full fact in the UK. And then we get to the really tough stuff. This is the place where people don't realize why misinformation is so hard for computers to understand. This is the subtle stuff. This is the stuff that sounds convincing. Politicians are professional obfuscators. That is their job. When they get up on stage, when a newspaper publishes, their job is to make everyone in the audience believe they heard what they wanted to hear, right? And so when you come to town to this sort of stuff, there's different ways of looking at it in the first place, and then people will believe it a little bit, and that's where you get into danger. So for instance, social media posts make unsupported claims about Zelensky's income and net worth. This is trying to discredit the entire Ukrainian attack, right? This is trying to subtly make people a little less sure that they should be supporting one side or the other. Same here. This was published. I actually, before I saw the fact check, I heard multiple very smart people I know discuss this exact piece of misinformation as a way to show how subtle and smart Putin is by releasing this and showing he's siding with Pakistan, right? You even see it says RT. Whoever made this is good. Of course, this isn't true. Somebody fact-checked this. The original image, if you do a basic reverse image search, is it's Putin on a massive video call with like 30 ministers or something like that. But whoever released this was trying to push a very specific agenda by making something seem convincing. It's not aliens take over the world. It's Russia's aligning itself with very specific parts of the world. And this is another one. This is from Nigeria. Every hour, 50 people die as a result of diseases related to poor toilet hygiene. The sad part is it's actually worse. It's actually 61. And so they're even underplaying it. And this is the really important thing that I want to get to. The point of misinformation in general, and this is why technology is not going to solve it, is that it is to convince you to trust nothing. It's not to convince you of a single individual fact. It's not to convince anybody in this audience that a very specific conspiracy theory is correct.  And everybody in this audience is susceptible to this.  It is everyone. I feel it's to promote apathy and benign neglect. Like, I don't care anymore, screw it. And that's why subtle just taking down and fact checking individual things is really, it's a band-aid, but it's not going to, you know, it's not a suture. And so you really, really have to think about it this way. It's like, I can write an algorithm that'll fact check this one piece of information. Great. That is not the problem. Thanks. And it's probably only going to be for a very specific silo, a very specific type of topic that you can write an algorithm for. General intelligence is not coming along, right? So let's do something and take a look at the current structure of fact checking, this sort of whack-a-mole. I want to say first, this is important. As I said, it is a band-aid. It's going to stem some blood, but it's not going to stop it in the long run. However, I'm not even sure you can fix it, right? I'm not even sure that this problem is fixable, but maybe we can drop it down just a little, right? So here's one fact check. And this is one of the big problems that you get when you do this sort of thing. Fact checkers are pedantic. They like to be very specific, which computers are very good at, too. The problem is you can miss context. Now, these people are my really good friends. They're full fact in the UK. They're one of the best fact checkers in the world. However, they fall into this trap as well. Sure. The picture wasn't from Davos 22, as it claimed. And Davos isn't technically a climate change conference. However, the point is that they're all still, that everyone's still flying private planes all the way over, which is ridiculous, right?  It comes up at these meetings. So is this context explored in the story? Yes, but are people going to scroll past this? Probably not. Here's another one. Ted Cruz said some stuff about James Comey that Obama appointed him, right? But that he couldn't vet anything. This is factcheck.org in the US. And again, they're missing these subtleties about what he's trying to get at. He's trying to cede a little doubt about everything, right? That James Comey in the US, if you remember the stories, can't be trusted anymore. But they're focusing on the very specific details about it and just trying to knock down this one fact while ignoring a larger narrative. And sometimes they get it right, but sometimes they don't. Here's a last one. Does Ron DeSantis oppose any background checks on guns, even for violent criminals? They rated it mostly false. The problem with this is that Ron DeSantis is the governor of Florida. Actually, it's pretty much true because he supports a constitutional amendment that would remove all background checks in the US. So yes, yes, he didn't specifically say that he doesn't support background checks or that he opposes them. But if you look at just a little bit further down the path, he actually does, right? And so you get these sorts of problems. And I'm running through my slides a little quick, but I imagine there's going to be a lot of time for Q&A. So if you guys have questions, please feel free to think of them and come up to me later. OK. So hi, everyone. I'm Mr. Happy. Fact-checking is, I've just been standing up here talking about fact-checking for a long time about how it's completely screwed over, how nothing's going to happen, how nothing can work. So what can we actually do? What can we actually do? What can technologists in this room? What can hackers in this room? What can academics in this room? What can just really smart people, which I'm assuming you all are, if you decided to sit in this camp for like six days at least and hang out with each other? What can we all do? Well, there's a couple options. But my favorite one is this. So I've been working on this sort of technology for a while. Not necessarily the tracking, but the collecting of misinformation and tracking of the misinformation itself. We have huge databases all over the world of pretty much every fact-check that's been made since about 2015. I have one. Google has another. They've actually, yeah, everybody hates on the big companies. And don't get me wrong, I will, too. But Google has been heavily supportive on this. And they mostly use it for putting a fact-checking like their search results. But it's a huge database. And it covers almost every country in the world where a fact-check's been published.  My question, though, is instead of just tracking how a single piece of misinformation spreads, how can we track who's making this? There's this really interesting statistic from the Center for Countering Digital Hate that came out in 2021. In February and early March, they tracked misinformation on Twitter and Facebook. And 65% of it, around COVID vaccines, I should say, 65% of it was created by 12 people. 12 total people created 65% of the misinformation around COVID, the lies. What's this called?  They examined, like, I don't know, a million-plus posts. And on Facebook, it was like 73. Twitter was another ridiculously large number. But if you average it out, it came to this. If there's only 12 people doing this, maybe it's not as big of a problem as we think, right? These are people with an agenda.   Like, it seems, to me, it seems sometimes, that it feels like. So what I'm curious is, are these profiles actually trackable? Not just from when they publish one thing and see how that spreads, which is important. And a lot of people are working on it, myself included. However, I want to see what sort of influence a single account, a single maybe person, has on the wider infrastructure.  And that's something that people are struggling with. Because it's hard. This is hard stuff. Has anybody in here ever tried to just drink from the Twitter fire hose? Anybody? Yeah? No? Nobody? OK. So if you ever have, I mean, I'm talking back in like 2010, was the last time I tried it because it was so much.  It's literally called a fire hose. And you will be pushed up against the wall. There's so much data coming through every single second that is literally impossible for a normal researcher to manage. So you work on small data sets. That is a problem. Because we need to figure out how we can more quickly analyze these things, maybe not store them, but really start looking into stuff. So a couple of the other problems that you're going to be finding. What is even misinformation to a computer? Context is important. If a politician stands up in the middle of the day and says the horrible thing, the horrible tragedy that happened this morning, what's the tragedy? What's this morning? Who's he talking to? Where's he at? This can all be wildly different, right? There could have been a train collapse in Barcelona and also a wildfire in California. He's probably talking about the train. If he's in Europe, he's probably talking about the wildfire. If he's on the West Coast of the US. But computers are not going to be able to tell these subtle differences, right? The audience, you guys are fine. Computers can't figure this out. AI is terrible at this. What is a tragedy? I don't know. I mean, it depends. The definition is very fungible to a lot of people. To everybody in this room, you probably have a slightly different definition. And then you get to the biggest problem that I've been working on, which is matching claims to something you've seen before. So you can start comparing and tracking that, right? Because especially, well, English is my expertise. But I know, and this is true in pretty much every language in the world, is that a piece of information can be reworded and restated in a totally different set of vocabulary, right? So someone can completely switch it around, completely change the words. And now if you're doing a word embedding model, you're pretty much screwed at that point. Because maybe a couple percentages off can infer some stuff. But as soon as word defect gets a completely different sentence, it's going to start going utterly sideways. GPT-3, if anybody wants to ask about that, is not good at this. It's actually terrible at this. Because it doesn't actually reason. It just looks at huge models. And when you get to the next point, which is recognizing new claims you haven't seen, it completely fails. Pretty much everything completely fails. Because you can't tell if something's a lie automatically. It's not just statistics, right? It's people saying various restating reasons for a war 300 years ago. So that's going to be the biggest struggle that we're running into if we can even get to the content in the first place, because private groups are screwing all of this up for us. I love encryption. Encryption is awesome. It really makes it hard to try to figure out who is saying what, and where, and when. So you have small WhatsApp groups, or large WhatsApp groups that have very specific membership requirements. Still is the problem here. Telegram a little less so, because it's a little bit easier. That's billowing. But trying to get into these, where the pieces of misinformation, my theory is, are often created and sort of battle tested before they get spread to the wider community is an incredibly difficult prospect. One that the companies don't really want to work on, because that's not in their benefit. People should be able to talk privately. I agree completely. I'm in a bunch of signal groups. I mean, heck, the camp that I'm here with all started as a private signal group. We wouldn't let anybody in. But it's an ethical dilemma. Feel free to ask a question about that if you want to talk about it a little bit more, or find me later. But at what point is misinformation worth invading privacy? I don't know. A lot of people in here, I imagine, would say never. But I'd ask you to think about that for about three seconds longer, and just sort of roll it around in your head. So why do we even want to do this, though? What's the purpose? What's the point? If we're never going to change anything anyways, what is the point of all of this? What is the point of trying to write algorithms to track this? What is the point of trying to figure out who is saying what? Well, it's really standard journalism questions. It's who's making it? Who is doing this? Who's paying for it, which is often the most important, follow the money? Why are they doing this? What is their purpose? Are they trying to discredit somebody? Are they trying to just throw chaos? A lot of reports have shown that, again, I'm American, so I'm going to speak to this. I apologize.  It drives me absolutely insane, but I'll just use this one. Actually, no, it works for Brexit, too. So the Chinese that it was found out, I was talking to a researcher just a couple of days ago, didn't support one candidate or another. They didn't support one person or another. In fact, they supported both sides. They ran campaigns to have them fight each other. I mean, Russia did this, too. They would set up counter-surveillance, or not, counter-protest against a protest they had also seated and set up. This is, my water bottle's annoying me. I'm sorry, guys, they would set up a protest, then set up a counter-protest, and then watch each other, then fight each other. That point was just so discord. It wasn't to do anything. It was just throw wrench in it, and that is a huge problem. When it comes to this, can we cut off this head? Can we stop these people ahead of time? Can we, censorship is another touchy subject, but can we sort of proactively censor these people after, again, after they've been doing a lot of things, but before they really build up the momentum and start huge, massive campaigns? This is going to be a problem. They start small, and then they gain credibility. They gain traction, they gain followers, and then it's so big that the major social media companies start asking questions on whether or not they should be taken down in the first place. This is a struggle. So can we cut this off? Can we stop this at the source? So my idea, and I want help with this, please help me, please talk to me if you want to work on this, is can we fingerprint these fact check, these misinformation and lie creators? Is there a way that we can come up with a system that looks at the vocabulary, the style, et cetera, and finds deeper patterns so that when we get a new piece of content or even we can look at it historically, know that at least they're probably created by the same people. We might not know who they are. We probably won't know who they are for a while, but if we could start grouping pieces of misinformation, it's important. So we want to identify the new actor early, as I mentioned, and then we can start looking back at historical records and information and see where we went wrong in the past. We can learn from our mistakes. We can better insulate ourselves. We can do a mea culpa. Everybody screws up, but let's see what they tried to do before and let's see what we can do now. Okay, this will also help us detect new trends, I hope, because new topics come up every fifth day. There's some new storyline they're trying out, and they're gonna try it out in small groups first. They're gonna try and see if it bleeds out a little bit, but if we can see that early, then we can start counteracting it even earlier, and maybe that's not important for some of the big obvious lies, but if the lie is instead around, I don't know, is Russia gonna invade Ukraine? I'm wondering if we could have detected that in November a little bit better. Are these big questions, are these big ops easier to find? So let's, I don't know, I think there might be a machine learning algorithm methodology to this that can find a deeper hidden pattern. We'll see, I wanna try, I wanna try everything, I wanna throw the kitchen, everything but the kitchen sink at this, and if anybody out there wants to help, I would very much appreciate that, okay. There we go, anyways, as I said, I talked a little bit quick at the beginning, so I sort of finished this up a little early, but thank you for hearing me out. This is a really important topic, this is a really complex topic for a lot of people to try to understand, and I'm sorry if I did not the best job at it, I think I tried. Please try to understand where we're coming from, recognize it yourself, talk to people about this, explain, it's not just that single lie, it is way bigger than that, and let's try to build some tools to actually make this better. This is my contact info, I'm not gonna put my phone number up there like I usually do because I don't want weird people calling me, but if you have any questions, I'm here to answer them, it can be about this, fact checking if you're curious in general, journalism, the computer science behind it, whatever you want, thank you guys. Thank you Christopher, so if you have questions, please line up behind these microphones, you probably know that well now. There's the first question. Thank you, hi Chris, the thing you said at the end reminds me of something called the attack framework from Mitra, which is related to exploits and APT groups, and it's both a framework for categorizing the methods they use, but also naming them and which groups use which methods, so maybe you could do like a parallel for that, and if somebody knows of a parallel for that for your uses, then please say so. Yeah, that's, yeah, absolutely. I think I've heard about that program, and it definitely was part of the inspiration for this sort of idea of like how can, in my notes, I didn't bring it up, but like can we create a CVE program basically or misinformation and spreaders of it? Yeah, thank you for that, I'd love to talk later. Some from the behind microphone. Hey Chris, nice talk. You should be swearing more, this is not US television here.  There was one aspect I was slightly missing, that is, I understand that some people just wanna see the world burn, I get that, but many do it for fun and profit. If I look at people from Alex Jones up to Zuckerberg, they make money with that. Isn't draining the swamp also a strategy? Absolutely, yeah, there are a lot of people funding a lot of this. However, in my experience, they're not the ones actually creating a lot of the initial push for the misinformation for the lies, as I showed earlier, there's that story that was on InfoWars, but it was originally published by Sputnik, so the Russian Propaganda Department actually had something to do with that, and then it was spread in these more local organizations. Well, hello, speaking of the Russians, I noticed from my personal experience that when the invasion of Ukraine began, the amount of disinformation that we were seeing on other topics disappeared, and we saw a new influx of different types, is that something that's modelable and trackable? You know, that's what I think it is, that's the thing. So if we can fingerprint these creators, that fingerprint will probably, I hope, spread across different content topics, right? So it's more than just, it would be fascinating to see who switched their misinformation creation from one topic to another at that exact moment. That would be absolutely amazing and really indicting against the whole organizations and such thing. Thanks. So please use both microphones. While the next person on the first microphone, please. Right, right. Hey, Chris, thanks for your talk. Yes. Really important stuff. So what do you think could be a role for legislators in this? Could you make laws or something that helps us in this quest for banning misinformation? Yeah, why did I get a stick mic all of a sudden? Okay. Can you hear me? Everyone, can you bring it? Okay, cool. You know, that's gonna, that's interesting because as an American, our standards of free speech are very different than in a lot of Europe. I mean, substantially they're quite similar, but there's very interesting specifics that are very different. And so, how do you do that? What is misinformation? There's so much, you can never say that everything's a lie, right? Like you can never, it's gonna end up filling up the court system so much.  Yeah, right, but there must be some role for legislators in here for like maybe big money, ban big money on Twitter accounts or something like that. There must be some way we can help there, right? You think? Maybe we'll talk after. Yeah, let's talk after because I need to roll that around in my head for a second. Yeah, please. The other microphone first, please. Yeah, I was first. Yeah, okay, then the first microphone first. Hello, so I find it interesting that you focus on identifying the original sources of misinformation, and especially in your response to the, when you talked about Alex Jones and how he doesn't create this misinformation, he just takes it and then amplifies it, yeah. So I find it interesting that you focus on the original creators quote unquote, instead of those who amplify the spread of misinformation because for me it is, you said that it's difficult to get into those closed groups and all sorts of things. It's difficult to identify those original creators, but on the other hand, it is quite easy to identify those who pump up and amplify, right? So why not attack the problem here and at the amplification level, right? When this actually becomes a problem, when this actually gets spread to millions of users, instead of trying to find the original creators. Sure, because that's being done. I wanna think bigger. Nobody's really looking at this question, for the most part. I want to go deeper. There's a lot of people, including at these major organizations, who by the way know who these people are. They could take them out tomorrow, but they're not gonna take Alex Jones off because he has five whatever billion followers. But yeah, I wanna think, this is a different problem that I'm trying to focus on. Not just the spreaders, but I wanna know who is behind it. You know, like investigative reporting, you wanna go as deep as you possibly can and then build it up from there. Thank you. Other microphone, please. Thank you. Yeah, fake news and propaganda has been around as long as humanity is around a bit. In your presentation, you mentioned that technology is amplifying this, but shouldn't you say surveillance capitalism is what's doing the amplification and not technology itself? Could you expand a little bit more on why how surveillance capitalism would be causing this? By optimizing all the algorithms for attention and polarization, because if you push around fake news and clickbait, people keep clicking. Sure, so yeah, so that definitely doesn't help at all. That makes it much worse, but also the fact that you can spread information instantly anywhere in the world, even before it's amplified by some crappy algorithm, shitty algorithm, sorry, I should swear more, is one of the big causes, right? Without the internet, you couldn't amplify it in the first place. The fact that Sputnik can post something and I can read it in Brooklyn, New York means that that is way more likely to spread in general. If I was living in Brooklyn, New York in 1975, I could not get a copy of Pravda, right? Okay, but the likelihood that you're reading it, that's caused by these algorithms that are pumping this around. Not disagreeing there at all. That is absolutely true. Like I said, technology made the whole thing much, much worse. That is, those are part of the technology. Is technology part of the social and economic system? Yes, but at some point I could have gone, if I wanted to do just a lecture on the economics of this, that's a whole different thing. Okay, thank you. Next question, please. So I had an immediate uncomfortable gut reaction when you started talking about building a tool to track down who said a thing. Yeah. And maybe you could reassure us a little bit about why you're not creating a scary tool that bad people will use to go after activists? Absolutely true, because I'm a good guy. Yeah, right, that's what we all tell ourselves. It's how I sleep at night. The bad guys probably have a lot of this already. They're just not letting anybody know. That's where I fall on that. It's a really good question and it's an ethical question that we would need to discuss. At some point, every technology can be good and it can be bad. There is no neutral technology and only bad users. And that's a good point. It's something that we do really need to discuss, but at what point do we just throw our hands up then and say, well, they could do whatever they want because we'll never find them? I don't know, what's the trade-off here? Okay, Baren needs to follow up on that real quickly. I guess one of the reasons I sat down and didn't ask the question the first time is I thought to myself, well, most of the people that are trying to make the world better are not particularly ashamed of it. So I guess it would be interesting to focus on what kind of activism needs, that kind of anonymity and whether you would be putting them at risk. Right. And a lot of it. But it's worth thinking about. Absolutely, and let's have a beer and talk that over a lot more. Yes, you can do probably a lot of more questions after the talk with a group of people. So going deep into the questions. Next, from the cyber guy. Oh, hi, I'm a cyber guy. Okay, two things very quickly. First, your premise is that technology will not solve the problem. But I happen to know that you've been developing some technology to help solve the problem. And I'd like you to tell us a little bit more about what exactly that is. Second thing, we know that there are fact-checking websites out there, but most people don't visit them. And so most people will never see a fact-check. And so can you talk a little bit about how, what kind of strategies we can employ to improve the general public's ability to defend themselves against misinformation to understand the context and nuances of what's going on? Thanks. Yeah, thank you. Those are great things. Thanks for asking me to promote myself a little. So the big thing, so at the Duke reporters lab, the big thing that I've worked on over a number of years, which is not a failure, but it's definitely not good enough what is automated fact-checking. So I said that you can't automate fact-checking. And I know that because I've tried a lot. The big thing that we built is a product called Squash. I did not name it, my boss did, because you squash bugs, I guess. The code base is called something very different. But basically you can listen to, it listens to speeches, by politicians, debates, et cetera, automatically transcribes it all, then runs it through another system created by a group at the University of Texas at Arlington that can detect fact-checkable claims from something which is harder than you really think. It's way harder than you think. But it's really good at it, especially for American political speech. There's another group in Argentina that's written something there called Chequeado that does something very, very similar and is excellent in Spanish, especially Argentinian Spanish, though they worked with Spain too, on the UK, full fact, did something similar too. And then so you get the full claim and then you try that claim matching. You try to figure out if somebody else has checked this before and then show that straight over the video. Sounds awesome. The problem is we're only at about 50, 60% of accuracy when it comes to claim matching and because the topics change so frequently, it's really hard to build up enough of a data set around a specific claim from a very specific race. The other thing that I'm working on right now is an archive of fact-checked media. One of the big, that's what's gonna help, I hope, with the tracing is that right now when a piece of social media is fact-checked, that video image, audio, is taken down pretty quickly, hopefully, that's sort of the goal. The problem is then we have this structured data that is completely meaningless because you don't actually have the image it was fact-checking or the meme or the video or what have you. And so we're building a entire archive that will automatically download all this the moment the fact-check is created, hopefully before the platforms take it down. And then that's searchable, that's reverse image searchable, reverse video searchable, text, OCR, et cetera. And with that, we're building up this huge data set that I'm hoping we can start looking at for this finger check. When it comes to how do we get people to read fact-checks? Good God, if you could figure that out, I have a lot of grant money for you. The problem, that is the big problem. You end up with a preaching to the choir issue. Basically, nobody would go to a fact-check website that doesn't already agree mostly with the fact that there can be truth that people lie. And so you sort of have to reach them where they are. One of the big things is Google putting fact-checks directly in search results has been really useful. Yes, they've done something good. They have the fact-check insights program where they pay fact-checkers to fact-check stuff that has been reported. And when that's done, everybody that ever shared it gets a notification with a link to that article that checks what was said. So every single person that shared it can read it. What else can we do? It's going to be challenging. We're thinking about putting stuff on petrol station TVs. It's a throw stuff at the wall. Yes, there are two more questions in the audience. First, the pink T-shirt, please. Yeah, thanks for your call. I appreciate the thought to identify the actors behind that, especially when there is very small group. But I have to admit, out of curiosity, because I can't see the effect if these actors are known, because my impression is it doesn't make a difference for anyone spreading misinformation what the source is.  They spread it anyways. What's the benefit of knowing these actors? So it's good just to let people know. People should at least be able to know who created what they're reading. But also, this is important for policy. This is not just for the individual user. It's going back to the legislation question. This is good for policy makers to understand where this is coming from, that what they're spouting, the facts or the statistics and misinformation they're repeating often on in-parliament buildings is wrong and where it came from. If we're talking about censorship, that is somewhere where you can start. We can't start if we don't have the information about who's creating it in the first place, right? So it's really a bigger policy. It's bigger technical thing. It's bigger than just to the individual user. The other microphone, please. There's one question about outright lies evolving in somewhat plausible theories that reach mainstream or reputable media. I'm trans. This is my topic I follow. And I see that a lot with transphobic talk that started as outright obvious lies, but certainly evolved into the gender critical movement as they call themselves. So what do you do about that? Well, as you said, they started somewhere. And so I'm hoping we can find where they start before they start evolving and moving, right? It's like trying to catch an invasive species at when it's rolling off the ship at the docks rather than scouring the forest for it. What you can do after the cat's out of the bag, that's a really tough question. It's something that I've been struggling with for a long time. And there's so many topics where this has happened, right? Flat Earth, Pizza Gate, et cetera, the US. And it just becomes really, really challenging at that point, but that's why I wanna see if we can cut off the head before it grows too big so that we can stop those early. So next question, please. Hi, thank you for your talk. I was wondering who should be trying to identify the head and who should be the one trying to cut it off? Like, should it be academics, journalists doing the research on who the heads are? Should it be government? I know government is really struggling with the question on how to deal with minsofo because you also get into the freedom of speech stuff. But also, I don't know, I guess government should do something, right? So what are your thoughts on this one? I mean, government should do something. And when it comes to cutting off the head, they're probably the only ones that can, in a lot of ways, them and the big platforms, of course. So it's not the message, it's the media, right? So who has the ability to stop that spread of those specific people when they're, or organizations or what have you when they're identified? That's going to be government, that's going to be big tech companies, that sort of thing. It definitely, journalists and academics, hackers, please, for the love of God, should absolutely, I think, should be the ones doing the research when government starts researching this. Of course they haven't, right? CIA, MI6, GCHQ, NSA. They've all been doing this for years, I'm sure. But they're not making it public because it doesn't benefit them to make it public. It benefits them internally. We'll see. If anybody, hey, where's the Fed? Come talk to me. Yeah, thanks. Sorry, I wish I had better answers about the government. It's tough and every country is going to be different because every country has different laws. The freedom of speech in Thailand versus freedom of speech in Argentina, you're going to struggle on both sides. Last question, please. Yes. So I was thinking a bit that for now, all these measures and research is going into somehow treating this condition that's affecting the public. But if you continue with this analogy, maybe it's also better to look at, let's say, immunization of the population because maybe there is a fertile substrate for this kind of discord to spread because, as you said, it seems like the real end game is to actually see discourse between the public in various groups or the whole public of a nation. And so maybe it's actually due. Of course, there is no control group for this because the technology group with these new problems we're facing like climate change, economic conditions, worsening, the decline of the middle class. So maybe it's also that we should think about changing also why the people are more prone to share and fall into this kind of misinformation and mistrust towards the government, for example. And yeah, I don't know. It's more of kind of an open question. Yeah, absolutely. And that's something that a lot of people are looking into. So how do you educate the youth? How do you educate adults? How do you make them less prone to believing this stuff in the first place? My usual answer is, that's awesome. It will solve the misinformation problem in 20 years when those kids grow up. We have a problem now. Adults, it's real piecemeal. The group Politifact, who I'm quite good friends with in the US, have tried this a lot. They've had a lot of success on very limited scale by going to public libraries in these small rural towns and literally basically doing this, like ask the fact checker, and then surveying. And it turns out, yeah, people tend to trust the media and fact checks more afterwards. Trust the people that you would normally get it from less. The problem is you get like 20 at a time. So what are you going to do at that point, right? There's a lot of resources. If anybody's out there, International Fact Checking Day is the day after April Fool's Day. Get it? April 2nd. And there's a ton of education resources, including curriculum and price. I don't even know how many languages, but at least eight. We hit the main ones for sure. I'm sure Dutch is on there.  And there's a lot of researchers around the world working on the education standpoint. I was just hanging out with one. She works at Temple University last week who's making some great strides on it. Yes, thank you. We are running out of time. Thank you for all the great questions. And thank you, Christopher, for the talk.


Hey guys, Dylan Schumacher, Civil Health Defense. I wanted to go through the gear list for handgun tool one, for those of you who are considering attending and give you a rundown of the things that you need to bring and why you should bring what we tell you to bring. So the first thing on that list is an open and teachable mind. I put that as the first thing on that list because I mean that. If you want to come to class, I hope that you want to come to class to learn. Otherwise, I'm not sure why you're coming. If you don't want to listen to me and do what I tell you to do, I guess you already paid me to teach you, so that's your loss. However, my goal in every class is that people would come away and learn things. I've taken a fair amount of classes. I've been doing this for a little bit and I have some reasons for some things that I think is the right way to do things and I'm not just making this stuff up. I would be happy to tell you my reasons for doing things but what I would ask is that if you would come and take a class with me that you would try it the way that I would suggest for today and if after that you think it's garbage and crap, you can toss it out and do it your own way. But I want you to come to class to learn. I want it to be a learning experience for you and I honestly want it to be a learning experience for me because over time, of course, I will teach things differently and I will change and as I grow and learn and become a better shooter and a better teacher, we'll change some things and we'll start to do some things differently because maybe there's a better method, maybe find a better way of teaching it, yada, yada, yada. So come to class and come to class ready to learn because I want to have a good learning environment and I want to equip you with the skills that you're here to get. The second thing on the list is a handgun, nine millimeter to 45 ACP. I also put on there a G19 size or larger and then a spare pistol is a good one, a good idea if you have one. So the reason I put a G19 size or larger is the class, as we're gonna cover here in a minute, we shoot about 700 rounds, give or take. And therefore, that's a lot of shooting. That's a lot of shooting in the day. And if you have a gun smaller than your G19, you're gonna feel it.  I'm not a big fan of single stack guns in general. I understand that some of us have to carry those because we have conditions where we can't carry a real gun, a G19 size or bigger and that's fine, I get that.  However, I would recommend strongly that you have a G19 size or larger and we have some guns available that you can rent, so I'll rent that to you for free so that you can come take the class. But a bigger gun is gonna serve you, especially when we start shooting at longer distances. And by that I mean outside of 10 yards, you're really gonna notice that full-size handguns are gonna serve you really well in ways that those trimmed down guns are just not going to serve you. Second thing, third thing on the list is 700 rounds. We do a lot of shooting, okay? Where we're loading or shooting all day. We do a lot of shooting because I want you to get as many good reps in as you possibly can and we don't always shoot all 700 rounds but that's what I tell you to bring for the day. I also highly recommend that you bring brass case ammunition. I've had some people bring steel case ammunition and as we all know, and if you don't, you're about to find out, steel case ammunition just is not as reliable and often causes more feeding issues. And it's very frustrating to be taking a training class and have equipment issues. I've had that myself where I've gone to take classes and had equipment issues. I was in a night class and I was a rifle night class and two flashlights on two different rifles died during a night class with rifles. And that was a very frustrating experience with me and after that I went out and bought some new lights so that I didn't have that problem anymore. But I don't want you to come to class and struggle at class because you have equipment issues when you're trying to learn. That makes learning much more harder for you. It makes it harder for the class flow in general and of course you just leave with a frustrating experience. And brass case ammunition really isn't that expensive. For nine millimeter, you shouldn't be spending more than 19 cents around anyway and steel case ammunition just isn't gonna save you any money. Note taking gear. I tell you to bring note taking gear because I would expect and hope that you would want to take notes at class. Whenever I go take classes, I come back with notebooks filled with notes because I am writing down everything I can as fast as I can because I wanna be able to review that later. I wanna be able to retain it in the moment and I wanna be able to continue to learn from that class after I leave it.  I would recommend that you take notes and write things down so that you're able to maximize your learning experience. Holster. So a hard sided holster is required and by that I mean something kydex or kydex-like or even leather if you really want. Big fan of kydex and kydex-like holsters. No soft sided holsters. You shouldn't be taking a serious shooting class with a soft sided holster anyway. The problem of course there is that those can fold in and when you go to holster your gun it can catch on the trigger or more than likely what happens is people will hold the holster open so you'll have to pry that nylon, soft nylon open and then you'll flag yourself as you put the gun in there. Now is there a way to holster that sucker without flagging yourself? Yes there is. However soft sided holsters do not have the weapon retention that we're gonna need and again if we're shooting 700 rounds think about how many draws that is during the day. It's quite a few and a kydex or kydex-like hard sided holster is gonna serve you. It's gonna pay for itself in just one day because you're doing these repetitive draws and holsters and reloads and whatever and doing all these kinds of things that require you to have a holster we also wear our guns all day. We load our guns up once in the morning and then we wear them loaded the rest of the day and having a solid holster is really gonna serve you both for your learning experience and just from a safety perspective so we don't have any accidental or negligent discharges. I carry my gun appendix when it comes to holsters you can carry your gun however you want.  I've never had anyone wanna show up with a shoulder holster. If you did we could probably put you on the far left of the line and deal with that but I wouldn't recommend that. I would recommend something on the waistline right and either appendix maybe strong side or if you wanna go four o'clock in the back that's fine too. But I don't have any restrictions about where or how you can wear that holster. Magazine pouches, look you can carry your mags in your pocket if you want but again a magazine pouch is gonna serve you a lot because we're gonna be reloading and doing a lot of manipulations with that gun today so I would recommend bringing some kind of mag pouch.  I've taught classes in the rain I've taught classes when it's like 40 degrees and like sleet raining that wasn't fun but we taught that class and we had a great time. So no matter what, no matter the weather we are gonna shoot that day. I do not cancel because of weather so bring whatever you gotta bring in order to function in that weather. A bag lunch and water, we're not gonna have time to go out for lunch so go ahead and bring lunch and lots of snacks and lots of water. Eye and ear protection, that one should be fairly obvious but we're shooting guns so you need to wear eyes and ears. I highly recommend electronic ears. You can find some on Amazon for like 50 bucks those Howard Light ones are great. Electronic ears are gonna let you hear me and hear what else is going on around you and still have hearing protection against gunshots. Bug spray and suntan lotion that's again pretty obvious. Bring bug spray and suntan lotion we're gonna be outside all day shooting so you'll need suntan lotion if it's warmer and bug spray because usually we shoot in rural areas and they have bugs. At least four working magazines I would recommend at least four assuming you have a double stack gun. If you have a single stack gun you're gonna need at least eight magazines. The reason we need at least that many magazines is we're all gonna load our magazines we're gonna go out, we're gonna shoot some repetitions and I base those repetitions on you having that many magazines. If you have less than that then we have to reload more often it slows the whole class down. So please bring at least four magazines so that we can keep the flow of the class moving along. If you have more magazines that just means you get to reload less during the day. A tactical light so if you have a handheld tactical light go ahead and bring that. This is not a night class this is a class that takes place during the day. However when we do single hand manipulations you'll have a chance to work that light and just get some dry run practice in with it. Knee pads for those of you who maybe have some knee issues or it hurts to kneel I would recommend bring some knee pads because we're gonna spend some time kneeling. A lawn chair, a lot of the classes I teach are in brutal areas and so in between and on breaks and during launch it's nice to be able to sit down and you need a chair in order to be able to do that and sometimes we don't have those available. So I highly recommend you bring your favorite lawn chair so that you can sit down and take a break while we're on breaks. I do not have any prerequisites for this class this is an introductory handgun class so we'll take you from zero to 60 as long as you are willing to learn and willing to follow instructions. Hope to see you at handgun 201, thanks a lot.


What's up everybody, Jason here from jamesfrakesprac.com. Last break of the night, 22-23 Panini Spectre Basketball. Two box break for your season's number seven. And again guys, buy two teams to get a chance to win up to $500. Two winners of 2.50 each will be announced later when we randomize the names to see who gets into that. So again, R&B customers aren't a part of that giveaway but you did have a Bogo. So if you bought one spot in the R&B, you got a free spot.  Now again, two boxes left, two boxes right, one, two, three for the left, four, five, six for the right. Got a physical diet here, there you go, four, one, two, three, four, five, six, we're gonna go with the right side. These last two boxes here will be for POT number eight which can break tomorrow guys. All right, here we go guys, good luck. All right, we got our Frini Simons. $2.99, that's for Portland. Trailblazers, 13 team R&B, 52, 122 is Nancy. Clyde the Glad Drexler to 99, Sky High Signature. That's another one for the Blazes. This time it's 26, that's six is Nancy again. Colin Sexton, I don't want it to be 49. One shunt. RJ Barrett to 75, mix. David. We got Orlando Robinson, heat, $2.99. Part of the 13 team R&B. Hey, hey, we're on with three. J-Dub, redemption. Aspiring autograph screen, 44, Kenneth Lofton Jr., nice one there for the Grizzlies. 13 team R&B, going to zero, James, there you go man. I think this dude's pretty good. Big boy too. Nice day buddy. Alright, one more in the first box. You got Bobby Portis for the Bucks. One shunt with the Bucks. And Canyon Martin Jr. Radiant autographs there for the Houston Rockets for this part of the 13 team R&B. 64, four is Nancy. Alright, last box here guys, good luck. I don't want 49. Pelicans on the mark. And Walker Kessler to 199, nice one there for the Utah Jazz, one shunt. There's Kenneth, and Kennedy Chandler, sorry. Five out of, or 15 out of 25. Five is James. And Austin Reeves, four out of 25 for the Lake Show, Matt L. Nice. Kevon Harris, for Orlando. 13 team R&B, 129, A-A run. Colour Grouse Astral, number four, Jordan Poole. There you go. And Jordan Poole, obviously there should still be Warriors. Just gonna be double checked though, but pretty sure. And yeah, it's going to say Warriors. 13 team R&B alone for 0 and James. Also, Astral is out of 49. Okay guys, good luck. So if there's nothing here guys, I don't know if there's still going to be a color batch or something. I don't know how rare those are still, but. Franz Wagner, 20 out of 99. Orlando, 13 team, 20, James. And the last one here. Josh Minott to 99, aspiring autographs. Minnesota Timberwolves, 13 team R&B, three. Winner, Aaron. So there you go folks. Again, I don't know how often the clips or color blasts pop out, but nice names here, but nothing super, super crazy. So let's see if potentially the last two can absolutely really nice guys for tomorrow. A lot of hits there for the R&B though, so that's pretty solid. Thank you guys so much guys. I think there is some non-numbered cards potentially for this R&B, so let's quickly randomize the names as well as the names for the $250 break order giveaway. So we'll have a new dice roll, new list, new list. So this will be four, the non-numbered. And then this will be for the people that potentially have a chance to win some money. So Brian just won, David B, you got four, which means you have two entries. Edward II won with four, so that's one and then two entries. Matt with two, which is one. Then we got Wenshaw and four with two. Really good chance we got one in each shot. Alright, roll it. Two and a three five times, good luck. One, two, three, four, five. Five, five. So Nancy gonna get the all non-numbered cards. And then we're gonna go five times here, top two win $250 back each. One, two, three, four, five. Five times, five, five. David B and Matt. Matt L, congratulations guys. So both of you guys will get back $250 each. I'll send that out to you right now, Jaspie's case present. Alright guys, thank you.