Hi everybody, my name is XanNorth. I'm a member of Distributed Denial of Secrets and you are here at Leak the Planet, or how you stopped, how you learned to stop worrying and love the Lea. Who we are?  As you can see, and dog lovers.  We have Emma Bess right here and the rest of these are people that also work with Distributed Denial of Secrets. But I all know you want us to bring back the pets. So first of all, we have types of leaks and leakers. Leaking is as value-neutral as farm, as pharmacology. It comes down to intent and application. We have insiders, activists, spillage, state-affiliated actors, and ransomware. And we'll go through all of those. What they are, what they mean. Insiders. So cognitive dissidence addresses itself through leaks. A lot of people, when they join an organization, they may have ideas about how that organization works that don't necessarily come to fruition. Examples of insiders are Daniel Ellsberg, Chelsea Manning, Edward Snowden, reality winner Natalie May Edwards, and Johannes Dethenson. Next, we have hacktivists. Not all whistleblowers are insiders. Some of them come in from the outside. Good examples of that are Redhack, Anonymous, Jeremy Hammond, Phineas Fisher, ACAP Gang, Cyber Partisans, and Donk NB. Next, we have spillage, which is pretty much when a stereo falls off a truck on the information superhigh highway. This is typically an accidental release of data that's a result of user error. Examples are Heritage Foundation, police aerial surveillance footage, and GiveSendGo. GiveSendGo particularly. They were warned repeatedly that their S3 buckets were misconfigured. They never took any action. And, well, they got leaked. Services like Gray Hat Warfare make misconfigured S3 buckets easy to find and search. Many leaks are avoided by disclosure protocols. Again, going back to GiveSendGo, they were warned repeatedly, and they did nothing. And then when their leak finally did come out, they denounced it as fake news. Nothing. They weren't really hacked. So, next, we have state-affiliated actors. You probably love them when they're yours. We have Sabu, Popov, Cyber Birket, Guccifer 2, DC Leaks, IT Army, and Conti. And then Ransomware. Very little of this data merits journalistic or academic review, but there are some gems. Example, Jones Day, Chicago City Hall, Metropolitan Police Department, DC, Illinois Attorney General, and Perceptix. Some declared political stances following the Russian invasion of Ukraine. And financial motives mean they can be hired by state actors. Example of that would be Conti. So, next, we move on to publication models, which also serves as a brief, all-to-brief history of recent leaks and ways of leaking things. First up, one of the oldest leaking institutions that's still active in the Western world, Crypto. Everybody's best friend of me. They've been publishing censored and leaked material online since 1996, using basic HTTP listings, and for a time, an FTP server as well. They helped found WikiLeaks. They were on the initial advisory board, and they helped register the domain name. But, like I said, they criticize freely, and that includes WikiLeaks, although they strongly oppose the extradition and prosecution of Assange, and gave testimony in one of the cases against him for the extradition attempt. And to this day, Crypto continues to be active, which is a minor miracle considering it's been nearly 30 years, and they've faced threats from multiple corporate entities, as well as government agencies that have visited them and had all sorts of questions, and we're often angry at them for making public documents to public, because what if people read them? Crypto very naturally brings us into WikiLeaks. They wore every hat until they wore virtually none. They were revolutionary at times. A lot of debate was sparked, and then they were used by Russian intelligence. We know from exchanged messages that they received the DNC emails from Guccifer 2, and they appear to have received the Podesta emails from DC leagues, also based on messages that we know of. Their last original leak was, I believe, late 2019, and that was one of the OPCW leaks, which that series has been tied to Russian intelligence again by a series of articles written by Daily Beast, New Lines Magazine, and Bell and Cat. Those are all very worth reading, because Russian embassies and diplomats are directly involved, and a sting operation was able to reveal quite a bit. And, unfortunately, WikiLeaks throughout its history has suffered a lot of instability and periods of inoperability. There was the 2010 incident where their infrastructure was taken down. There was 2011 or 2012 through 2015, I believe, where they had no submission portal. They recently lost their submission portal again, and only very recently launched a portal that almost works. It will upload files, but sources will never know if it succeeded in uploading them, which brings us next to Anonymous's answer to WikiLeaks, launched during a feud in 2012 and 2013. They called it paranoia, short for potentially alarming research from the Anonymous Intelligence Agency. Anonymous was frustrated by WikiLeaks' slow releases, and WikiLeaks, for their part of the feud, was accusing Anonymous of insecure methods. And, at this time, WikiLeaks' submission system was offline, while paranoia was meant to be a quick, simple leak platform. But, unfortunately, that was one of the problems with paranoia, as it was very easily exploited. Some of the release notes were highly inaccurate, and given the context of those inaccuracies, that certainly had an agenda with them. And it was in particularly used by Max and Popov to help launder some of his data. Next, we have groups like The Intercept, where they are literally building journalism from the ground up around leaks. And I know we all have thoughts about The Intercept, and certain people that worked there and no longer do, mistakes that have been made by individuals there, and, yeah, we could say that about any outlet. But the lessons from The Intercept are still very useful, because they built it from the ground up to allow this kind of research. And they even tried to extend it beyond their own borders a bit with the Snowden archive, which, unfortunately, was closed. But they continue to be active and report on large leaks to this day, including partnering with regional outlets in some instances, such as data sets relating to the Russian invasion of Ukraine, which brings us to the consortium model, where we have groups like Areej, EIC, Forbidden Stories, ICIJ, and CCRP. You can see on the slide a list of a number of high-profile projects, most of them dealing with finance, that the consortium model all dealt with. And because of the international nature of these projects, consortiums are uniquely able to go after the data, because you need local experience for each realm, you need multiple experts, you need boots on the ground, and no one outlet can realistically handle that. They rarely publish raw data, but sometimes they do publish relational data, which you see with ICIJ and some of their databases, which are publicly available. And then there's these people called DDoS Secrets. That's us. In the last four years, we've published leaks from over 50 countries and over 200 releases. That number's a bit out of date, but we're constantly trying to balance source protection with source transparency, which is a major issue, especially with things like the Russian Ukrainian War. We have people sending us data sets, and there's a very obvious question when it's something that gets the cyber war tag that we created, alerting people to the context and that there's no specific suspicion for these data sets, but it's still something that you should look at more closely and be more aware of. We also strongly focus on separating editorial and what the meaning of the leak is from what we do, which is presenting the data. We will help journalists interpret it as best we can, but we don't tell them what to write. We don't present our own analysis of it, because one, we have data from over 50 countries and over 200 organizations that range from topics of artificial intelligence to manufacture of nuclear power plants. We are not and cannot ever be experts on all of these subjects, never mind the language barriers. It is better left for people who are already experts in that than for us to try to become everything. I seem to have hit, sorry about that. And this also helps prevent our biases from getting in the way of what the data says, because when you touch this much data and you bring it to the public, it is very easy to infect it with your point of view and your messaging. And it can't become an inherent part of the data. One thing we witnessed with WikiLeaks and the Vault 7 releases was they wrote about how certain devices could be exploited and how that could then be leveraged to get past the encryption on things like signal and WhatsApp by making the encryption literally irrelevant. But people took away the wrong message from that for several articles and a large number of people on Twitter because of a imperfectly crafted tweet. And so we have a great responsibility to not do that. Our goal is to collect the data, review it and then entrust local reporters, investigators and activists and experts to follow through and in many situations to find the right consortium. Moving on, we get into actual data. In the last eight hackers, DCLeaks was an exercise in persona and access management. While some of the data was public, not all of it was, which was a way of playing favoritism, a way of creating mystique and hiding some of the overlap between various releases. DCLeaks was later linked to fake journalists personas by investigations and especially forensic ones which followed different digital accounts. And there were early and unclear and inconsistent ties to Gucci for two established publicly early on. For instance, the email in which Gucci for two said that DCLeaks was a wiki leak sub project and they provided restricted access to DCLeaks releases. Thus, we have Gucci for two selling the narrative, which was launched obviously to counter the revelation that was then new of Russian hacking of the DNC, DCCC, etc. They used a combination of blog and social media posts with individual documents, some of which had been tampered and larger dumps. We know from government investigations that Gucci for two communicated with over 12,000 Twitter accounts, exchanging over 15,000 messages. We've only managed to identify so far about 10% of those messages and far fewer than 10% of the accounts that have that communicated with Gucci for two. But the known, the known discussions are part of the transcript cited at the end of this, thus spoke Gucci for two. And we also know from the messages that they were active nearly 24 hours a day when they were active, which is another giveaway. So what does state affiliated actually look like? Because everyone has ideas of what they think it is or what they think other people think it is.  Maxim Igor Popov was a Russian contractor who infiltrated anonymous and before that the FBI. He was arrested by the FBI and turned into an asset. And on his first job, he was sent after the people that he used to work with. But the FBI didn't actually watch very closely what he said in Russian. And by using some unique idioms that were a little, you know, less known to the translators, he warned them what was happening and they were able to get away free. But he then went on to help them with a number of other things before being let go under questionable circumstances. And when I say questionable, I mean the FBI agent responsible was investigated and nearly indicted over it. He was involved in the 2004 2005 FBI email hack. Didn't get a lot of coverage at the time, but they didn't hack the FBI directly, but rather the AT&T servers that the FBI used for a number of their things, including their emails. And these were sensitive emails. And Popov was never so much a technical hacker himself, but he was the person on the crew that made things happen. He developed relationships and he monetized things. And so he got the FBI to pay him and his people $10,000 for a meaningless lead on the hack that they had just committed against the FBI. They identified one of the people involved, Leonid Sokolov, who cannot be indicted and is never going to leave to go to an inditable area. Popov used numerous false personas and some of them are linked to some very interesting other hack and leak situations. For instance, the hack and leak of Ukrainian Prosecutor General, which was done by Popov, overlapped significantly with cyber recruits, similar release, right down to the same office and timeline, although their release was much more limited to a few documents. Popov released several gigabytes from that office and several others. And he's obviously financially motivated. But aside from his own admissions to being a state contractor for Russia, which he said to Jeffrey Carr and is implied to other journalists, there is the fact that if you are living in Russia and you hack the FBI and it is that high profile and you do not pay patronage to Mother Russia, you lose all protection. It is exceedingly unlikely, simply based on that. But also the reward, there's no reason they wouldn't have pursued that. Which brings us to Sabu, everyone's favorite American state-sponsored hacker. A member of Anonymous turned to an FBI asset. While he was active, the FBI closely monitored and controlled him after his arrest. They watched what was on his monitor, they took pictures of it, they were involved every day and they had the opportunity to direct him to do things differently or to stop entirely. Through their observation control and failure to stop, they directed him. That is beyond a doubt. And those directions included having Sabu direct cyber attacks on foreign government and commercial systems, as well as orchestrating the strat for hack and leak and his unwitting collaboration with Popov. Just a very fun thing to discover when you're going through leaked chats. But Sabu was more than just a criminal informant. By doing these things, the FBI was using them for foreign and domestic intelligence gathering for the FBI. So what about the links between people like Popov and Sabu? When the lines blur, these people enable each other. And that's not just Popov and Sabu. That's a general trend. Sabu and Paranoia both laundered Popov's leaks. We can see in chats where Popov literally gives them to Sabu and Sabu puts them forth saying it's an anti-sec release. It's a low-sec release and they're the most shady ones. And in several instances, people who were actually involved in those groups then became very concerned afterwards about issues with those data sets. Sabu also offered Popov early access to the Syria emails released by WikiLeaks. And that's significant as well because not only was the one banking transfer email missing from the WikiLeaks release, but over 1 million of the 2.4 million emails is not been published by WikiLeaks. You can review this on the WikiLeaks website, or I wrote about it on my site Emma.best where you can also download a script and manually verify it yourself. And that is one mystery that they have failed to follow up on or comment on, but maybe it's the same secret lizards thing that they commented about the Clinton leaks. It's a true story. You can find it in the wayback machine. While they were working together, Popov and Sabu actively discussed using cyber attacks on government systems and using false flag cyber attacks to quote, create real cyber war. Keep in mind, during this time, Sabu was under the FBI's control. He was having ongoing discussions with people about false flag cyber attacks with the goal of creating not just chaos, but conflict. And Popov's activities included faking funding of terrorism by one country.  And Popov even gave Sabu targets and credentials, and we know that those were passed on. Some of them were even given to Jeremy Hammond. And if you look at the unredacted paperwork for his court case, some of which was leaked to me as part of his discovery, you'll see that those are in fact the same ones that Popov gave to Sabu. So we have a former FBI asset and a Russian contractor giving a FBI asset who is under FBI control credentials to attack foreign government systems to an American hacktivist who is then convicted for doing those things. And these are the kinds of states sponsored actions that are never looked at. They are not understood. And why would they be looked at? There's already a scapegoat.  And we don't need accountability beyond that because that would just make everyone look bad. And no one could stand that. Looking bad is worse than people dying, at least in the eyes of the government. We move on to Gooch for two. One of the most telling quotes about Gooch for two comes from Julian Assange, where he said, these look very much like they're from the Russians. They almost look too much like the Russians. It was, of course, a persona managed by Russian intelligence services. They were the source of the DNC emails. We know this from public statements as well as private statements. They made in their chats with WikiLeaks. The transcript shows the communications with and submissions to WikiLeaks discussion about timeline. And WikiLeaks, of course, explicitly stating, we want to do this on, you know, in time for the DNC so we can have this specific political impact. But despite the overwhelming amount of evidence about Gooch for two, there are still truthers out there who insist on Seth rich this or vips that or forensicator, something or other. And that is one of the things that happens when there's a failure to communicate beyond thick, incomprehensible reports, which are also unfortunately very easy to manipulate in terms of media. A much more recent example that I think many of you may not have heard of is free civilian, the data, the data merchant that was a terrorist. They claim to sell data, but they posted it freely, including Ukrainian medical data and about one million government emails from Ukraine for their internal development. They used Cloudflare to protect their Swiss servers for distribution. And after the servers were the company was notified, the private layer ink took their servers offline. However, Cloudflare, last I checked, continued to protect the now defunct domain, despite being told explicitly what it was being provided proof of what it was. And outside analysis that confirms that, yes, this was almost certainly Russian state affiliated actors. And it was almost certainly a hack and leak sigh out to create fear as part of the Russian invasion of Ukraine, making it not a criminal act, but an act of war. Cloudflare simply responded, we are not hosting the content. We're simply helping them host it. This brings us to the issue of misidentifications, because it happens both ways where we either miss someone who was a state actor, or we look at someone that wasn't a state actor and someone decides they were. And for Sugu, the answer is simple. Agent Brutus says he was a hero. Agent Brutus is an honorable man and he has brought many captives home to Rome. So when he said, Sugu did great. He prevented all these attacks. I mean, he started them, but he prevented them in the end. It's so great. He's the terrorist and the hero. And, you know, normally that's a crime, but not this time. And for the most part, people, especially in the mainstream, especially mainstream outlets, went with it. They trusted Agent Brutus because he was an honorable man. And he had arrested many people. So surely he knew what he was talking about. Then we have Popov. He was simply missed because he was a con artist with too many fake IDs. That and the FBI agent responsible kind of just stonewalled when he was asked to help solve what was going on. And yeah. And then we have people like Donk Envy, who are hit with the worst kind of reasoning. Only a state actor could have done this, which you inevitably hear from the least educated people when it comes to how computers and networks and hacks work. Because they will look at something big, some big accomplishment. And for Donk Envy, it was the parlor stuff. And they'll say this was big. So it had to be something equally big to do it. And this sort of thinking is also what leads to a lot of conspiracy theory thinking. Like the assassination of JFK was a big event. So something big must have been behind it. It could not have been a crazy dude to be way overly brief. I've gone into that stuff a whole lot. Please don't think I'm actually simplifying it that much for realsies. But yeah, people simply not knowing what they're talking about. You'll see an article that says maybe they did this with the help of whatever country. There was one article recently about the narrow police force leak. And they a cyber firm pointed out some issues that they thought were inconsistent. And they mentioned me several times in the report never never reached out to me, which I thought was an interesting choice. But they did say I didn't appear to have any involvement in anything. It's a very sloppy report, but it pointed out some things with timestamps. And then a journalist saw that took that to someone who worked at a think tank. And that person said it was probably China. Because China would want to do something like that. And then that became the headline and several other people reported that headline. And there was zero evidence of China's involvement. But that's how that sort of thing can get started. It's Oh, this was a complicated leak.  So it's got to be a state hacker. No, it's, it looks fishy, because it's complicated, and you don't understand it. Which brings us to the big question, or the big questions.  And it's a surprisingly broad question. And it's not actually even the right question, because both sources can be exploited and material can be exploited. And if the answer to either is yes, it's not necessarily an emergency shutdown. It's a red flag. Because just something just being exploited doesn't make it untrue or dangerous. Soviet propaganda used to point out racism in the US a lot. And they were right. It's not a problem if it is accurate and not being embellished or twisted towards foul ends. And there are ways you can, you can help make sure that it's not or that there are counterbalancing forces. But usually the first clue is the source themselves. If you have contact with the source at all, any questions without getting into the question, without getting into their security at all, can tell you a lot. One, are they trying to micromanage? Are they trying to tell you what to do, how to handle things, when it needs to be released, how it, how it needs to be labeled, you know, are, are they trying to turn you into their agent? If so, that's a problem. Next up is, can the source answer questions coherently and consistently? And it's amazing how often bad actors are not able to do this. We had one source submit a group of emails allegedly from China and they were unable to tell us what email accounts they were from or what agencies the email accounts were from or anything about the contents other than a lot of stuff.  You're not just picking up trash off the floor and then bringing it to us and saying here. And those kinds of failures often mean you're dealing with a cutout or with someone who just, there's some other red flag. And again, we come to the question of are they trying to sell a narrative? And that doesn't mean are they trying to shine a light on an issue? Because there's a big difference between saying these refugees are being abused and treated horrendously and denied their rights and the police are part of it and the government knows about it. There's a big difference between that and the CIA is, is bad and needs to be turned into something else because of these exact things that also further this person's political narrative and goal and whatnot. You know, you could see a narrative being formed now, not related to leaks so much, but the kind of narrative I'm talking about just by watching the fallout from the Mar-a-Lago raid. It's the same, it's the same kind of concept. People trying to build a narrative and once you publish that's going to be unavoidable. You are releasing it into the world. Even if you restrict it to just journalists and researchers, which is one thing DDoS Secrets does on sensitive data sets, you're still letting go of how it's going to be interpreted and what they're going to do with it. But if the source is clearly trying to sell or push a narrative, you have a problem. Other questions include timing. Does the situation or the timing force your hand? Are you being given information three days before it becomes crucially relevant? And if so, why right then? What relevant external events are going on? Elections or armed conflicts? And what is the goal for those things? Is it to shed light on it or is it to intervene? Because both can be valid. Intervention through freedom of information is absolutely a fair way of participating in the world. That's what journalism is. That's what academia is. That's how the world gets changed by people who don't carry guns. Next question brings us to counterbalances. Is there time for it? Because one of the things we saw with the DNC leaks and the Macron leaks was no. They were purposely released right before an event that they would have maximum shock impact on and they would have very little time for, one second, they would have very little time for anyone to then go through and review the initial findings or critique them or get a better understanding of what's going on because you're you're just giving people the tiniest bit of information and time to analyze it and it'll naturally always go to the most shocking things and things that are devoid of context. Next question is why now? Which is pretty self-explanatory. So now the really big question.  Chat logs is going to be a recurring theme here for most of them. Cebu caught by chat logs. He kept trying to entrap people and the chat logs, you know, if you're a participant in things it's a bad idea to keep chat logs but if you're a defendant in things they're useful to have but they started communicating with each other. Different hacktivists and they noticed patterns and they shared it within the larger community and then later leaked files confirmed the allegations and what was going on. For pop-off again we have chat logs but this time reviewed in hindsight several years after pop-off had stopped using those targets. His big problem that got him noticed most recently was extorting too many not most recently there were other things since then but one of the things that caused some real problems and as well as his former FBI handler was he began extorting simultaneous targets that he wanted revenge on and when the when the FBI was trying to get at the bottom of this and in particular the U.S. Attorney at Boston wanted to get into it they were asking for information on his pseudonym Dennis Pinhouse which agent Hilbert pop-off's former handler knew what that was and who he was and Hilbert refused to cooperate. Pop-off was also identified by looking in the footnotes of certain DOJ filings he just seemed connected to something and so good journalists just started following up on that and found a very interesting story. He's also identified through his posting to Jeff Carr and simply the Russian operating environment again if you are in Russia and you hack the FBI you give that to the government and you have a very nice life. How are Gutsifer 2 and DC leaks identified and combined this because they share a lot. Chat logs again the money in Bitcoin trail was a huge portion linguistic analysis stylometry forensic analysis provided more confirmation with as at least once Gutsifer 2 mistakenly logged in without a VPN and analysis of files shows overlap between Gutsifer 2 and DC leaks in their files and that one of the first documents that Gutsifer 2 published came from an attachment to one of the Podesta emails. Free civilians identification was fairly straightforward given the circumstances the inconsistent behavior being more interested in selling proof than in selling data posting already sold repute already sold data for reputation as well as the timing their attacks began right around the beginning of the Russian invasion of Ukraine within a day or two and they exfiltrated the data just prior to the wiper attacks so now it's the part where I get to yell at you I'll be gentle hopefully most of this will come across as constructive criticism and not actual aggression newsrooms you won't need to get ready for the next big leak now there's a lot happening a lot more is going to happen and most of you are not equipped to deal with any sizable leak DDoS secrets thankfully is is relaunching Hunter Memorial library and will be making it available to journalists and researchers but that is not a full replacement for newsrooms investing in themselves because you need the proper staff the technology and the training and it is lacking on all three in pretty much every major newsroom that goes for associated press the New York Times the Washington Post the list is pretty hard to come up with an end point need to focus on the long view because I'm tired of seeing articles that examine leaked data for two weeks because you're looking at emails from the last month y'all are better than that y'all have data that's better than that you have a hundred terabytes of leaked data that a lot of it overlaps dozens of mining companies of agencies that work with them that enable that stuff the world is literally on fire and the data sets behind the people who are doing it are open and available but where's the investment I see I see newsrooms with climate teams climate data teams of three maybe four people and none of them are real tech experts and remember that a collaboration that makes the cut is worth two exclusives that get shelved and way too much of your nonsense stuff just gets shelved anyway American newsrooms have definitely trended toward the wrong column in the in the last year we've seen a lot of reporting on fake hacks with no proof very little reporting on the trial of the largest cia leak in history which revealed a lot of information about the leak about wiki leaks about how cia works y'all dropped the ball on that and there's no there's no god damn excuse I've seen a lot of reporting on and promoting worthless nfts but not much following the money beyond it went somewhere and that is not useful reporting I see a lot of emotional headlines being made out of the opinions of inherently biased people the mother of so-and-so the father of so-and-so they have these opinions that doesn't matter you need to focus on recent infirm sorry I skipped ahead a number what you need to focus on is informing people what they need to know before they need to know it because otherwise when the next thing comes they won't know what to make of it and they'll be playing catch up and that's a problem for everyone that's a problem for your outlets too because if people are trying to play catch up when they're reading your articles or watching your video presentations they're already going away because they you didn't give them the tools to understand it and follow it see a lot of people confusing news with stories instead of reporting on non-breaking news that might not have closure even though it has crucial data in context and way too much investing in production instead of investing in the newsroom itself it's absurd cable news is absurd print news is becoming worse and yet there are so many talented people stop shooting yourselves in the foot and editors rethink your shit government hi come see the cognitive dissonance inherent in the system you all love to look at security and address that but the incongruences are what cause and amplify leaks everyone is uh i'd say very impressed with the size of your legal case against dissonge uh go ahead and drop the publication charges because it's bullshit i'm a critic of his do not like him he hates me but i'm up here saying it's bullshit and it seems like y'all are compensating for something do better also just pardon all whistleblowers and uh never let anyone hear you say that talking to journalists and selling t-shirts is a part of a cfa violation again ever ever i don't know how you can look at yourself in the mirror after writing that or filing it with the with with whoever that's that's bananas industry you just need to own your own shit security theater is going to get your companies killed it is going to get people wrecked it is going to destroy personal data and not even in a way that lets journalists use leaks productively be someone who isn't worth leaking and never pay the ransom one area i actually agree with fbi on for a lot of reasons that we don't have time to get into here but it's not just because i want that data leakers and leak publishers real quick study mosaic theory stay focused on results not splash or ego learn what the law actually lets journalists do and what it doesn't because a lot of sources seem to want to ask like publishers to do things that they can't do and you need to understand that state actors can and will look to exploit you on your leaks whether you want them to or not talk to a lawyer listen to your conscience and honestly leak the planet there's no way that we're going to be saved through secrecy recent report from cia found that something like 90 percent of classified documents could be safely released to the public that shit's absurd it's the same with corporate there is no reason why we can't turn to leaks as a vital source when the world is literally on fire and we don't have enough data to figure out well what's causing it specifically in these areas and what's making it worse at this rate we don't have enough data to refine the projections except we do and people aren't fully engaging with it and that's a problem because leaks are only as valuable as you let them be people have risks their lives and their freedom to bring us this data and it would be irresponsible and unconscionable if we let them down by just oh yeah this came out there's i found i found an interesting document and now that's it so leak the planet magic is real have fun thanks for listening


Good morning, everyone. It is currently 645 a.m. I am on my way well Zoe. Oh, hi by the way you guys in here subscribe Today's vlog is going to be a very exciting vlog because Kinsey and I are going over like the 21 day reset basically I'm starting food combining today with Kinsey at Burke Which I'm so excited for so you guys will see that but currently like I said like 645 I'm on my way to Verve coffee shop on Melrose Zoe has like connect groups. This is the business leaders one So it's like early mornings Tuesday So that's what I'm on my way to and then after that I have a workout with Paulina and then I'm going home And then doing all that Kinsey and then I'm recording two very exciting podcast today Actually, I like cannot wait and yeah, I might work out tonight again. I haven't really decided What I'm gonna do and like very excited for today So if you guys have not already be sure to subscribe follow me and Instagram because I'm gonna be posting So many meals and all that stuff from the food combining. I'm so excited I know obviously Maggie did it and then Margot did it and I know there's so many other people who want to do it So also you guys should just do it if you want to do it like do it right now because you can do it with me But I love early mornings. They literally put me in like the best mood like I woke up Oh, like oh my gosh and like being out and about really early. I'm like on one the whole day So I'm really excited have my like workout clothes under this. I just threw this to fun. It was like But yeah, okay. Bye. Okay. I made it to Verve.  Oh My gosh, why am I tired? It's like 9 a.m. now and I Have a little bit of time to kill. I'm like sitting outside ready for as nations like right there was so close to where I got coffee this morning, but I Right now currently in my life. I'm getting very Self-discipline. I'm like really really really monitoring the influences of my life really monitoring whom around And just like really focusing on not like like myself But like I want to grow and I want to get better and I'm being like very serious and borderline not extreme But like I think I'm just taking it myself seriously So anyways, even from like I just really felt like I needed to unfollow when people on Instagram Like I was following like 800 something people. I do not need to be following 800 people. That's like so crazy So I have it done to 471 if it were up to me I'd be following like 100 but people get so offended and like me up following someone like I still like love them and care About them with the best of them.  Does that make any sense? so I don't want anyone to be like You know it's like hard, but I'm halfway through and Also, you know anyone unfollow someone and you've wanted to unfollow them Like you've never really wanted to follow them and then now if you were to unfollow them because of like stuff That's going on it would look bad and like you care But it's like I really just don't want you on my feet like I just don't care, you know So let me know in the comments down below what you guys think of unfollowing I don't think it's that deep if someone unfollows me. I don't think twice. I don't even think that we're like not friends I'm like, oh cool. Like I literally don't care I wouldn't even really know if you had unfollowed me actually, but like it's just not that deep Anyways, I know that I'm like thinking it's you but that's just because I know these people so like So anyways, I'm sitting outside Ryzenation. My class doesn't start for 25 minutes Paulina just texted me PB ready for Russell to walk rock our world cannot wait Neon lights and rap music If you guys get it.  She's probably gonna see me blogging them off, but Because don't know Ryzenation is there a climbing class look it up. I don't know how to describe it It's like rock climbing but on a machine. It's really cool. It's like a 30-minute intense cause and It's right up there, so they're that rooftop and you have like a really cool view But they like close the blinds and stuff when you're working out so it's dark There's neon lights and rap music. Yeah, so I'm going right now Finish our work out I was blogging when you sent me that text and I like laughed out loud It sounds so inappropriate. I know but he walks our world at 9.30. He really does and sometimes in neon lights Sometimes 815 the neon lights and the rap music. It's all it's our vibe. Okay home from the workout I just took before and after pics because I'm doing it the 21 day like reset do you talk to things? Also, this tag is always sticking out on the sports bra But I'm gonna shower and then get ready to go to kensies and then we're gonna be grocery shopping for food combining Which is very exciting. We're gonna talk more about it at kensies and if you guys want to follow along with me You should I'm gonna have it linked down below you guys should get the guide follow along with me It's gonna be so fun, but I'm having off ready. I just need to change like really quickly actually in shower and get ready So I showered and got ready. I'm wearing. I'll show you. This is a little bit too high I'm wearing just biker shorts and a long sleeve black shirt both from a Ritz yeah I just threw this denim jacket on over top just my gold jewelry the gold hoops, and I'm just wearing easy's Here is my look Okay, so I really don't like wearing this denim jacket with this I need like a really good oversized black bomber that would look good with everything It needs to be like borderline varsity jacket bomber I kind of have it in my head, but I don't know I'm gonna look online today and see if I can find one But I got ready. I'm just over like denim jacket. I don't know like I don't really I'm just over it Like I've been over them for forever, but they're just so easy to wear So I wear them, but I'm not like obsessed with it by any means so I wish that I just had like I don't know I know that I have 30 jackets, but I can never have enough jackets, you know So I have two bags packed for today I have just my purse and this is the bag I'm taking into the studio So I'll have my iPad in there And then I just have like my books and stuff and some spray just random things for the studio, but yeah I'm ready to go I'm gonna head over now because I'm gonna go drop a camera roll off before I go to kensies Which she was really close to me, and I need to be there in 20 minutes, so I am off alright guys I'm with Kinsey we are starting the 21 days to your best body like I said before I'll have it linked down below but Yeah, basically I gave her my meal log, but like it's kind of it's like a fake one I mean it is what I eat, but like I eat more than that Yeah, she has some notes, so she's gonna fix my diet basically very exciting stopping in at a juice bar. Sorry out of juice bar Yeah, actually though we're stopping in at a juice bar for lunch. We're still hungry Here's some lunch. We got a veggie wrap with veggies, hummus Balsamic and we're not gonna eat the pineapple. Oh, yeah, we can't you're right You're right.  We have our grocery list kensies taking me around my treasured so they go to every day, but This is exciting Okay, so this is what we got. I'll show you guys in my home a little bit more of a haul And it'll be incredible. I can't wait. This thing is literally they gave me a box guys I have so much stuff He's like do you want a box and I was like, uh, yeah, okay, so I'm gonna show you guys my groceries We got groceries for you to make pasta Smoothies a grain bowl. I'm trying to think of what else Avocado toast and stuff you guys will see I'm just gonna be vlogging like I do normally I'm let me know if you guys want to see what I eat in a day and I'm gonna have Kinsey better explain what Like food combining and stuff is because we're gonna go to the studio and record later today as well Actually pretty soon But I'm home and I'm going to show you guys a little overview of what I got. So we have some hummus Rice cakes earth-balanced butter cake cups bananas frozen mango garlic broccoli carrots Brown rice pasta cucumbers kale avocado dates my favorite the everything about the bagel sesame seasoning blend Then some toasted sesame oil and then she said this is like really great With vegetables, so it's the mushroom and company multi-purpose umami seasoning blend some brown rice I think I already said avocados chia seeds marinara pasta or sauce Coconut water for my smoothies then also she said I'll be using this a ton the organic coconut Amino's Whatever Amino's that's it and then This mineral water She like says is the best I've had it before I actually love it So I picked up like six of them that's why I have this and then she gave me this bread because she's obsessed with it and It's the food for life Fred You can't get it at Trader Joe's you can get a whole foods, but it's sold out like literally everywhere So she's like you're gonna be obsessed with it And she actually you know Kenzie loves you when she gives you this like actually it was so nice of her I thought she's gonna be like two slices and she just like gave me this whole thing So I was like well, thank you, but yeah, that is what I have and I'm gonna be showing you guys some stuff. I feel like I'm not Really officially I'm not really officially starting until tomorrow. So this is like my prep and just getting all the stuff ready But I'm really really excited Maggie has done this with her Margot is currently doing this with her I know other people are like really interested So if you guys want to do it with us, you totally can I'll have her guide and stuff linked down below I also put it as like a swipe up on Instagram stories and stuff a lot But you guys should definitely do it because we can do it together and I've only heard amazing things I've actually known like I found out about this thing Maggie McDonald and she's already made like a what I eat into a video and stuff So I'll have that linked down below and she's an entire video on food combining You guys should check that out because this is like fully I fully found this from Maggie But I'm very excited to get my health in order because I feel like I'm very healthy for the most part But I just don't know like I'm not very knowledgeable and like food if that makes sense So I'm just really excited to like know more and just be better and healthier and it's gonna be really exciting So I'll keep you guys posted. I will do what I eat the days of you guys We'll see let me know everything down below But let me put this away and then I'm gonna go to night for Culver because I don't know if you guys could tell The past few hours. I'm just Exhausted I think I felt so about we filmed from Kinsey and I was just so tired also Kinsey started a YouTube channel I will have it linked down below If you guys are interested in any of this like her Instagram is amazing huge. It was awesome just as a person She's amazing. We literally met last week And I feel like she's a very good friend of mine already like I love her and we are neighbors so they can cook together I like hang out all the time. It's gonna be great fun. So I'm gonna put this away and then that is it okay, goodbye is the best part about this bag is that it literally fits everything and Look I can fit I can guarantee I can fit like my water or coffee everything like that is perfect It's absolutely perfect. Okay. So this is why I'm on my way to the studio I have my bags the mineral water a banana and then I had to stop and get a nitro cold brew because I am so exhausted I'm so tired. I'm like just see I can't even speak. I'm so tired Okay guys well First off I just recorded an episode with my friend Nico and it's gonna be really good It's like very Ryan can Ernie podcast style like what we've been learning He's friends with Ryan so you guys will like love him Can this year and we're gonna record an episode about a bunch of stuff a lot of food combining stuff But for this vlog for now, I realize that we haven't really explained what food combining is so I'm gonna have her quickly explain it to you guys Okay, so food combining is a method of eating so basically all foods have different use different enzymes to digest so proteins use different enzymes starches use different enzymes and It uses a lot of energy within your body to break down and to digest So food combining is all about pairing the right foods which use the right enzymes to Give you more energy because our digestive systems use the most energy out of anything in our body so if we eat and we live in a way where we can free some of that energy up from digestion then we'll have more energy to You know break down or to detoxify our bodies to build our skin's collagen to Remove toxins from the body so and then also when you have good digestion everything just flows a lot better And you feel a lot better and you it will be new out It will give you just more energy all along all around you won't feel bloated. So I'm not magic. I can't wait guys. This is gonna be a really great thing. I'm like wait This is gonna be I was also Instagram storing her so it was a lot of pressure. Thank you so much Um, oh, I didn't even talk my podcast if you guys don't follow me an Instagram or the podcast Instagram, please go follow Don't you I have a lot more energy now that I did earlier. Yeah, yeah Yeah, you were kind of dead. I was dead earlier. I was like I feel like I literally just couldn't get it together I don't know what was up, but I got better. So it's all fine now I'm like so into Instagram stories right now. It's like not even If my story I'm the person with the dots, you know So It's fine though. Like sometimes it's how it has to happen, you know So we're gonna record a podcast. I'm not sure when this will be up, but I have a podcast I love you so much if you guys don't listen to it. That's insane. You should definitely listen to it I like my podcast, but other my YouTube channel honestly, so check it out. Hey guys Okay, well anyways and I was recording back together going to zinc for dinner That's it. I'm like really hungry and tired and I need food so remember that like that's like one of the main things I remember is this is my last hurrah meal My favorite meal in LA the chicken label from zinc. It's incredible. You're salami toast Yeah, I'm like literally this is my favorite meal in the world. I'm gonna have to start changing it, but it's so good That is actually very on but yeah, I'm back home But I really quickly want to show you guys two lush products that I'm obsessed with I meant to show you these earlier But I forgot so the first thing I have is this rose argan body conditioner This stuff is literally the best stuff ever. It's basically just like a shower. So like a shower gel. I'll show you It's like thicker and it locks in the moisture in your skin and it smells amazing and I've used that a ton This whole tub is like 40. I wish I'd got the bigger one That's 50 so you know a lot much like a lot of more stuff But it's the best-smelling thing ever and your skin will look amazing and it just actually like you don't even need to We're lotion like that's how moisturized it keeps your skin and that's saying something stuff It's not both me and then this odify fresh face mask You guys only open this so I have water in it, which makes me so mad, but it's like Normally it's a little bit wet right now because I just use it But it's like dry like oatmeal kind of stuff and you just rub it together in the water And then you use it on your skin and it is the best thing you will ever use It is my favorite thing ever and I just use it in the shower, but I'm like obsessed with both of these Okay, I almost said good morning guys. I mean good night guys. I just got home I hope you guys enjoyed today's vlog. I'm really excited to start my new Like 21 day guide tomorrow So stay tuned for the vlogs if you guys want just like what I eat in a day videos where I'm like way more specific I'd be down to do those like a weekend So I at least have like a week of like what I've been eating and I kind of know But I'm also gonna be trying new recipes and stuff So like I feel like it's still fine to do what I eat when I'm like trying new things But if you guys want to check it out I will have Kenzie's guide and stuff linked down below and you guys can buy it and you can do it together And it's gonna be so fun and so many of you guys are like so excited that we are working together Like the secret Facebook group if you guys are not in it you need to be in it You're missing out the secret Facebook group You guys have been like going off and so excited and anytime I post with Kenzie you guys are like so excited So I'm really excited for this episode to come out I don't know when it will But subscribe to the podcast and listen to the other episodes We have so many really amazing ones coming out right now And I'm just so excited for the podcast and I just like love it. I just love life right now guys I really do But I love you guys so much. Hope you guys enjoyed today's vlog and I will see you in the morning. Bye


Alright, so my name is John Paul Posada, I'm the Educational Technologist for the Faculty of Engineering at UNSW, that's the University of New South Wales, and that's mouthful. For the past few years at UNSW, I've been working a lot to manage very large courses. And by very large, I mean a course of about 1,500 students. To do this, and to run these courses, it's mostly project-based courses, and a lot of teamwork that happens in these courses. So, students join projects, they form teams, and then they produce an outcome from those teams. And that's what we've been doing for the past about five to eight years. And with those needs, I needed some help from a developer and a very talented developer. That's him, Shox. Yeah, I'm Morgan Harris, I'm the Plug and Developer for the Faculty of Engineering at UNSW. My mic should be on. Yes, it is. Right, everyone can hear me? Yeah. Good. So, we've made some cool new teamwork plugins, and we're going to show you one of them. It's called Team Evaluation. So, what is Team Evaluation? Why should you bother with it? Well, think about what happens when you mark a group assignment. Well, you generally are going to go with this solution. Can everyone read that? Yeah, it's fine. You give everyone the same mark. That's not really optimal, and it's kind of not fair. This is something that a lot of students will complain about with group work. It's that they don't end up with a fair grade of complexity amount of work they did. Of course, you have the other option, which is to assess everyone individually, but that's not really feasible, especially with the kind of courses we're talking about where there's like 1,000 or 1,500 students. So, there is another way, and that is to ask them. Ask the best way to determine the amount of the contributions of each team member is to ask their teammates. And you actually should get them to ask themselves as well. You get them to self-reflect because that is actually a very important skill that you need to hone in higher education. And to do this, we basically take the approach of team-based learning. And we follow four main possible groups of this, which we think Team Evaluation addresses. And that's, first, we manage the group. And we manage the group well through early evaluation. So using Team Evaluation, we can give the team members early feedback so that they know how they're doing within their team, what their teammates think of their progress, and so on. Instead of waiting until the end of the product to give them feedback then, which really isn't going to be helpful because they're already done with their teamwork. And we give them timely feedbacks, as I mentioned. It comes throughout the whole process of their work together. This helps with accountability. So every individual within the team feels accountable to the team because they know that they're going through this process. It's not just about what they produce, but it's about how they produce that. And that's important. And Team Evaluation helps with that. And it also helps promote team development, which is part of the assignment design. And that really separates the outcome and the process. And we really want to measure both. So the outcome is important because it's the product that they're going to produce. But we also want to make sure that the process of getting to that outcome was also evaluated and something that they need to consider about how that went. Because you don't want one person in the team to feel responsible for the whole team's work and do all the work, basically. That's not what team works about. All right, so what actually is going to happen in Team Evaluation? So let's take a look at the regular first, the way we do things now. We assign a mark and everyone gets that mark. But these people aren't these faceless gray automatons, they're individuals. And they've all brought different things to the group. So you want to know who brought what. So how do you find out? Well, if you ask them the right questions, you'll find that they have some pretty well-formed opinions about who did what and how the group works together. And in aggregate, that actually forms a pretty decent data set. And if you throw self-reflection in as well, then you're well on your way to understanding that group dynamic. So how do we use these emojis to turn that into fairer grades? Well, that's where Team Evaluation comes in. Team Evaluation takes them and turns those opinions into numbers. And then we can use those numbers to adjust grades. So I'm going to take you quickly through a little worked example. It's a very simple one. There's just a sort of one-to-five question. One's the worst, five's the best. How much work did each team member put in? So for this one and only question, Bill has given himself a four out of five. He's a big fan of himself. Mary has given Mary three out of five. He's given Lee five. He thinks Lee's pretty good. Mary is less of a fan of Bill. She's given him a two. She's a big fan of herself, though. She's given herself a five out of five, and so on. And so we can see with all these scores a couple of important things. First of all, Roberto here has not completed the question, and that'll be important later. But also, these bars are all different lengths. And that's actually kind of a problem, because we're not doing a quantitative assessment here. We're actually asking them to compare teammates against each other. So really, all fives would be the same as all ones. Whether you think your teammates are all equally terrible or equally fantastic, it doesn't matter because they all put in an equal amount of work. So we have to fix this. We have to adjust these scores. And the way we do that is pretty simple. We just stretch the bars out so they all add up to 100, except they actually all add up to 125 because Roberto didn't mark anything. We actually need the sum of all these grades to be equal to the number of people in the group for a reason that will become obvious in a second. So after that, it's very simple. We just take all those bars and we just stack them on top of each other. And you end up with people's team evaluation score and a reflection of how much work they put in. So you can see Roberto is obviously super lazy. He didn't even complete the questionnaire. But Lee has done the bulk of the work. So how do we adjust the marks? Well, here's the mark we gave the group. So the group assignment received a 65. And normally, you just give 65 to everyone. You can see on the face of it, that's not fair. So we just multiply those numbers together and we get our adjusted grades that you saw earlier. So I know John Paul is going to tell you a bit about the existing tools and why you decided to go into it. Yeah, so basically, the state of play. The state of play is when I first started looking into this, there were a few tools out there that I took into consideration. So the first one was WebPA. And WebPA stands for Web Peer Assessment. It's out of Loughborough University. It's an open source tool. And it was my number one choice. And my number one choice because of the fact that it was an open source tool and the fact that it had a lively developer community and user community. The other tools that are important and worth mentioning are CATME. CATME is a whole suite of tools which really takes the whole team process from building a team to evaluating a team's performance. And it was a project that's been going on for a long time. It was a National Science Foundation grant funded project. It was free for a long time up until next year. Next year, they're going to start charging a small fee to students or per student, not to students. That the fact that it's not open source and not easily integrated into the systems that I use was a reason why I didn't choose that tool. The other tool is Spark Plus. Spark Plus is developed at a UTS. It does a very similar thing. It basically tries to even out the score based on how students feel their team members performed within the team. And the reason I didn't choose Spark Plus was also because of the fact that it's not open source, not easily integrated into our systems. So we used WebPA for a while. It was a good tool. It was open source. I was able to host it myself and manage it myself for a while until it became a little bit too much work, too much administrative work. I had to put users in, take user marks out, notify users of their marks, get Morgan to help me develop special reporting tools from it. Finally, we decided, let's make this a little more integrated into the systems that we use. So we put it into Moodle with a plugin that pretty much functions like an LTI. It basically sends our users over to WebPA and then sends their marks and responses back into Moodle to give them a final mark. That was good for a while. And then I decided, no, let's make it better. So Morgan made it better. So what's different about a Moodle plugin? Why would you bother doing a proper Moodle local plugin? Well, first of all, it's about integration. So a local plugin obviously is gonna integrate a lot better into the tools that are already in Moodle. So the most important thing is that you find it where you expect it, which is in the activity, the group assignment tool. And you can see right here, you've got your assignment on the top, you've done your submission, and right below it is team evaluation. It's right where you expect to find it. And the other thing about it, of course, is that it's extensible with subplugins because everyone loves plugins. And the M in Moodle stands for modular. We're all about the plugins. So it is almost entirely plugins. We've got the best plugins. We've got the amazing plugins. We've got the question error. The question error is the first thing you'll see in Team Evale, your students will fill it out, you'll create it. All the question types are plugins. Then not question bank plugins because question bank plugins have the idea of right and wrong, whereas these are a lot more ambiguous. So we've made our own plugin architecture for that. Activities are obviously plugins. But we've got a little thing inside an activity that wants to adopt Team Evale called an evaluation context and that's how it talks to the team evaluation. And then obviously we have to have question error feed into the activity plugin to adjust the grade. So we've got an evaluator. Even the evaluator is a plugin. There's only one at the moment, but there'll be more coming. And then obviously we've got to get some data out here. We've got to do evidence-based learning so we've got a bunch of reports and all the reports are also plugins. So the plugins we've got at the moment. There are two question types. They have the like it scale, sort of zero to 10, one to five, whatever number to whatever number you wanna do. And the comments type, that doesn't actually provide a value, but just allows you, your students to enter feedback that their peers can see. We've got in future coming versions, we'll have a split 100 where you have to sort of divide up a pie amongst your teammates and a contribution matrix which is sort of a big matrix of checkboxes and you can say who contributed what to the group assignment. Activities is actually only one built in moodle activity that supports group submission and that's the assignment plugin. There is also a third party, well, we developed in-house a version of Workshop that supports group submission and that is available on both the NetSpot GitHub and the version that supports Teamaval is available on my GitHub. You can find a link to that in the boot page for this talk. So those are the activities that are supported at the moment. If you develop your own activity plugin and you want to adopt Teamaval, please come talk to me and please read the implementer's guide on the GitHub. There is, as I said, only one Evaluator plugin. We call it Loughborough. It's based on WebPA. WebPA was developed at the University of Loughborough. It weighs self-assessment equally to peer assessment. So if you think that might be a problem, probably the best thing to do is turn off self-assessment. And it, as you saw earlier, is just basically a simple weighted mean of all the grades. And yeah, it's the worked example we saw earlier. That's exactly how the Loughborough Evaluator works. So in future versions, we will have some more Evaluators based on the plugin, based on the tools that John Paul mentioned earlier. We're gonna have a Camperdown one based on SparkBus. We have Raleigh based on Katmee. Noticing a theme. It's named after the place where it's from. And we're gonna have Kensington which is UNSW is in Kensington. So we're gonna sort of try to devise our own cool take on how to best evaluate those scores. And then we've got a bunch of report plugins. Oh, good, I did screenshot these. Got scores plugins, just the scores everyone got. We got responses. So that's like the actual detailed individual responses that everyone's given to those questions. And we've got feedback. And the cool thing about the feedback plugin, feedback report is these little switches down the side. Because what they do is allow you to reject feedback that you think the person it was given about might not want to read. It's a good way to sort of protect against trolls and bullies and people who just write the F words six times in a row. And yeah, well I did that. And in future versions, we're gonna have a self-assessment to peer assessment ratio. So you can see people who are like big in themselves up or maybe a bit down on themselves. And a outlier assessments report which would sort of try and highlight people who you think or who we think might be trying to gain the system. And we've got a bit of built-in API. Do I have time to come to this? Not really. I'll dive through it. If there's any developers, are there any developers in the room? There's like a handful. Okay, I'll dive through. So we just got a bit of that accept and reject before if you're developing a plugin that does feedback, you should opt into that. We've got report downloads. It's a really easy way to do report. Downloadable versions of reports. And readable response formats. This is actually really important because we do it with the comments question type. If you just had the entire comment, people could potentially dump huge amounts of text into your reports. And you might not want that if you don't wanna avoid the scroll of death or the sideways scroll of death, even worse one. So we just take the first 50 characters and give you a little button to expand it. So if you're developing a question type plugin, you should take advantage of that. And there is some cool API coming in the future as well. That's my Twitter handle. If you're a developer and you wanna get in contact with me, I've got a slightly more dev-focused version of this slideshow, but I didn't have time to give that presentation yesterday. But I'll do like a lightning talk or something. So how do you get it? Well, it's in public alpha right now. It is currently not feature complete and the API isn't totally stable. So for that reason, it's not on the mutual plugins directory yet. Part of the reason is we really need modules to adopt it. Again, if you're a developer, please come talk to us. But it will probably be in future with a script and instructions on how to modify the assignment plugin so that it will adopt team evaluation. And that is where you can download it. And if you can't remember that URL, I don't blame you. You can find that link on the Moodle Moot page for this talk. All right, thanks, anyone? Everyone, any questions? Nope. Great. Cheers everyone. Thank you.