Alright, we get started. So today I am going to be talking about Vitesse, I actually have some Vitesse stickers later in case you are interested, you can come and collect them from me. So my name is Sugu, I now have a new title, it is called CTO for Planet Scale Data, it is a new company that we started recently whose sole purpose is to support the Vitesse project. I have my co-founder here Jithin who is also sitting. And previously many of you probably know me as the software engineer from YouTube, I was at YouTube for over 11 years and that is where I co-created this project, it was around 2010 and have been working on it since then. So one disclaimer or warning is I may accidentally regress into thinking that I am still at YouTube, so if that happens wake me up and remind me that is not the case anymore, alright. So I am going to talk about a couple of problems that have been growing in the MySQL industry. So whenever I say MySQL, assume the base class MariaDB and all other variants of MySQL, sometimes you can even like abstract it out to any RDBMS. So the one problem that has been growing with MySQL instances is that they all run only on one box. Like when about like this has been now 8 years when we started the Vitesse project, we thought we were actually solving this problem for big companies like Facebook's and Twitter's because they are the ones who have a lot of data and they need to shard in everything. But over the years what we have noticed is that even small tiny companies are beginning to outgrow their single MySQL instance almost in no time and this is actually, we should have actually seen this coming, people have been talking about data exploding, big data and all that stuff and now it is really affecting the choices that people are making about storage solutions. Often they are looking at MySQL and they are thinking is this going to carry me all the way if my business grows suddenly in leaps and bounds or if I suddenly need to store a lot more data, will I be able to continue to work with MySQL and some of them are actually looking at other alternatives like what else is out there if it is not MySQL. And I will talk about some of the choices that I have seen people making and we basically I think we need to fix this problem and Vitesse is actually trying to help there. And the other big trend obviously is that the cloud is here and pretty much most RDBMS's are kind of left behind mainly because culturally or traditionally they are all built to run on a single machine, they have the whole machine for themselves and they have all the resources there. But in this new container world you are supposed to live within the smaller box that you are given and those containers do not stay up for 6 months and stuff like that. They go out, they go down any time they like, like cloud environments like to move your instance all over the place and you do not have the whole box anymore, sometimes somebody else starts doing a lot of I.O. and you are short in bandwidth and the fact that all this is happening changes the way your application has to work with the database. You cannot have these config files and static IPs that you use to connect to the database anymore, you need to have a more dynamic discoverable API that applications have to use to connect to the databases. And the other player now that you are now hearing more about or will hear more about is Kubernetes which is now pretty much being seen as the future operating system of distributed applications and there is actually no solution right now. There are some people working on operators and stuff to run RDBMSs in Kubernetes but right now there is no good solution if you just have a single MySQL instance just to move it there. So these are some of the problems and the cloud problem actually aggravates the previous problem of the transactional data exploding and is kind of you many people are like almost feeling stuck with MySQL. So what do I do with this thing right? So fear not, there are ways out, that is what I am here to talk about and so here are some of the solutions that people have used when they find that they have outgrown their single instance MySQL. By far the most common one is application charting and it is now becoming the least favorite option because one it requires your application to be rewritten and it means that you need additional engineering effort to be able to do this and the second problem which is even worse is you have to live with it for the rest of your life. That is actually now the problem that companies are beginning to feel and some of these companies are not really core technology companies, they are like retail shop and they are like a phone application and stuff. They do this application level charting and now they have this debt or burden that they have to carry and sometimes your shards outgrow a single instance then you have to reshard and they do not even know how to do that right. So there are various challenges with application charting and some people have said just forget all this just let us go to NoSQL at least it will scale, we will deal with every other problem that comes up with NoSQL but at least we will not be stuck staying in one box. There are also some paid solutions which I will not talk much about since it is an open source conference and they are also actually not all perfect that is all I can say for now if you can later come and talk to me about what solutions there are and what problems they have. There is actually a newer category that is called NewSQL, I am sure you all heard of it, Cockroach, TIDB and Yugabhite are some new companies that are actually building a ground up SQL scalable solution. The main drawback is actually a time problem is that they are all new, they are just beginning to get their first versions of SQL running and if you look at MySQL it is been tuned for over like what 20 years now and these people are where MySQL was like approximately 20 years ago. So it will be a while before they get there but they will if they have enough financial backing they will eventually get there but right now it may not be immediately possible to go into production with them, it is a kind of my opinion obviously these companies are here, they will probably help you go to production to the extent that they can. And the other problem is that these companies, some of these companies have a open source offering and commercial offering and there is, we do not know whether you can just take the open source and run with it in production yet. So there is not enough data to know that that is possible or whether there are any examples that exist. So those are some of the things that I see with the new SQL but it is definitely something to watch out for. So at this point today if I have to make a data decision, the way I see it is these are all inferior solutions to what MySQL can do for you. So functionally if you look at MySQL and what it can do for you, you move away from that and come into this, you are actually downgrading in terms of functionality, right? And why are you doing it? You are doing it only because you have to scale. So the obvious question is why not both? So that is basically what Wittes tries to do is gives you all the functionality and power that MySQL has but at the same time you can keep scaling indefinitely. So why is, why Wittes capable of giving you both is because underneath it just runs MySQL. You basically, if you have a MySQL, you deploy Wittes on top of it and then afterwards you can reshard your database without the application knowing much about it. And how big can we get with MySQL? It's pretty big. We call it massive scale which is pretty large. Actually I was talking to Yoshinori earlier. That's what he defines, Yoshinori is from Facebook. He defines that as a medium scale. But we call it massive scale because we call it, oh we can go to tens of thousands of nodes. But the cool stuff is that it is open source and we have been working on it for 8 years now. So a lot of time and effort has been spent to make the project production viable. You can take what is available in Wittes, take it to production and scale it to millions of QPS and it will run for you, no problem. And for the longest time YouTube was the only company that was using Wittes but now we have very large, not very large, a decent number of companies but running at very large scale. Some of them are, many of them are running a 6 digit QPS in their instances and many of the companies have completely rely on Wittes for all their data which is super exciting. And which companies are using Wittes? Here is a list. There is actually more. These are the companies that have given us permission to use their logos. As you can see there is a name you may not recognize which is JD.com but it is because it is a Chinese portal. It turns out that they are the second largest portal in China after Alibaba and they have completely migrated their business over to Wittes. They run over 300 key spaces or databases and they run again comfortably 6 digit QPS. You have Slack which is actually made a major commitment both in terms of engineering effort and operations and they have actually contributed a lot to the project too. They are in the process of migrating to Wittes. There are actually plenty of talks that Slack has given. We even have Guido here who was working on that migration. There is a few others. There is HubSpot and Squire. Squire for example uses, is using Wittes for their Squire cache application and they are already sharded. They started with a single shard and then deployed with this and started resharding it. But the more important thing here is if you see there are some names you will not recognize. These are actually pretty small companies but they have huge data. Many of them again run pretty large QPS and this is an indication of what is coming, how the industry is changing. It is not the YouTube's anymore that require massive scalability. There are some small companies probably that I have never heard of with us and they probably went to the route that you chose some of the routes that I showed before. But I think there are lot more companies that are coming up that are going to require big data and are going to wonder can I go to MySQL, will it scale for me when I need to and at least Wittes is there to take care of them when that situation comes. There is actually a third kind of need that we are beginning to see. Many of these big companies like Pinterest, Twitter, they already have sharded, they already have sharded solutions but their secondary databases is beginning to grow and they need sharding for those databases and many of them are saying I do not want to go through this application sharding exercise anymore and some of them are actually coming to Wittes to see if they can actually deploy Wittes on top of their MySQL instance and do the sharding. So first of all good, alright. So what does Wittes do? Actually this slide usually takes me 1 hour to cover if I drill down into all the stuff that Wittes can do, like 8 years of development you can develop a lot of things in that time. The one thing, so I have categorized it into 3 parts. The one thing that Wittes will do is make your MySQL happier. So if you are running tens of thousands of nodes, simple tasks that you usually expect a human can do should not be done by a human because if problems appear at that scale it is very hard to manage. So many of that thing about query mediation, making sure that queries do not run for too long, making sure that transactions do not run for too long are all built into the proxy that Wittes has. If there is a sudden overload, Wittes can take care of throttling it, throwing away, throwing errors to the application saying I am overloaded, I am not accepting any more transactions. So all those things are built into Wittes. The other thing is, this actually interesting story, Wittes was originally in 2010 we built it for actually bare metal, we were running in YouTube, YouTube's own data centers and at some point of time we had to migrate to Google's work cloud. So if you are a Google, how many of you have worked at Google before? Oh, there you go. So if you write an application, oh you, Mark himself. So if you write an application that runs on Google Cloud, it is pretty much guaranteed that the application cannot be open sourced because the Google ecosystem is so unique and there are so many internal dependencies, they are such that if you write an application in Google Cloud, you cannot open source it. So we had an interesting reverse problem is we have an open source project that runs on bare metal and we have to make that run into Google's cloud. So how did we do that? We basically built an abstraction layer for every Google Cloud feature that we had to integrate with and saved this open source project and fought for that being for remaining open source. We will say we will do whatever it takes to build these abstractions to make sure that Wittes runs in the cloud but we want to keep this project as an open source project. So it was kind of hard but we succeeded but it paid off, the reason why it paid off is because when Kubernetes came, we were ready for it, we said oh cloud no problem, we just build all these other pieces, put them together, we are ready for the cloud. So and right now many of those companies that I showed you actually run on Kubernetes, run on the cloud, run on Docker, I mean for the longest time people were wondering can MySQL run on Docker? We do huge QPS on Docker, some of these people, GKE, AWS, Kubernetes AWS, Mesos, Azure, like you take all these things, combine them differently, there is probably configuration there that Wittes runs on. And finally icing on the cake is it is indefinitely scalable so you do not have to worry about that part. And even in the indefinitely scalable part, you can say indefinitely scalable but the important part is that you do not have to rewrite your entire application to be able to scale it because all the queries that you send to MySQL, to send to Wittes, it will figure out how to break it up and send it to different parts and get you the results back as if it was a unified database, yes. I will answer that question on my next slide. And the other part is that even if you once you have sharded your system and later you suddenly realize that you are running out of capacity and you want to reshard, there are workflows that will do this for you with application noticing absolutely no change at all. We do this, YouTube does this multiple times, like 2 to 3 times, like every 2 to 3 months they do a resharding, major resharding and we used to make big announcements before now nobody tells anyone anything, they just go and reshard it and there is usually never any issues. So this is the MySQL, this is the Wittes architecture. So the most important part of this architecture is the Wittigate. The Wittigate speaks the MySQL protocol and the app server instead of connecting to MySQL just connects to Wittigate and nothing else, after that it just treats the Wittigate that it is connected to as a giant database except that underneath it is actually a bunch of smaller databases and this architecture actually I can spend a lot of time, we really really like it. Each and every one of these pieces plays a very important role in this whole thing especially for example the lock server is the one that makes it possible for Wittes to run in the cloud so comfortably. I am going to rush because I will be soon running out of time and I wanted to cover a couple more things. I wanted to cover the pluggable architecture I have already covered, it is actually the most important thing that we had to take care of while making this architecture pluggable is to not compromise on performance. So in spite of all this pluggability Wittes latency is still pretty low, it adds about 2 milliseconds as total overhead to your database but then you get this indefinite scalability. And in terms of project I wanted to cover how we manage the project because that is actually the best part, the reason why I keep wanting to go back and keep working on this project is because we have maintained very high standards in this project to the extent that some people get irritated especially when we start insisting on unit tests and strict code coverage requirements they are like do I have to do all this I do not care this thing people are going to take and run into production every code that you write that is going to serve a query it has to be tested it has to be covered and if needed you need to write end to end tests also and so we followed this discipline from day one and have been very strict about it all coding standards are enforced we have all kinds of CI tools that make sure that you do not break those things and if your code is not readable we do not even take your pull request. So but then that is actually caused over time some really very high quality contributions to the project and this is big news that is coming CNCF actually is going to accept Wittes as one of its projects it is actually pretty much a done deal the official announcement is happening tomorrow so keep watching various blog posts they took the CNCF a long time to accept Wittes almost a year and the reason is because they could not find a project similar to Wittes they could not find anything else to compare it with because in reality there is not one right now I do not know why but they just could not and so they did a lot of due diligence they interviewed all many users they looked at maturity they looked at contributors and then finally said yeah it is right there is no project like Wittes but it is awesome it really does what it is supposed to do and after all the due diligence we are now reaching the end tomorrow we will be accepted in CNCF which means that Wittes will get to benefit from all the cool stuff that CNCF as personally for me the reason why I am excited is because until today Wittes was always seen as a YouTube project but now it is becoming a truly community owned project so I am hoping that this will attract more contributors and attention in terms of adoption. So in terms of road map Wittes is not perfect but so there are still things that need to be done so if you are a good open source person if you know query engines if you know things like that we would like to welcome your contributions the one place where Wittes is short on is some cross shot queries it says sorry I cannot handle it it does most of them but not all of them if you do some complex aggregation with ordering and sorting it says please rewrite your query it is such a way that I can do it. So help with migration tools and configurability those are the areas that we think we need to focus on. In conclusion the main message is if you are hesitating about using MySQL because it cannot scale do not worry about it we have it covered and there is plenty of examples of people having scaled with Wittes at really really large scales I was actually telling somebody else today if somebody came and if your engineer came and told you I want to store 100 terabytes of data what would you tell them oh is there any way you can prune it is there way you can reduce it say no do not worry bring it on we will take care of it you know so that is the kind of attitude we should have when people come to us for storing data or running in the cloud is also not a problem and finally help build Wittes because it is an awesome tool yeah that is it thank you very much.


Cool, it's the audio coming out fine Yeah Cool, so this is just a short overview of writing shell scripts in closure or to be more specific closure script I had some Data manipulation from Jason files and endpoints that I was doing and it was getting really hairy doing it with a Combination of jq and javascript and some shell tools and I thought well closure has a lot of great You know standard library functions for manipulating data So let me try using closure or closure script to write my shell scripts And then I started looking into this and I'll just share what I've learned and Some of the useful functionalities that I found for doing that So this is writing shell scripts in closure script using plank What is plank? Plank is another repel Similar to your lining in repel or your boot repel plank is this It's a closure script specific repel I think it runs on top of the Javascript core engine and It's available for at least Mac OS and Linux. I'm not sure if you can get it on Windows They don't say that so maybe not but it's very lightweight and For shell scripts the thing that is really important is it starts really fast so well, let's do a comparison if you start your line repel versus your plank repel So line repel Still taking time blank is ready Right, so that's the main difference for me in shell scripts You don't want to wait for the repel to start and to stop. You just want it to completely execute very quickly and Plank gives you that ability So being a closure script repel closure script traditionally lacks a lot of the tools for interacting with files and the shell and things like that Because it's targeted more at the browsers as the first first sort of target platform But plank includes a lot of these things. So I'll walk through what all those functions are in plank and how they can be used So the first one is Invoking external shell tools. So if you're writing a shell script and you want to You know invoke some shell tool that already exists. You can do that in plank using the sh function It's built into the plank shell namespace So it's just straightforward just the name of the command and the arguments and what it gives you back is This triple of the exit status of the command The standard output what I was printed there and whatever went to standard era One thing that keeps tripping me up when I don't use this for a while is Remember to separate the command and all the arguments as separate strings if you send it off as one string it's going to give you an error that Makes no sense and it's really hard to debug so the launch path not accessible But I spent like a lot of time Breaking my head over that error and googling it and stack overflowing it and then I realized that I just have to separate the command and the arguments and And just like any other closure thing once you've Executed this you can manipulate it however you like you can print it out or you know anything else Cool the second thing that's quite useful for shell scripts is Getting all the command line arguments that are passed into the shell script in plank They go into this global variable called command line args So I have this script here called args.clgs and all it does is it prints out the command line args So if I just execute that With these are some arguments it's just gonna print that right back out cool so far any questions The next one is writing and reading files So enclosure script slurp and spit that exist enclosure are not present by default But plank includes these in the plank core namespace So they work pretty much exactly how they work in Sorry, let me just stop this again. So they work pretty much exactly how they work in your regular closure stuff Once you've required it you can just use Slurp so I have this file. Let me show it to you It's just some lyrics from a song. So when you slurp that you get that as a string and Similarly, if you take some data you want to put it in a file You can spit that back out with plank and now you can see there is a math dot Text in here with the result So pretty regular stuff One nice feature that I like with slurp is you can give it a URI So if it's like maybe some JSON data or an endpoint or an API that you're accessing You don't have to shell out to curl every time if it's a get request You can just slurp it So this for instance will go to that URI and give back whatever response it is in this case It's a JSON response. We can go to that website and see Yeah, so it's just a regular JSON response and Plank will just kind of download that a remote endpoint for you and give you the result in a string JSON string The next one is getting data from standard input So usually when I write the shell scripts, I try to make them as Independent from the data they're reading as possible So I try to like pipe the data in rather than read from a specific file or something like that so in plank you can do that by Requiring the in special variable and the slurp method so you can essentially slurp the in Stream and that way you can read whatever standard input has So for example, we have this file standard stdn.cljs so I require slurp and in and Just in this case. It's like an echo.  You don't need any external libraries for that You can so as an example What we'll do is again, I forgot to start plank So Require this so now I'm setting data to be this data from the remote endpoint So it's just a string of JSON text and then This is the JS interrupt syntax in in closure script So from the JSON global variable called the parse method Oops, sorry from the JSON global variable called the parse method on the data Variable and it'll give you this Passed version of that JSON string But if you notice here, you see this map here is actually tagged with this hash JS syntax This is actually still a JS Map and a JS array. It's not a closure map or a closure array or a closure vector so when you pass using the JSON parse you're still getting that Native sort of JS based map and array structure, which is not very nice for us because we want to be able to manipulate that using closures Functions and we want, you know to be able to use all the immutable data structures So the way you do that is this this is a closure script feature not a plank feature. There are these global methods JS to CLJ and there's another one CLJ to JS. So what this does is take that native JS object and map or basically an entire structure of that It can be as deeply nested as as as it is and if you pass it to that it'll then convert it into The closure equivalent. So now it's a closure data structure and you can see there's no hash JS and stuff anymore One thing that we have lost is The previously when we passed it the the keys in the map were keywords Now they've changed into strings The keywords are still pretty useful because keywords in jail in closure functions and you can use them to access the maps so If you want to keep them as keywords, just remember to pass in the extra option keywordized keys as true and then you'll get the same basically your closure version of the JS maps and Objects cool so far All right. So the second part of Jason. So that was the parsing part. This is the serialization or the stringifying part So CLJ to JS will take a closure map convert it to a JS map It basically just looks like the same thing with hash JS in front of it You can manually construct these literals by hand if you want you could do hash JS One two three four and I've got that JS literal but most of the time It's preferable to just work with the closure data structures and then at the end convert them back out to JS It's a similar to before instead of dot bars this time we call dot string if I and it takes the closure map converts it to a JS map and then string if serializes it using JS Using the Jason string if I method if you do it directly without the CLJ to JS transformation then you get a whole bunch of crap and most of the time I think we don't want that so Remember if you're seeing that whole bunch of crap, it's probably because that CLJ to JS has been forgotten One thing that people don't generally recommend But I find is okay to do in shell scripts is you can do a shorthand version of the parse and string if I So instead of doing dot bars JS slash Jason You can directly use Jason dot bars and Jason dot string if I the reason it's not recommended is if you or some other Library overrides the Jason global then it's not gonna it's not gonna inform you that something's gone wrong Whereas the other method does but mostly my shell scripts have been at least the ones I've been writing have been pretty small And I know the dependencies they have so that sort of succinctness is just is worth it in this case But yeah, it works basically the same way So it'll give you the same results One thing I found myself making quite heavy use of is the threading macros in closure I find basically when you're writing shell scripts and you know piping your data through various tools The threading macro kind of emulates that so you take a bunch of data in this case a URL pipe it through slurp and then pass it with Jason parse and then Pass it to JS to CLJ Q what I think the keys so it inverts that nested function call and kind of flattens it out and for When you have long bunch of Transformation functions, it makes it very easy to see all the steps in the transformations You can see okay now. I'm getting the key now. I'm you know reducing it to some other value now. I'm manipulating it some way So instead of this nested it's so this and this are basically the same But it's I find when it gets very nested it's much easier to read the threading macro version me Yeah, so This one and this one Will give the same result Has anyone used the threading macro before yeah? So if you're not familiar with it what it does is whatever you put as the first argument This one it's going to then take that result and pass it as the first argument to the next function It's going to manipulate that with that function and then pass it as the first argument to the next function And this is where I also find the json dot parse is a bit nicer because if you're using dot parse JS slash Jason Then Threading macro will actually try to hang on yeah, the threading macro will actually try to place it somewhere here and That's going to mess up the flow. You have to wrap it in a in an anonymous function or use the other threading macro The thread last instead of thread first Cool, so that kind of wraps it up. I have an example script that I can show you So I'll walk through what this does So basically I just kind of it's a very contrived example, but I have here a file called albums.json it's Jason a file which has an array and each element of the array is an object with an artist and an album and What this script will try to do is for every item it So basically tries to count all the number of albums each artist has and then sort it in descending order so the Yeah, so the artist with the most albums is going to be on top and then the next one and the next one So we start with this she bang line, so it tells the script to be invoked with the plank interpreter Then I require slurp and in and Here I've kind of separated the input parsing section So getting the input json converting it to closure structure and storing it in a variable called input and the reason is that Well part of the reason is What I want to do in the output requires the thread last macro and what I want to do in the input requires the thread first macro And I've separated those things out The other option is there's another version of the threading macro called the as threading macro And I'll show you what it looks like. This is the same file, but using the as threading macro And what that does is? It still lets you pipe your data through that pipeline of functions But now you give it a name you give it a sort of placeholder symbol. So you say my input is called D after I slurp This D argument whatever that result is now that's going to be called D And then after json parsing it whatever I get from that that's going to be called D so now I can kind of Whether that D is the first argument of the next function or the last argument or whatever argument doesn't matter You can just put it in the right place and you still get this pipeline of functions it Has a few more parenthesis than the other version, but I think Like potato potato it has its benefits as well So here what's happening is you read the standard input parse it Converted to a closure or data structure and then to the output you get all the artists from each element of that array Frequencies is a closure very neat function that basically takes in It's built into the standard library takes in a vector and for every item in that vector It'll tell you how many times it appears So it returns your map. So here it says the number one appears once two appears once Three appears twice four appears once and so on So if you have an array of strings for instance It'll tell you cat appears twice dog appears once So I'm just piping that list of artists to frequencies So it counts how many times it appears and then I'm sorting it by the second element because in frequencies The first element is the thing itself. The second element is the number. I'm sorting those numbers in descending order and Then finally for every item in the output. I just print out the frequency and the name So all put together it looks like this So I can pipe it to Sample that CLJS And that gives I realized after contriving this example that it's fairly easy to do this with normal shell tools as well you can basically Sort of parse the Jason by hand by cutting on the colons and the and the double quotes but yeah, it's it's more fun to do in closure script and And for the things that I initially tried writing these in closure script Those are quite quite involved with a lot of group groupings of things happening a lot of merging of maps happening Which closure was quite useful for? But yeah, yep, that's basically it. I think I don't have Yeah, that's all any questions The editor is Vim the terminal is just using Tmax and Vim has a plug-in that like basically lets it Kind of shell out to Tmax and execute stuff in Tmax So here I can just say so I can type it in Vim and then it'll execute it in the pain below and Can zoom the pain then stuff like that? It's called Vimax Yeah, yeah The presentation is actually just a closure script file if you see here and If I zoom it out, you'll see it's just a really long file with a lot of spaces in between Yeah, I made it pretty manually and the what I did was If you notice every slide starts with this comma Followed by the the title of the slide, right? So I have a little function. I have a little mapping here in my VMRC So basically it Looks for two line breaks followed by a semicolon and a space and it jumps to that So it jumps to the top of that so when I press that key binding it goes to the next slide the next slide Yeah, so it's it. Yeah, no not much is actually happening here Oh The fonts I actually found this program while I was looking at something else called toilet so if you take a piece of text and Send it to this program called toilet It'll format it nicely for you and you can pass like So they have some pre-built fonts and things like that So that's that's how I got that Toilet and then you can say some there's some arguments for colors and stuff like that Yeah That's about it Cool anything else All right. Thank you for your time. I'm feel free to ask me later


For me, it was way bigger than basketball. I wanted to help kids who didn't have a lot of hope. And I knew through sports, you could spark something. And I wanted to do that. You're in a predominantly white area. They don't want to be beat by black boys. We just differently talk different. I do feel it's wrong to choose. And that's what it is. They are recruiting a ball team. You got a black kid from New York making Billy Bob here look bad.  The situation in Knott County has become violent and could get worse.  The ball players dressed up as gorillas. This is stuff that should have made the national news. Sometimes I want to quit. And they're asked from all sports, did this just happen? How do you fire a coach that just won an all-estate championship?  But use it as fuel because they hate it on you. This is my team. These are my kids. We got your back. We got your back. We got your back. We got your back. We got your back. We got your back.


Ie, dylai. Fi'n가요 i'n gweithio gyda ni'n cael am yr eich cyffredin iawn. Mae'n fuddwch chi'n grannu. Rydyn ni'n ddynch i'n ddpanpeidwyd ac amroeddiadu cyfleon hwn i unipro. Dyna'r eich cyffredin iawn o'r oedd o'r ffluodol mewn rhain o'r holl, oedd o'r camau gan gobiadu chi'n g seamless. Mae'n ddim amser mwyaf o'r nghyrchu dda ni, hynny i'r cymdeiligau chi arno a oedd yn ffliweddo'iım o'r ddod o fflusiol, may be time for a few questions or maybe not now because we've had tech issues. So, I'm not here to talk about an unusual situation, an unusual situation would be for a fully mature team operating on a single product from the same space with well established communication methods and execution methods as well. More realistically you might be joining a team, or you might be onboarding a new team member or even generating a new team in a greenfield environment across different countries and across different languages. You might have difficulty with your laptop or somebody else's laptop, which I'm having. You might have time zones of an hour or even 10 hours apart. Don't even get me started on time zones that start with a half hour. You might have seniors and juniors who need aligning and onboarding despite client commitments, and you might very well be dealing with people who have varying of familiarity with agile processes as pertain to your company, or to your project, or to your client. And given that these are people not machines, they will also have varying levels of productivity, accuracy, commitment and knowledge from each other that you'll need to factor in. The programme I'll be talking about over the next few minutes has some 40 developers and QA, six scrum masters and six product managers from Unipro, a dozen UX and design staff and six product owners from the client side or from other agencies. We've got up at eight of those other agencies contributing data, insights, code, content, messaging a strategy at any one time, and all the usual hangers on, like programme managers and client services and delivery directors whose role has frankly never been made particularly clear. All these people are ostensibly working towards the same aim and in roughly the same fashion, and all those agencies are also trying to stay profitable with an eye on contract renewal, which can easily get in the way of the programme's actual aims. So I'm going to talk for a little bit today about some of the things that we find work well, some of the things that we could keep doing better and I hope this will give you an insight into building scalable teams using scalable processes. Okay, who's Unipro? Well, we had our 20-year anniversary just a couple of weeks ago. That was celebrated in all five of our locations across the world. We've mainly come from a systems integration and database background and what we really do is build websites that deliver to our client goals. Standard. We're a development partner to global organisations and we have little or no design or UX function as you've probably realised by the low production values on my deck. There's about 110 people in the company and the programme I'm talking about today touches over half of them in some capacity. It's split into eight streams, including a separate stream for rollout. You're not supposed to be able to read this, don't worry about it. That means eight Scrum teams, most of which have their own Scrum master. The client stands up the product owner and we stand up the Scrum master, the product manager, the developers and the QA. A partner agency provides the UX and the designers. All resource is ring ffans across the Scrum team and allocated several sprints in advance via the quarterly sprint planning process. So resource is one thing that people don't tend to have to worry about, which is quite refreshing. This visualisation shows data usage across the globe over a 24-hour period for those who are interested. Unipro have worked very hard and frankly missed out on a lot of profit by keeping all the developers in one room as much as possible. We started with one office in Chichester. Hang on, there's an animation. Now we have three in Chichester, two in London, one in Australia and a wholly-owned testing facility in Mauritius. So everyone in the room has probably experienced issues with having clients and Scrum masters in one country and developers in another. In our Scrum environments, the benefits of being in the same room for stand-ups, collaboration, ceremonies and execution are really obvious for us all to see. Each of our Scrum teams have their own room with the Scrum master and product manager in it and the developers in the same room too. They all have their own virtual conference setup for communicating with the client, ceremonies, other agencies and testing and they will have their own permanent whiteboards and walls for their planning and their reference. The client also has their own room in our Chichester office and they're invited to physically attend all appropriate ceremonies and releases so they can be on-site for QA and UAT. Co-location is obviously very agile for developers and designers. They can sit together and nut things out but only if they're given the authority to do so by the client and the programme. Lean UX actually has a chance of working if you put people in the same room and we all know that the agile manifesto encourages more face-to-face working methods. All that being said, what's bad about co-location? We put one of our Scrum teams, the mobile team, into a partner agency, the UX and design agency because we felt that the mobile team would benefit a lot from having always on access to a design team for that. It was true to a point but we learned we're still two teams and two agencies and it became hard to remember this when we were guests in someone else's house. There were never enough meeting rooms and although the designers and developers sat on the same pod, they didn't work together enough at the same desk regular enough for us to feel a benefit. Meetings became unfocused and basic Scrum principles were forgotten. The Scrum Masters Authority was undermined and a lot of people became a bit sloped shoulder about their responsibilities. We had to step in and do a full Scrum research. The situation we discovered at that partner agency has some interesting parallels with the client relationship too. They're genuinely trying to embrace the notion that we're all one team. It's the product owner's responsibility to commission the Scrum team's work for the sprint and if the sprint or the program fall behind, the product owner is the single ringable neck. However, the product owners are all humans and they're all answering to their own bosses and they're the same people that we answer to as well. So despite strong encouragement, the product owners find it very hard to take the blame for poor output when there are so many other people involved who they could do it to instead. After all, we're the digital experts in tech or in UX and they're just people that know their product. We have to bear in mind that they have careers and pride and other agendas that we might not know about. So things can be going just fine. A bad sprint can be perfectly well explained away and the retro run well to prevent repeated errors and then the Scrum master gets thrown under the bus at the next sprint report meeting. We can also be stuck in a situation where our co-agency see our large team as a chance to increase their own involvement and missing the opportunities to throw an elbow in our direction if it might present a revenue opportunity for them. I think we've all been there. I have to say it does keep everyone honest. If the client just had one agency on this, there'd be missing all the checks and balances that come from having multiple agencies running disparate processes and a QA and diligence on everything that happens. The client and the agency seniors also understand that the teams need to be self-organising and that we are facilitators of this. So we maintain very transparent, very informal, safe space back channels to speak candidly about difficult subjects for the good of the programme and steer the team to self-organise, if you're not any, to adjust or improve any pinch points that they might have. Self-organising teams tend to level out and produce the kind of output and governance that's necessary for their project. That's the definition of it. That's fine across one Scrum team. It's fine across maybe a couple if there are common touch points. We have multiple Scrum teams interfacing with multiple product owners. Each of them have enforced or not enforced different methods of planning, reporting, jeera use, acceptance criteria, sizing communications. The list goes on. Again, in a fully autonomous environment, that's absolutely fine, but the client seniors struggle to get a view of the programme's progress and efficiency. Credit to them once again for operating according to very pure Scrum principles, but they're still running a business. The two things that tend to produce sticking points are when they ask for more output than a single team can provide or when they try to get involved in reallocating resource across multiple teams in a hope-for-the-best kind of way. When this happens, it's our responsibility to help them acknowledge that quality is a fixed constant and therefore scope should be adjusted or highlighting and pushing back when instead of agreeing that a small improvement in shipable product is desirable or they adjust the nature of the shipable product without proper planning. This tends to be when the agile manifesto and the minimum viable product clash with real life. I'll leave that there for a minute. I'm going to talk next about roles within the programme. I've always liked this image. It's remarkably accurate in the way that people see themselves and the way that other people see each other. That's me right in the middle, project manager. I'm going to focus on the difference between designers and developers as this is the area that tends to cause the most friction and the two that tend to be the farthest apart. Okay. Design dreams, development wake-ups. The product owner will identify for us a vision and sometimes that can be just a couple of words like voice ordering. It's then the product owner and the product manager's job to turn that vision into executable goals. The UX and the design staff are a key part of this and tend to be the first people the product owner will talk to. The design process, any design is in? Okay, cool. The design process is... A different slide. The design process is relatively simple compared to the development process. As you can see here, it's basically sketching and colouring in some unicorns and then there's this word here, build, which is represented by this much more complex process down here where all those unicorns have to be turned into reality using the tools available at the time plus a lot of diligence and testing. So a lot of our unicorns tend to end up looking a little bit like this. How can we prevent that? Well, we can get involved earlier. We can help the product owner and the designers understand what's possible now and what's only possible after a lot of work. We can help to prototype and test so that we're not just producing one person's interpretation of a vision but the correct interpretation of a vision based on solid data and testing. Most importantly, as a business, we can make sure that it's understood that whether we produce a unicorn or a pink rhino, we've all agreed that the end product fits and meets the acceptance criteria in more than one interpretable way. I mentioned a little bit earlier that one of the ways that we can help the programme is to understand when in-sprints adjustments need to be properly re-planned. In a sprint, every action will have a reaction. According to the basic rules, accepting more points into a sprint usually means that something of equal value will be discarded. Of course, that message tends to get lost amongst the humans along the lines of, can you just or best efforts? Then we end up with technical debt at the end of a sprint, a task that could have been completed during sprint 10, rolls over to sprint 11, despite sprint 11 already having been fully planned. Suddenly you're facing 10 days' work in 9 days. This repeats in sprint 12 because you were over-spread, stretched in sprint 11 and everything's already buggy. So now you're down 2 days. It doesn't take very many sprints, in fact it takes 4 sprints, before all you can accept is technical debt and no feature enhancement at all.  It gets bigger and dirtier every time it rolls and at some point it's going to own up in someone's face. The client has to understand that the very nature of sprints means this can't happen. An understanding has to be supported by the scrum master and by the product owner. In the name of the program, pru-gram, is key, as although on a human level failing in one task might seem bad at the time, taking the whole team out for an entire sprint to fix what should have been manageable levels of tasks is going to impact the program much more heavily. So we have to reinforce that every time somebody has to adjust even the smallest of things. Okay, stepping a confident path towards openness and transparency internally, including the client. Transparent communications and reporting are something that we're all familiar with as a concept. Our bosses on our clients will all say they want us to be transparent, but that tends to end when the reports are bad. In this program, the client genuinely sees us as a partner, not as a vendor. This massively helps us with transparency because it means we can also treat the client as a partner, not as a client from a commercial point of view. In turn, this means that we're not aggressively trying to save money, but to spend it in the most effective way possible with their agreement. The whole company are freed up from difficult end-of-month deliverables and profit margin scraping and we're never challenged to deliver something substandard in order to submit an invoice. As we're planning our feature enhancement and releases years in advance, we're able to play a long game rather than a project-to-project game. Inevitably, of course, this has an impact on our approach to sprints and sprint planning. As often our teams see their work as small iterations of bigger programs and poor habits creep in because they're not operating according to sprints. This is where constant project hygiene appraisal and resets are vital to keep everyone engaged on the sprint level. We're able to act as the voice of reason between our staff and the client and the other agencies, as we're the ones who have an eye on how everything ties together throughout the program. There's no benefit to us changing the message. We can't hide behind the code. It's also true to say that mistakes are a very agile way of operating. Apart from anything else, the truth is the easiest thing to remember, so it's best to stick to it. Using common language and definitions. It's really very important that everyone on the program is talking in common language. Although it's not very agile to have documentation, we have removed a lot of friction by establishing things like races and jargon busting glossaries and ways of working and roles and responsibility documents. We usually have more than one person who could be seen as the end-of-most of almost anything. Delivery director versus program manager or account director. Development manager versus quality manager. So it's important there's also a clearly defined process owner and an escalation tree. Most importantly of all is how everyone knows when we are done done. And that comes from the original story. The product owner needs to tell everyone what the acceptance criteria are and there are so many people involved in the process of defining and executing each ticket autonomously that has to start right from the beginning. The autonomy is key and that's where the agile can come back in. It's a framework, not a set of rules. You're free to play within design guidelines, just don't break out of it. Here are some of the tools that we use for planning and some of the tools that we use for executing and some of the tools that we use for communicating. I didn't have room to fit them all in. Town familiar? This is probably nowhere close to all of the tools that we use on a day-by-day basis much less across the sprints in the program. Everyone's got their own preferred method of talking to each other. I've been on, as I'm sure you have, calls where I've got one method for voice and screen share and another method for typing and that changes from a sprint to sprint basis. We try to cut these down but realistically if a tool is right for the job then you'll generate more friction trying to prevent its use than you will absorbing it. Some tools, Slack for example we can commission new builds on UAT via Slackbot and it can surface communications to us via Skype, so that's nice. But in reality it doesn't matter if we're exchanging messages using coloured stones or skywriting or a t-shirt cannon as long as it gets the job done correctly. Somewhere deep in the Angelo Manifesto there is still a very strong instruction to allow time for knowledge transfer, cross training, collaboration amongst developers and a decent work-life balance. But sprint mentality and commercial reality also tend to confound this a little bit unless a strong scrum master books time within the sprint for it. At Unifer we do have islands of success here. It's an area of continuous improvement above all others. Ironically we've actually had to train people out of the notion that they have to go hammering into their code 24-7 and to take time out to reflect and to learn. Okay. I've called up the time with the technical issues. These are the takeaways then. Make geography work for you, don't fight it. We found that we had a client that was making demands on us in Australia so we hired someone in Australia. Straightforward. Maintain cordial back channels. Think house of cards if necessary and just call people and say what would make you happy here. Don't go spending a lot of effort trying to make people happy until you find out what that is. Homogenise your ways of working so that everyone is doing the same thing. It'll reduce friction and make people more comfortable with their escalation processes. Enforce a digital design standard. Get involved early. Developers who understand what the end product should look like. You've got a better chance of actually creating it to look correctly.  Be transparent and honest about everything. It really is the only way you're ever going to get anything done. Establish and enforce common language. No need to go into that any deeper. Use the right tools.  Just pick something up. Keep up with new developments. Link them together. If there's a tool that links together your estimates and your actual effort done within JIRA and services of timesheet like Tempo, then use it. Then you don't have to have duplication of data entry and you avoid that pinch point. Allow time for training and the work life balance. Our client allows us to schedule the developers to a maximum of 75% across a sprint because they understand that meetings and training are important and they're going to be a better product. That's pretty grown up of them. But it's something that we have to actually work quite hard to enforce ourselves and always keep analysing and adapting and changing and challenging. Those are all of the takeaways. We've got a few minutes for any questions. Cool. All right, George. I want to sit here quietly for a minute. We're at stand 3 if you want to go into any of those any deeper or talk to my colleagues who are frankly a lot better at this sort of thing than I am about what Unipro do and how they do it with Drupal. You're either very, very late or very early. Say again, sir. Get a bullet. Yep. Yep.


Hit your heart, Institute of Photography.


I want to talk about the H3 controversy, scandal, cancellation, if you will. I have stated before that I am one of the biggest H3 simps on the internet. H3 podcast, Ethan and Ela Klein, have consistently been people who I really enjoy listening to over the years, and I feel like they've only gotten better. And so there was this episode where he said something with a caller, and we'll get into the specifics here, and his fans thought it was homophobic. He apologized. You'd think that from there it'd be over. But apparently the scandal is still kind of an ongoing phenomenon. So we're going to kind of work backwards because I want to look at what Hassan says here because Hassan is coming out to defend Ethan Klein here. And what Hassan says here I think is basically he hit the nail on the head. So let's listen first. This isn't too far. It's too much. Sorry, Gary stands.  So of course our wonderful fans have taken upon themselves to rid of our sponsors and to have them sponsor or not to support us. So we are, I'm very, you know. So he says that his fans wrote in to these sponsors. I kind of have a conspiracy theory about this. So remember when him and Keemstar like at the height of their beef, Keemstar had a lot of sponsorships that basically bailed out and he blamed H3 for it. My theory is that Keemstar like he's kind of behind this and maybe he instructed his fans perhaps covertly to call Ethan's sponsors and get them to cancel. I'm not sure either way, but assuming that it really is a case of Ethan's own fans calling the sponsors to get them to cancel, based on what I've seen, I think that is extremely fucked up. You know, again, like I do this show with no sponsors, but I think that like trying to go out of your way to punish Ethan for making a mistake, fessing up to it in and then apologizing. I feel like that just, that's so counterproductive that if this is really just his own audience, it's not okay, right? But I'll let him finish what he's saying here. I'll just say this. It's a good, I'm very thankful to our members. It makes this show kind of bulletproof to stuff like this, even though it's painful and emotionally, it just doesn't, it's just painful that, you know, people would do that. There it is. And you know, the other thing is like, you like, you expect a little more of the sponsors in a way, but I get it there is, you know, it's just transactional for them.  Hassan says, right-wingers will always keep winning. We treat people trying to learn and do better worse than we do right-wingers. We have no power over the system slanted in their favor.  It's not about progress or justice. Now this is assuming that this is Ethan's own audience turning on him and it's not a bunch of right-wingers who are trying to get his sponsors to withdraw to kind of prove a point because that's kind of what this person is doing here. They're kind of loving this. It looks like Ethan didn't listen to Jordan Peterson's warning about cancer culture, you reap what you sow. And when Ethan took down or I think maybe he unlisted his interviews with Jordan Peterson, Jordan Peterson claimed he got canceled and was like, well, Ethan, you reap what you sow, I should do the Jordan Peterson voice. You reap what you sow, Ethan, cancer culture will come for you. Like I'm paraphrasing.  First and foremost, I don't think that what Ethan did in unlisting videos with Jordan Peterson is tantamount to cancel culture. What cancel culture really is an actuality is you getting fired from your job for doing something, for making a social media post. I mean, anything can be cancel culture, right? At some point, Ethan was going to say something that offended some signs, some fans. So, you know, Jordan Peterson saying that it's not like he was proven right. Like it's impossible to not prove that because if you say, oh, well, you're doing cancel culture and you're going to get canceled.   So you could say the same thing about me. Somebody can apply that to me as well. Oh, well, Mike, you engage in cancel culture. I've been accused of this just wait until you get canceled and then people have done the same thing to me. So it's non-falsifiable and it's not a real argument, right? So whether or not this is warranted, I don't think that this is warranted. I think this is wrong. But let's go back to where the controversy started. And I'm going to use this Philip DeFranco video because I didn't get to see that actual episode of the podcast. I usually watch them all, but I've been playing Horizon Forbidden West. So, you know, I usually podcast while I game didn't have a chance to watch it. So he, I hope, does an adequate job of showing the controversy right here. Oh, keeping with the theme of something you did online, potentially having a negative effect on your real life. We should talk about the latest creator controversy and apology and that because you had the massive YouTuber and podcaster, Ethan Klein, coming under fire for something he did with James Charles. James recently uploaded a video about makeup on Amazon and in it, you can see adult diapers in the buy again section of James's Amazon homepage. But even then speculating, right? Is it a medical condition? Maybe for an older relative, but then quickly making a joke about James's sexuality that was like small size, like twink size.  Now we were saying, well, if he's getting blasted in that hard enough, potentially you may need to wear a diaper. Maybe he's just got railed real hard the night before or maybe it's for one. Maybe it's for his boy, his boyfriend. With a meeting later, running an audience poll about why James bought these diapers and he made power bottom one of the options. And even immediately in the live chat, people started correcting him on the terminology of power bottom, with a listener named Matt even calling in to correct him and kind of call him out, not just on the way that Ethan used the terms top and bottom, but also the stereotypes. So it doesn't just have to do with how he used them while referring to James Charles, but Ethan's use of these terms in general. I know often you feel inclined to describe someone where you think they might be a bottom. And I think that that's often a misconception that like bottoms may be passive or the. Well, I just mean that he wants to partner.  Correct. And but how can you know that about someone? Matt's saying that Ethan throws these words out in stereotypical ways and adding that you can't just tell who is a top or a bottom to which Ethan says.  Is that the problem to make? Absolutely. Or arguing that again, I agree with all these. I agree with this caller here. And I think that this probably wasn't necessarily the attempt, the intent of Ethan.   And less cringe you are, something like that.   So he's a very crass person. So to him, just like since he's already an ally and LGBTQ plus affirming, he probably didn't necessarily think. And so like if he said this and then you just leave it at that, you know, then I would take issue with that. But he said something that was not good. It was, you know, very insensitive and, you know, not OK, right? But he apologized and that's what's important, right? Like nobody's perfect. And if you're straight, then you don't necessarily know what will and won't be acceptable for gay people, right?   And that's what I think is really important. That's what matters the most.  But what matters is that I know that they are allies. I know that deep down they would fight for me. And that's really what matters to me.  But I think that he did the right thing by apologizing and, you know, explaining how, yeah, I didn't know about this. I didn't know that that's the way that it was perceived. And I don't expect him to know, right? Because he's straight. But just the fact that he's trying to learn, I think in and of itself is really important, right? A lot of people, when they get corrected on something, a lot of times they'll become defensive. And perhaps like this went downhill further because Ethan got a little bit defensive, Dan did too. But at the end of the day, they slept on it, they reflected and they acknowledged that this wasn't this wasn't necessarily my best take. And perhaps I could do better as an LGBTQ plus ally, just saying I'm an ally maybe isn't enough. Perhaps I should understand these things a little bit better. So, you know, I think that he handled this good. It's just one of those things where it's like, OK, I'm still learning. We move on. I think that the overly punishing nature of the audience, if this is the audience, is counterproductive because, you know, it one makes people think, oh, well, I don't want to say anything about gay people.  Like I don't want that to be the takeaway. And I'm afraid that this will be the takeaway.  And they'll come after you. Look at Ethan Klein. He's an ally. They came after him. So I mean, like the right, regardless of who did this, they're going to make an example out of this. And I think that that's bad.  That should be the end of the story. But, you know, when it comes to social media and when you have a lot of bad actors who try to make examples out of good people, then, you know, it just end up it ends up devolving further. But I'll finish this off here. So that's when it comes to femininity and masculinity. Saying this term implies some sort of level of passiveness and using it in these ways is not productive. Ethan then starts to ask if Matt is a top or a bottom, but he declines to answer and Ethan eventually says, can I guess what is your preference? You can. Will you tell me if it's right, if I guess right? No, even then joking that he can already tell which one Matt is, which Matt says he just hopes that Ethan has learned and understands. It doesn't appear that that ship has docked for Ethan. To be honest, I don't really. I'm trying, but I don't, I don't, I mean, I get it. And then closing the show by he and parts of his team dismissing the criticisms. Of course you were right. And honestly, just so over these people.   I mean, I was just trying to make the call fun. Yes. It's a entertaining show.  Someone says, I don't know what's so disappointing. Someone said, I don't grow up. Jesus Christ, Brian Dan. I don't know what's up with Ethan today, but something's off. Bad vibes this episode.  Unsubscribe. Go away. I think this with peace and love, you might be too sensitive to watch this show. Yep. And in the live chat, of course, you had people laughing at Ethan, but others sticking up for Matt, saying Ethan shouldn't have asked about his sex life. In the comments, you had people saying things like, Matt tried to explain that Ethan would not register anything. Arguing that it was cringe worthy, how Ethan never let up and it was just uncomfortable to listen to.  If you go on Twitter, adding to the conversation, saying things like, why is Ethan Klein and capable of making his disgust of James Charles known without making it homophobic? James Charles is an evil person, but Ethan Klein doesn't get to use him as a vehicle to mock us.  Hey, we also saw our fair share of fans and people saying, you know, people these days are just too easily offended, but ultimately all this backlash resulted in Ethan apologizing. Okay, so that's where we will stop and I'll pick it up from there, basically. Thank you so much. Philip DeFranco, he's our H3 correspondent.  Like, would you ask a straight couple, hey, do you guys do anal? Like who goes down on who more? Like you wouldn't necessarily ask these intimate questions to straight couples, but gay people, because it's still somewhat elusive in society and because these things aren't discussed.  Like for example, like my family have asked me, oh well, who's the man of the relationship between me and my husband? And the answer is always, I mean, we're both men, so I don't know how to answer that question. So you just have to educate them. But the thing about Ethan Klein is that he would ask these crass questions to straight people as well. Like he always pushes the boundaries and the limit. Again, he shares way, way too much information oftentimes, there was an ongoing debate throughout one arc of the podcast where he was talking about wiping your ass from front to back or back to front. So he's known for these types of things. So I think that he just probably assumed, well, because I always ask these questions, it's not different. Like I'm gonna treat everyone equal, regardless if you're gay or not. So I'm gonna ask these questions, are you a top or a bottom? That's the way that I saw it, that was my perception anyways, as a longtime fan of the show. But I think that what matters is that regardless of what he said and what was said on the show, this happened. He apologized.    Like they wouldn't say this out loud, right? But I feel like, you know, this is the perception sometimes where like people around, like my family, they've told me and I'm not, I'm like, my humor is very edgy and it takes a lot to offend me but my own family members will say like, oh, well, I don't wanna offend you by saying this.  But I mean, it just depends on your intent, right? Like you can say something that's homophobic but if you don't mean it to be that way then I can understand if you explain your position. And this is what Ethan said here. To my LGBTQ plus fans, I'm sorry for comments on today's show, the sexualization of gay men and the grouping of tops and bottoms is a stereotype that I will be trying to unlearn apologies to the caller too who I shouldn't have pressed inappropriately. I hope you guys know I always mean well. And this is it, we do. Like I've watched the show long enough to where I understand that he means well. Having the context, knowing that he, he doesn't just do like edgy humor.  Like I think that a couple of weeks ago when he was sick, I think they were worried that he had seeded it for something like he literally posted a picture of his shit on the subreddit. So like he's not like against posting these really intimate details. So I get where he's coming from. It was a bad take, it was a bad look, but he apologized. For me, I think that is enough. Now he talked about this on the following episode of the show. And I wanted to share a little bit of that where he shares what happened and then he apologizes on camera as well but explains this position and kind of defends himself because I think that people did take it too far in calling him homophobic when he's not homophobic, right?  I think all of this is important to take into account.  That I guess made people angry and then I'll call or called in to explain why that whole thing is offensive and I made insensitive remarks and jokes to him. They explained to me that I should probably apologize and explained why and I thought that I understood. So this is my heartfelt apology, which I'll read to you now after consulting with people I trust who I value their judgment. To my LGBTQ plus fans, I'm very sorry for comments on today's show, the sexualization of gay men and grouping of tops and bottoms as a stereotype that I'll be trying to unlearn. And apologies to the caller too who I shouldn't have pressed inappropriately. I hope you guys know I always mean well. So that's my apology. Some people accepted, some people don't. That's fine. Keemstar of course has my number. Keemstar a great ally to the LGBTQ plus. Absolutely. You know, this man has always been on the right side of things. So Keemstar is addressing his one LGBTQ fan. He says to my LGBTQ plus fans, he's not sorry, just sorry he got caught. See this is why it leads me to believe that Keemstar was behind the sponsorships because Keemstar is notoriously petty and he was butthurt that the drama between him and Ethan when Ethan exposed him back in like 2019, 2020 led to him losing sponsors. So I feel like Keem has gotta somehow be behind this or maybe he like directed his fans or maybe he has the same sponsors, has contacts. Like it's a conspiracy theory. It's all speculation, right? But this is why I have the feeling that it wasn't Ethan's own fans because like I saw, like I read the responses to Ethan apologizing and a lot of them are like, look, I'm gay, it was a bad tag, but we appreciate you Papa bless, right?    I'd have like no friends or family, right? It's part of the process, right?   So, you know, we have to live with imperfect allies but we appreciate the intent, the hearts of people. So that's what, that's my, that's my take. I think Keem's behind it, but who knows, right? Still either way, I feel like Ethan, you know, I appreciate him apologizing and it means a lot to me. I'm doing my best here, guys. Like y'all can't fuck, I'm getting like woke whiplash and it's like last week or with the Will Smith thing or there was people like, well, sometimes the people take it too far, but this time it's right.   Or it's always like, oh, the veil is down. I always knew they were racist homophobes.  And whoever's saying this, like, they've got to be bad faith, right? If anyone's saying that, I think there's a lot of right wingers who liked Ethan because he kind of did edgy like anti-SJW content to an extent, like he wasn't an anti-SJW YouTuber by any means, but, you know, he did some videos criticizing overly woke people. So the right kind of felt like they lost one. Ethan was the one that got away. So I feel like a lot of this is bad faith and I'm just trying to like, confirm that Ethan was wrong to be a woke, to go woke, right? And I'm gonna show him, go broke. So we'll go after your sponsors. A lot of this is bad faith, in my opinion.  And understandably, and we all are.   Dan knows I'm not homophobic, okay? He's standing up for me, which he's being a good friend and a good supporter.  But, too heated, understandably, too heated, okay? But understandably.  But we're all human beings, we make mistakes. So just saying, like, I got too heated.  Maybe I was wrong and, you know, I listened to people who told me otherwise. I was a little bit too heated, I reflected. Like, you never see this. So that's why I appreciate it and that's why I feel like I'm good. I'm not mad at Ethan for this. He apologized, he did what he needed to. No, it's not. Sometimes it just doesn't feel reasonable with the woke whiplash we get. And I understand. Look, and there's always gonna be outliers as well.  I reject it. I'm never gonna accept you again. I'm done with you. And that's just gonna be something that happens if you're big enough. There's always going to be people on the margins who aren't going to accept that. But what matters is the majority of people understand that you had a bad take and apologized for it and human beings are imperfect, so it's okay. It's okay. We know you're an ally. Like, I appreciate how Ethan Klein has defended LGBTQ plus people. I love that. I think it's great. So he's imperfect. I'm imperfect. We're all imperfect. What matters is that we're constantly trying to do better and improve. But nobody's gonna get it right 100% of the time.  And that's okay. It's part of the human experience. There's an infinite amount of things on this planet to learn that you won't be able to learn with your short time on this rock. What matters though is that we always try to build each other up, not break each other down.  He wants to rip that motherfucker's heart out.  I'm sorry, but me and Dan both know we're not homophobic, okay? And I just refused. I refuse that title. I do outright. He should.   You know, again, I worry that people are going to see this and think, ha ha, I told you that these woke sculls are gonna cancel everyone. Like I don't want them to make an example out of this situation, but bad faith actors will. I am glad that you acknowledge though that, you know, especially the thing with the caller and in pushing him when he had established a boundary. I didn't like that that happened either. And I'm glad that you see that that was too far and that you accept that.   That did not happen. That did not happen.  I don't know what you're talking about. There was a moment. People said you were mocking. Yeah, there was a... He made like a voice, like kind of mimicking, I don't know if it was the caller or someone in the comment section. What I interpreted that as is like, he was making fun of the person or the people in the comment section and he's made this voice before. So it wasn't like an anti-yea voice. It was more like a valley girl type voice. This one he gets a total pass for. He was not trying to be homophobic.  And I think I did a voice that was more so just like a annoying whiner voice. Yeah, yeah, that's the way I interpreted it. That's the way I always interpret his voice. He's done the voice before. So 100% Dan gets a pass. Dan is a good person. Ethan is a good person. These are good people here. I was mocking and somehow people connected that to me mocking the caller's list, which the caller happened like two hours prior to that. So it's really a leap to think that I was referencing that.  And again, that just, that does not feel like people are being genuine. I don't know how you can watch that and make that conclusion. If it came across that way to people and you genuinely thought that's what I was doing, I apologize that that was not my intent by any means.  Right. And that's the thing, right? There's always going to be bad faith actors that really take these situations and they make them exponentially worse. And it's hard to not get in your own head about this, from Ethan's point of view, he probably thinks like, come on, like I press everyone. I ask inappropriate questions about people's body parts regardless of who they are.  Now they're calling me a homophobe. I mean, I can't win. Like I understand that feeling, right? Like we've, if you're on the internet, at some point you've gotten that for sure. We're going to make mistakes. Our allies are going to make mistakes.  We're all human beings. Just keep plugging away and making the world a better place. Right the wrongs when you do them and then enjoy life. Peace and love, right?  Ethan Klein is a good person. Dan is a good person. I love H3. Not canceled in my book.  Yeah, so I'll leave that there. Mike is a total shitless. Shitless. Once he started chilling for the DNC, I stopped watching. So I definitely won't be hitting the subscriber or turning on notifications by clicking the bell. No way. It's very sad, I know.


I'm the jump master. Follow me. I'm going over there. Enemy spotted. Be a good place to land. This is a good LZ. I say we land here. Target over there. Get ready. Moving up. Let's go this way. I'm spotted. Ten-ten. Some of us aren't inside the ring. Okay. Moving up.


That was me introducing myself in Te Reo Māori, the language of tangata fenua, the indigenous people of Aotearoa, New Zealand. I'm Lucy Schrader, Digital Channels Outreach Manager at Te Papa, our National Museum. Welcome to the whole Glam Package. Mahi ana, hei kairauhi ma tai tīpū, taku turanga mahi, ko Haidimite aho, tēnā tātou katoa, and that is me introducing myself in Te Reo Māori. My name is Haidimite. I'm an American-born New Zealander, whose ancestry comes from Germany and Slovakia, and I'm a botanist. I'm a botany curator at Te Papa. So, what is Te Papa? Te Papa is short for the Museum of New Zealand, Te Papa Tongarewa. And it is located in the capital city of New Zealand, Wellington. It is our national museum. It started out in 1865 as the Colonial Museum. Moved on to the Dominion Museum. Then we became the National Museum, and now we are at Te Papa. Te Papa holds about two and a half million objects in our museum, and many of these can be found on our online collections online, and many of those are openly licensed. And we do have objects from art, history, natural history, from New Zealand and Aotearoa. So, we used some of that material last year when a tiny little team of us ran a project to contribute some of our stuff about New Zealand's native forget-me-nots, these lovely plants here, to wiki. I should mention we're going to say the word Project a lot. That's lowercasep, not capitalp, wikimedia projects, just that everyone should know. When we did this project, we got a few things done, like writing C-Class articles in English language Wikipedia for each of these species, the information about their name, their classification, geographic distribution, conservation status, all of that linked to public research, published research and online databases. We edited and created wiki data items for species and for the people who described them, plus the collectors of specimens and the publications that we were referencing. We carefully selected and then loaded certain collection images to wikimedia commons, and this was to fully illustrate all kinds of different aspects about each of the individual species of forget-me-nots. Then we created a new wikimedia commons template that more effectively displays all of the detailed information that we hold about each of these specimens in our database. Then we linked all these images and records in wiki data by basically turning all that same information into structured data statements. We also tested new functionality in OpenRefine, so we not only loaded this data, but hundreds of images all at once. We improved our own collection records by finding data that needed fixing up, and we added wiki data identifiers for later use. We did all this while collaborating with both local and international wikimedians, as well as getting our own museum colleagues a little bit excited about joining in to this kind of projects. But best of all, we basically worked out a way to do this kind of project quickly and sustainably, which means we can do it all over again for another topic. So why is Te Papa basically doing this work? Well, Te Papa is a Glam institution, so Glam is galleries, libraries, archives and museums. And we, in today's Glam world, basically, we feel that reaching audiences digitally is just as important as reaching them in person. So doing in-person exhibitions, public programmes and collections-based research is important, but we see our digital wiki work really as being museum work. And so this is in a new area that we call digital outreach. Now, Te Papa holds amazing collections and deep knowledge about all the material that we hold, but for most of our history, people have had to come to visit us first physically and then online, but people all around the world could be seeing, connecting with, using our collections. So why should they need to come to us or even really know who we are to have those sorts of experiences? Instead, we're now finding ways to send our collections, data and knowledge out into the various digital spaces that our audience already are. Now, that means we're a media, of course, but we also use a whole bunch of different platforms in different ways depending on who that audience is. So we send images of collection items out to Google Art and Culture, Unsplash, Art Store, other places that people are looking for cool pictures. We're also working on publishing our specimen data and images on G-Biff, which is like the place for researchers to get large biodiversity data sets. And we're creating continuing education courses on future learn that draw on our collections and our publications. And we can basically do all this digital outreach work because we've literally had thousands of people working for 150 years at our museum we've collected, we've catalogued, we've analysed, we've researched, we've staged exhibitions, held events, we've reached out to our communities, we've imaged collection objects, we've structured our data, we've openly licensed our images. And of course, all of this is still happening today at the museum. And so with all of that in hand, we can basically send out not just images or information, but connections between these objects and their deeper context. And so with our project that meant we wanted to create a rich package of contributions that included the three main Wikimedia projects, namely Wikipedia, Wikimedia Commons and Wikidata. So on our Forget Me Not project, the package was made up of 65 English Wikipedia articles, most of those brand new about New Zealand's native Forget Me Nots. 350 images of specimens on Wikimedia Commons that included examples of every species we were talking about, a mix of images from the field, herbarium sheets, pollen, and hundreds of Wikidata edits, adding data to support both the images and the articles, and letting us link our own records back out to Wikidata. And we did this with a core team of three people. It was us two and our awesome database specialist, Gareth Watkins. And together we created a project structure that used each of our strengths and encouraged collaboration and learning. No, just before we get more into the project, there is some stuff we didn't do. First, we actually didn't add any Mataranga Māori or traditional Indigenous knowledge from the Māori people of New Zealand. And this is because Te Papa doesn't actually hold any Indigenous knowledge about these particular species of Forget Me Nots. In addition, none of our three team members were Māori. And finally, our museum as a whole is really still in the process of making in, building these relationships with iwi or tribes that would inform all these kinds of projects for in a mutually beneficial way. And I think more generally, we do need to acknowledge that there's still a lot of work to do in this space in the wiki ecosystem to make sure that it works for Indigenous peoples everywhere in the world. Now secondly, we did not upload every single image or object that we could have. We focused on publishing what we reckoned was going to be useful in a wiki context. So we went deep rather than going really broad and we used that as a way to stay focused and avoid trying to do too much. As we said earlier, we are actually going to be publishing all of these specimens and images on GBIF instead and that will still be openly licensed. And when we were done with this project, we found that Te Papa had actually gained quite a lot more than we had expected from just this one little bit of work. We were of course able to achieve our stated goal and make some of our collections, data and knowledge more generally accessible. But also the three of us really built our skills. We've learnt how to use tools like open and fine, advanced wiki functionality, stuff like templates, gadgets, plugins. And we got our heads around wiki data realising that the Glamoans really got a chance to get somewhere with linked open data finally. And from this project, our own museum data is actually better because we had the chance to make all these updates to our records and fix some errors. And we also now have wiki data identifiers in our own museum database and that's going to be useful moving forward in all kinds of ways, specifically for species and for people. We've now got a big bucket of new ideas to work on. We want to match more records, more of our records with wiki data. We want to hold some community engagement events. We're hoping to turn a few more of our staff into wiki editors and we're figuring out other ways to make our data more generally shareable. And of course, we now have this process that we can actually do another project on and that it's not just going to be limited to natural science, natural history projects, but also humanities projects. And if you're at a Glam, you can do this as well because we have written a manual about it. So our manual takes you through this whole process from getting your team set up all the way to publishing and promotion of your project. And it's got practical tips, it's got prompts, and it's got examples in plain English. Sorry, it's only in English at the moment, but if anyone wants to take up the mantle of translation, we'd be happy to have it in more languages. So it's specifically written for Glam's like ours, but it will also be useful, we think, for volunteers who want to nudge their local organisation in the right sort of direction. I also want to be very clear. This is a first version and it comes from our own point of view. Right now the manual sits under Tepapa's wiki project page, but we would really love to see this get genuinely wiki-fied and bring in lessons and examples from other organisations. And of course improvements from editors who have more experience to fill in a lot of the gaps that are still there. So we reckon that this is a process that generally works. Can't promise it'll be easy going necessarily, but we think that you could get a full project along these lines done in about three months, even when the team members have other work on. So we're going to share the link again at the end so you can check all that out, plus it's on bookmarks that are going around. But first we want to talk about a few of the essential things that made our particular project work. So that's collaboration, staying focused, using the richness of a Glam collection and keeping an eye on the bigger picture. So the first one's collaboration. This style of project actually does not require a big team. As Lucy mentioned, we were three people who did this. What's important is that you're excited about wiki, you're working together, and you stick to the decisions on the plan that you make as a team. So you want to respect and make good use of each member's expertise and skills, but you also won't get far if you're not able to stay in touch and be flexible enough to help each other out once in a while. So on our project, there was absolutely no way any one of us could have done this on our own. It needed the combination of knowledge and skills about the species, the technical know-how, and enthusiasm so that we could actually make progress. But on top of that, I really feel like working together, we were able to solve problems together and we were able to motivate each other, which really kept the project on track. Also, in terms of collaboration, don't just keep it a secret within your team if you're doing a project like this. Tell your colleagues at your Glam because this can get you unexpected support from the museum and it can also see the way for future projects to happen. And then finally, collaboration can even go outside of your Glam walls. So there are editors and specialists out there in the local community, as many of you know, that will be absolutely thrilled that your Glam is wanting to share stuff and they're going to want to help you. And so if you can hook into your local community and find some volunteers who are willing to help you, make sure you treasure them as well. Focus? I think it's really important to be deliberate on a project like this and stay focused. The thing that I think can really help is if you keep in mind what matters to your organisation and your audience. Think strategically in this way because then all kinds of decisions actually become a lot easier and you do improve your chances of getting help, as we were just talking about. In our case, we got internal support to a large degree because we showed how our project fit in with Te Papa's current strategy. We've committed to supporting Papatūānuku, our relationships with and responsibilities for the natural world. And making information about our native Forgive Me Nuts more available, accessible, usable is definitely a little step in that direction. Focus is also about making smart decisions about scope and sticking to them. We think it's better to complete a small project rather than run out of steam on a big one. So, resist the urge to commit to doing it all at once. You can always do another project later. Richness is another point. So, as we said earlier, GLAMs are literally rich with all this interconnected knowledge. So, you can use that. It can let you share more complete or more complex metadata. You might be able to provide a more unified story about your objects that links different topics or disciplines. You're also surrounded at a museum by a work that's already been done. So, you have exhibitions, publications, research, the list of selected objects, the labels that were on exhibition walls, recordings of events. These can all be used in this kind of project and reshared. So, for us, the knowledge that we had was basically my in-depth knowledge from the biodiversity research that I do on Native New Zealand Forgive Me Nuts. So, we had specimens, we had images, and I've been doing fieldwork and collections-based research for over a decade on these plants about classifying and identifying them. And so, I could contribute these images and also data and also the publications that can then be cited on wiki-PD pages and such forth. And then, of course, there's the bigger picture. Don't forget to look up every once in a while to see the relevance of your project in that big of view. You can use your project to actually help your wider organisation and future wiki projects. You might feel like the work you're doing is just a small, contained little thing, but it turns out it can have a pretty big effect on your glam. So, on the one hand, you can make the next project you do easier by keeping track of how it's going, talking through problems that come up. Really can't stress enough. If you come up with a good solution, you write it down. And that's even more helpful if it's not actually you running the next project. It could be someone else nearby that you've inspired. Agate to Papa, our work's inspired another curator who's already started his project about New Zealand's endemic and native spiders and harvesters. Then at the organisational level, your project's likely to find places where more work's needed. Stuff like digitising certain collections, applying open licences, improving data standards, finding ways to directly collaborate with your audiences. Now, if you'd linked your project to a glam strategy in audiences I mentioned earlier, it becomes far more convincing when you add you for re-think of priorities and resourcing. OK, so here's our main take-home message. To sum up, you can share your glam's collections and your knowledge together by using the links between Wikipedia, Wikimedia Commons and Wikidata, and you can do it in a quick, sustainable and repeatable way. So, head on over to Papa.inz.com slash whole glam package and give it a go yourselves. As mentioned, there's bookmarks with the links, so you can grab one of those. And otherwise, we would absolutely love to talk to you about your possible project, our project, work that you've already done in a similar area, improving the manual, absolutely. So, again, you can find us here for the rest of Wikimedia. You can get in touch with us through English Wikipedia and Wikidata and the various other platforms we're on. And of course, you can just email us at any time. And finally, we would like to extend a massive, massive thank you to Wikimedia Aotearoa New Zealand or WANs, who basically supported us to come here to Wikimedia 2023 to give this talk. And of course, thank all of you for coming along. Really great to see you so many faces. Na mihi nui. And we've got a minute and a half for any little questions. Totally okay if there's not any. Okay, I think you need to use a microphone. It's Diego from Spain. Big contribution in comments. I wonder the order you followed because you said the first thing you did is choosing images you wanted to upload. I was expecting rather that you say the first thing we have to do is look for gaps in Wikimedia articles. So that's, can you elaborate on that? So we have a bit of stuff on how to build up your topic and how to identify those gaps in the manual itself. There's quite a few factors that we think can go into that and it's partly about where the gap lies, not necessarily on one particular platform, but in that larger picture. It's also where your own expertise lies, what material you think you've got available, what's important to your organisation and audience. We specifically focused on New Zealand's native forget-me-nots because there weren't many other organisations out there in the world who could do that. We could do that. But even then, that's only one part of the bigger picture. So we'll be happy to talk more about that later, but we seem to have hit the red, so. Thank you, thank you so much.