Hello, and welcome to the Debian Contributor's Talk where I will present the status of the Debian Contributor's idea towards the end of the conference after we worked on it or talked about it for most of the conference. I spent about half an hour trying to lay out bullet points in the open office and then I thought it was too hard and there's already something that does bullet points very nicely indeed. So I switched to a much more advanced bleeding edge presentation tool which I think is fantastic and we're trying it out. So Debian Contributor's, a revolution in Debian project headwear fashion, were making new hats for the Debian project, virtual hats, I mean titles. I presented the idea at the beginning of the conference and so to start the discussion and then collected a lot of input on it, did a little bit of prototype code as well and now I'm going to show you. So in Debian we have many kinds of contributors. Debian developers, Debian maintainers, Aliot, Committers, bug reporters, Wigging Contributor, translators, mailing list contributors, sponsored uploaders, package reviewers, language reviewers, conference organizers, events people, press people, system administrators, service developers, video team volunteers, conference organizers, food people, barbecue people, poetry people, dance teacher, massage people, cheese and wine organizers and I'm sure I forget lots. So it's too complicated, therefore let's just do Debian contributors. Easy, that's it. Innovation. Everyone that helps Debian is a Debian contributor.  I wonder why it wasn't that before, but anyway nobody noticed so now we do it and we say we invented something. We don't say that we missed something, we say that we invented something. So what is a Debian contributor? There is no process to become a Debian contributor, you just contribute to Debian and you are a Debian contributor. And when you are a Debian contributor you get such diverse elements of power as having your name on a list, which is very important indeed. You can also choose not to have your name on a list. You can see a summary of all your contributions, get acknowledged for your work, get thanked for your work, get reputation for your work. It is unacceptable that we still fail to do this for so many parts of Debian. And then we all think that Debian is only uploading packages because the only thing we can do, we can see that happens in Debian is package uploads. Hey, there is so much more, right? So the idea is that there is a central place that shows the list of Debian contributors and this place gathers data from all of Debian. But the central place does not do data mining, it only accepts submissions. Debian is far too diverse. It is impossible to have a central place in Debian that knows everything that goes on and can understand commit logs from translators or perhaps email subjects in translator mailing lists and the dark database and the bug tracking system, hacked mailbox files, and I don't even want to go into more details. So each team knows what goes on and they are the people that best know about communicating that. So the idea is that each team can submit a track of contributions to this Debian contributors list and of course it's going to be as easy as possible, otherwise teams will not do it. However, many teams did not track contributions, not because they didn't want to, I think, but because they had other stuff to do. They didn't have a strong enough motivation to invent something because, well, if you are the games team, it's very good that the thing you mostly focus is in having good games in Debian and the point of a team should not be to list the members of a team. That should not be the priority of the team. However, now if there is a little motivation to tell who does work in the team, then perhaps people in the team can feel more a part of Debian, can actually feel that they are Debian contributors, which they are. So this gives us a chance to make visible a larger range of contributions that at the moment we cannot see and I very much look forward to that. So people contribute and then maybe after some time people don't contribute anymore and in Debian it's a problem to track when people stop uploading packages, for example. So in here, instead, it's very simple. When a contributor is listed in the list, then we also know the last month that they contributed something and if people haven't contributed after year 2012, we can move them in a list of people who contributed until 2012. It's so easy, right? So the list manages itself when people become inactive and inactive people still get credited, which I think is important. I forgot to add one thing. If you are a Debian developer, you are not automatically a Debian contributor.  And it's good that Debian developers are the same as everyone else. If you want to be a Debian contributor, you contribute something to Debian. It is okay to be an inactive Debian developer. It has been observed that some Debian developers had kids recently and perhaps for a year you have to focus on kids or two years or 18 years so it's fine to be a Debian developer and not a Debian contributor and then at least you get credited for what you do. It's a different thing than being trusted by the project to vote, for example. Now, design choices for this system are not about perfection. We make the idea is to... So when thinking about design choices for Debian contributors, I think it is useful to make a parallel with thanking people. So I want to make a good effort of thanking people, but I do not need... I do not want to go out of my way to thank people. If I were to thank a friend for driving me home because one evening, I don't know, my car wasn't there, whatever, then I'll just say thank you.  I don't need to say thank you for driving me home tonight and going out with me yesterday and a week ago I recall we exchanged a few words on the phone. A year ago I needed some change for a drink and you had it ready and list all the contributions. I would have no friends if I did that every time. So it's good to make an effort to thank people, but that's it. We shouldn't... Listing all the contributions of people goes beyond the scope of this project. And if a person isn't credited but can get credited by contributing a little bit more, that is okay. We don't need to go and look for the single contribution that if we miss it, that person can't be counted anymore because it would be very hard to do. We would need to try to think about listing contributors if they wrote anything to the backtracking system. There is spam in the backtracking system. So things need to be filtered somehow. So it's perfectly okay to say you need to have 5, 6, 10 males in the backtracking system over a span of a month. If that keeps spam away, then it's fair that it's okay that we cannot thank people that report only one bug. It's a shame, but on the other hand, it means that such a system can actually be built and we don't focus on spending one year trying to get real contribution and not spammers outside of the backtracking system. However, if a person has done a lot of contributions already, then we should find a way to credit them. So if we find out that there is a team that hasn't started to contributing to this once the system is deployed, we should help that team to thank their members. And I repeat, we do not need to be perfect. I repeat, keep that in mind in the Q&A, in the question and answer session after the talk. We do not need to be perfect. So, you know, sometimes it's worth saying at that point that the intention is not to be perfect. Otherwise, you get asked very corner case questions by very precise people. So yeah, we are somewhat lossy. We are saying that person has been contributing to Debian from 2005 to 2013. That's it. Actually, you know, that person has uploaded packages in that time range. That person has made translations between 2008 and 2012. That person has been active in the backtracking system since 1998. That's the sort of thing that I would like to see on that list. So course time granularity, a month will do. There is no point to say thank you for buying me a drink yesterday at 8.37 and 52 seconds at those coordinates. You don't thank people like that, right? Just thank you. You bought me a drink recently. That's fine. Only begin and end months are stored for each contribution. It's okay not to track gaps. It's complicated to implement something that says you've been active from January 1998 to March 2001 and then you've done nothing from March 2001 to April 2006 and then from April 2006 and June 2006 you've uploaded one package and then from... What's the point? Well, thanks for uploading packages between 1998 and 2013. Yes, everyone's been inactive for some gaps. I don't upload packages all the time. Sometimes I sleep, right? Just storing begin and end, it's enough. You have contributed to Debian in that time range. The system does not want to say how much people contributed to Debian. Only that they did. And there could always be a link from here to a team page which has details about contributions. This system should link to those information but should not try to import them. It is okay if new contributions take a day or two to show up on the list. I would like that data sources can just put their data mining thing in a cron job that runs once every night. There's no point of doing anything more complicated than that. And if one night the cron job doesn't run, it's okay nobody needs to rush and run it manually. We can just wait for the following night. The time granularity is a month. It's okay if it takes a day or two to show up on the list. And I don't think people should contribute to Debian and then start refreshing the list to see if they show up. The point of contributing to Debian is because somebody has a reason to contribute to Debian, not because they want to end up on a list. Otherwise they can just contribute to a wiki and add their name on a list. Actually, we could make a separate tool which is a site where you add your names on lists so if people really want their best goal in life to have their name on a list, they can go there and list themselves. But that's not the point of this. This is just to look at people's contributions.  I think that's been mentioned before. And in terms of requirements for teams, these need to be very simple. If a team spends more than a couple of hours hacking a script to send me stuff, they'll give up. So submitting data must be as simple as it can be. Some teams can track history.  So that can be queried and they will know that I've started uploading in year 2000, whatever. Other teams, they will only know who's a member of the team today. And they cannot tell me that person has been a member since that date. And that's perfectly okay. So both kind of teams should be supported. And of course, there's the problem of privacy. People, if I report a bug, perhaps I do not expect that my name shows up on a list of Debian contributors. Perhaps I was just typing something at the terminal. I didn't even have any idea that a mail would be sent out. In fact, I don't know if people, when they first report a bug, realize that their email address will be disclosed in a publicly archived Google indexed website. So people should be able to know that they are going to be thanked and choose not to be thanked. If, for example, they contribute to Debian during their boring paid time as nuclear facility monitoring people. And so ideally, you start contributing to Debian and after some time you get an email saying, oh, I noticed that you started contributing to Debian. Thank you very much for it. We would like to list you in that site. If it's okay with you, click on this link. That would be the general idea. We should not publish email addresses by default, otherwise we create a nice tool for spammers or for Google recruiters. Which... And we should allow people to manage what is credited to them. Maybe I would like to hide sometimes of contributions because my boss thinks that I only do cloud stuff and then you can see that I also contribute screenshots to the game team and maybe I don't do it in paid time but my boss is a bit weird and it's just best if they don't see it. My life will be easier and I don't want to make people's life harder because I thank them. You can be very obnoxious if you thank people too much. Actually, I worked for a month in teaching web app development for a course and we were hosted by a center that dealt with psychiatric people, people with mental problems. And so we were together with also the people that were in the center. So one of them said that he liked puzzles very much, the broken picture that you put together. And so one of the students of the course gave him a puzzle and he was so grateful that every single day they met he would thank him. Very deeply and heartfelt thanked, it was very hard to hate him for it but then every time we would go for dinner that would be that minute of thank you. And sometimes we were teaching and the door would open and he would get in and say by the way, thank you so much, I really appreciate your gift. That was too much, thank you. So people should be able to say, okay, I've done this, fine. You don't need to thank me for it, actually. I'm fine just having done this. Yeah, people should be able to manage their online identity and what is visible for their contributions. However, since I am Debian account manager and I need to say whether a person should be a new Debian developer or not, it would be very useful for me to see that somebody has only uploaded one package yesterday, whether they were active in translations for the last 10 years. I should be able to see that because then I can say that person must have some experience of how Debian works, although they've only uploaded a package yesterday. So in the last 20 minutes, I'll show what we have made at Debian so far and take questions. I can take five minutes of questions on the general idea and then move on to show some details. Demo, demo. Okay, oh, this one was good. So this is the data model. What we are collecting. For one data source, and for data source you can think the Wiki team or FTP master that manages package uploads or the Italian translation team or all the translators. For each data source we have a name of the data source, so Wiki. A description, the Debian Wiki. IURL, http colon slash slash www.wiki.debian.org and a list of possible contribution types with descriptions. Now for Wiki this would be editing or admin. For FTP master it could be uploading, sponsoring. For translators could be translating. For other team may have a complex workflow and there could be reviewing and so on. So what kind of contributions? This person has done uploads to FTP master from that date to that date. Now these are all the kind of contributions you can do in that source and it is used so that you get a description to show on the website. And that's it. A description of a data source should just be this. Something to show the name of the data source and show a description of all possible contributions that can be done in it. And then data about one person doing something. Is it any readable? Okay. So this has a list of many kinds of contributions. So we are saying that the person has done uploading from this date to that date and perhaps a list, a URL to the page with tracks uploads in that source. And that's it. Now that describes the kind of contribution and the time it happened. And then we need to identify the person somehow. And so there is one or more but generally one identities. Identity could be the login to Debian systems or to Alliot, email address, GPG key fingerprint, wiki name. Something that identifies a person in that data source and then the name. So the person with this email address has done that from this date to that date. That's it. The idea is that this should be submitted via HTTP post. Because if we have each data source make it publicly available on the internet, we risk disclosing people that people would not want to be disclosed. So a data source will collect this information and then post it via HTTP to the central Debian contributors list together with a secret password so that you don't wake up one morning and start contributing trash to the system. That is the general data model. Understandable? Then I can show you the database model. If you have a person, a person would be me. That does many things with many different identities in Debian. Identity not in the sense that one day I call myself Enrico and another day I call myself John. But identity in the sense that one day I call myself Enrico at Debian.org and one day I call myself GPG fingerprint with ID 797EBFAB. Because we don't have a single naming system in Debian. So it's hard to tell if an email address corresponds to a key fingerprint. But a person can have many identities and choose to not show up on the list. Then we have identity, type of identity, login, email, fingerprint, wiki name possibly more will show up. Name, what person has claimed that identity and again I can choose to hide everything done in the name of my email address or of one of my email addresses. So I can choose to upload game packages with one email, non-work email address and I can choose to upload cloud packages with my work email address and then I can choose which of the two I want to show. Probably both, maybe one of them, it depends. Now there's nothing wrong in working with the game team. It's just an example. Yes, a question? A microphone for sir? Can we have a microphone? Will it be possible to maintain something like two identities like one private identity and one work identity? Ideally, yes. I would exactly like that to happen. So in Debian when one becomes a Debian developer there is one identity because Debian developers take responsibility for everything that happens in Debian and we need to know where they live, sort of. Actually we don't, but we don't tell people that we don't. So generally, yeah. Anyway, but here we're just saying thank you so there's no point, there's no reason to be that strict. So yes, definitely. Now with GPG key ID it gets tricky but then I can hide contributions that only show up by GPG ID and only show the one by email. I can only manage two person records and associate different identities to one or the other. That's the idea in the database model that does not exist yet in interface but I would like that to be supported. The model I have in mind is that you see contributions by email and you say that email is mine, claim this identity and you get sent an email with a link, you click on that link and you attach it to your person or you say that person is me, that other person is also me, make one only. Details to be sketched out, yeah. So is that a bit like the OLO website that tracks open source contributions on different VCSs? I mean do you can actually claim an ID in a VCS? A bit similar to what they are doing but you are the only one that's allowed to claim and OLO also allowed other people to say something about you which I find dreadful. But yeah, that would be the idea. By default each identity reported by data sources is separate. I can make an effort to merge some. I can take the key ring and check all the identities on a key that have been signed by a Debian developer and therefore say all those correspond to the same person and to that key ID and then automatically merge by default. I can do something like that. Perhaps I will have to do something like that because otherwise I cannot see a fingerprint and then send a person an email because the fingerprint is not an email, right? But, well, it's a bit to be seen. I don't want to do too much, yeah? A couple of suggestions I have. One is, so first I'll give you a couple of suggestions that I don't think matter very much and talk about things that do matter. You can decide to throw those away. First, I would suggest changing your identity name to identifier name because identities are kind of about humans. Cool. Secondly, I find it kind of weird that you're not using RSE 822 in your data format. If you could, that'd be cool, if not, whatever. RFC? Like email headers, key value email headers. What? A key fingerprint is not an email. But what I'm saying is you showed us about five minutes ago the data model, the syntax for what gets posted to you. Yeah. And that looks like JSON or something. Yes. If it could be RSE 822, that'd be cool. Oh, the data format, do you mean? It also doesn't matter, so whatever. Possibly more than one data format as well. Some data sources may be very happy serializing to JSON, some to YAML. And it doesn't take much effort to... Okay. And then I've been working with some people to start a Debian welcoming team recently, which we actually have to emerge with a different welcoming team, but that's another story. But in that welcoming team, it seems like we would also like to have the historical records that you have. So maybe we can talk afterwards about if other internal teams with some privacy requirements can use the same internal data that Dan would get. Yeah. I could see that the missing in action team using this, even just the user to identify mapping, it could be useful. Obviously with some privacy requirements. Otherwise, we are making some nasty echelon of doom. Just to make people a bit less uneasy about that, I guess the idea is that if you have different persons with different identifiers attached to them, nobody will match them from the system, that they will keep separate. Yeah. So if you want to have these separate things separate. So this data would be private, and the idea is that it shouldn't be exposed or leaked. Speaking of exposing or leaking the data, in addition to having other teams have access to the database that you have, it would be nice if other teams, like the welcoming team or MIA, might maybe have access to the raw data that you have been given. So, I think that's it. Yeah. No problem with that. The raw data is stored below. So that's the data source. Name, description, URL, authentication token for posting. Shared secret will do. SSL can do the encryption. A contribution type. So for one source, if I say uploading, means this description. And one contribution of this type will be described as this. So this is, oops. This is package uploaders. This is uploaded a package. This shows up on top of the page. This shows up for each person. Then for each contribution type, I need to have settings that is for this identifier and this contribution type, I don't want it to be shown. So do not show wiki contributions from this wiki name. I was silly at the time. And then the contributions, which have a type, which is specific to a source, an identifier that has done that contribution, begin and time, URL for tracking all package uploads of that person in the team site. And of course the URL is optional. It is a very simple database model. Would somebody want to see it in SQL? Was it, it would be good or was it, we are good? Okay. So I've done a random generator for contributions. Which is good for testing. Now in this case, the data source does not only knows when the contribution began. And the end defaults to today. In some other case, is there any with begin being? No, of course not. Oh, there you are. This data source knows that a person has done a contribution today, but doesn't remember when it began. So begin can default to the previous time I had in the database. A data source does not need to track history. It all gets imported nicely. This knows that that person is a sponsor. Doesn't know when it began, doesn't know when it was last a sponsor. But we can assume that it was a sponsor until today.  Finally, I've done an import function for this random stuff. And let's find a web browser. The beginning of randomly generated data, obviously. This needs a lot of thinking. At the moment, I show emails. I should not. But then what do I show? As I start getting data out of data sources, what will be shown? I will have a better idea of what I will show here. Obviously, I cannot show key fingerprints. So I will need to retrieve somebody's key from somewhere and have a look at the primary user ID. But then I cannot look at the primary user ID if it has not been signed by anyone, otherwise people may troll the system.  We have their key ID. So we can say that's the last time you upload to Debian. It depends on how it works there. Those are considerations that are yet to be made and can only be made once we have real data in front of us. Links go nowhere. That's what I managed to do this morning. But I already have the system that can import those kind of submissions, compute the missing begin and end fields, and show people on the list with the Debian website team and compute the last contribution date. The day should be hidden as well. There's no point showing the day. Lunar? Easy thing. If you want people to obtain, being in that list, let them specify, let them give the system the name they want to show in that list. Oh, yes. Totally in my plan. Absolutely. Tricky is how to let them know that they can log in.  But when my gut feeling is that when I actually get to do it, the result will be obvious. There's no point in thinking too much before starting in this case. So with the demo, last round of Q&A, if there is still time? Three more questions, I think. I have a comment about... So it's a list of contributions, but what we really are introducing is a new concept in the project. That's what I care more than technical details about the list. That's why it's so loosely, it's so imperfectly specified. I want to have a concept of Debian contributors. I want to be able... I want that shortly in Debian, shortly in six months, in one year in Debian, we will be able to say, oh, we are setting up the welcome team. It's cool. And do we really need Debian developers only in that team? There can also be any Debian contributor with some reasonable experience. Any Debian contributor can do it. Any Debian contributor can join that team or any person can join that team if that team can take new people. But have a concept of Debian contributors and all Debian contributors can do things, are recognized as a resource to the project. I have a question, but it's really not for Enrico, it's for the audience. So I think this sounds great. Does anyone here not think this sounds great? Okay, cool. So I think we all love this idea. Thank you.


I would like to introduce Zhatan, CEO and co-founder of PlanetScale, who will be talking to us about the test today. Take it away. Awesome, thank you Chris, and thank you all for being here this Monday morning of Veterans Day. What I'm going to talk about today is how to spin up clusters that are jurisdiction aware. More and more jurisdictions are passing data locality laws. And what we are going to look at today is how you can build clusters that are jurisdiction aware, but which you can build without changing the application logic. So your applications continues to think that it's talking to a single monolithic database, whereas depending on some marker of locality, your data can be distributed in the correct jurisdiction. So that's what we are going to talk about. A quick introduction about myself. Most of my career, I have been a backend infrastructure engineer at companies like Google, YouTube, Dropbox and the United States Digital Service. At YouTube, I manage the SRE and DBA teams. These are the groups that operated with test clusters at pre-massive scale at YouTube. Planet Scale is a company that me and Sugus Sugumaran, who is the chief engineer behind and the founder of the Vites open source project. We both founded Planet Scale in February 2018. We are a venture backed company backed by Anderson Horowitz and SignalFire. We are about 30 employees mostly in Mountain View, and of course we are hiring. And with that, I'm going to give it to Chris, who wants to talk a little bit about the upcoming KubeCon. Yes, it is the week before KubeCon and all through the house. Not a creature was stirring, not even a mouse. KubeCon is next week, everybody. There are still tickets available, I believe. There are still day zero events that you can get into right now. So grab your tickets, enjoy KubeCon and at San Diego Glory. I'm in Detroit. It's snowing. We're going to get six inches. So I cannot wait. Next slide, please. Also, CNCF is setting up Kubernetes forums. These are, think of them as like small KubeCon, but very like regionally based. So everybody gets as much knowledge as possible. There are two here that we're coming up here in December, Sydney and Seoul. Sponsorships are available still, and I believe tickets are still available for both events as of right now. Please check them out. Learn more about them, and we'll have more of those here in the near future. Yeah, and just to talk a little bit more about the forum events in Seoul and Sydney, me and another engineer from PlanetScale Deepsea would both be at both these events. So come there and get some more Vitas goodness. Yes. And before we kick off, I would also like to invite all of you at an event that PlanetScale is putting together with AWS and Vitas at on Monday, November 18, 6pm to 10pm. There are more details on our social media and on our web page and so on, so please come and attend. We would love to see you there. And now, without further ado, I would like to jump into the webinar. So here is the agenda for today. What I'll be covering in the beginning is talk a little bit about the Vitas architecture and concepts that we need for building jurisdiction of our clusters. Then I'll present the problem. We will think through and talk through the cluster design. And finally, I'll show you a live demo. So let's talk up, talk a little bit about the Vitas architecture. Those of you who know Vitas probably know this diagram fairly well. Everything on the left hand side of the green line is in our application server, which thinks that everything on the right hand side of the green line is just one humongous MySQL database. Those application servers talk to VT gates using the MySQL binary protocol. VT gates are stateless proxies. They implement the MySQL parser so that they look at your where clause and depending on the where clause, they send the queries to each shard. We are going to go into a fair amount of detail about what a shard is. But each shard is basically a traditional MySQL cluster. With masters and replicas, but each MySQL D in the cluster gets a minor process for VT tablet that does the job of protecting the MySQL D and it protects it by doing a connection pooling, adding row limits by doing adding timeouts for transactions for queries and so on. Basically, even for a single sharded database, it's impossible to write a query of depth if it's running under Vitas. All of the cluster topology is saved in what we call a lock server or topology server. This can either be an it city cluster or a zookeeper cluster or a council cluster. VT CTLD is the control plane that gives you a web UI. We will be interacting with this during the demo and it also gives you an API server where your command lines can talk to. And this is the overall architecture of Vitas. What is not shown here is the dimension of cells. So for any shard, this collection of the master and the replicas that constitute that shard can be distributed across multiple cells. So keep this in mind as we go further in the demo. Anybody has any questions about the architecture? Now we are no questions right now. Alright, let's move on to the next slide. Okay, so let's talk about what a cell is, right? So a cell is a group of servers and network infrastructure. It's a failure domain isolated from failures in other cells. And you can define a cell depending on, you know, depending on how you're running and what makes sense to you. So a full data center could be a cell. It could be a subset of a data center. It, you know, it can be a region or an availability zone. It could be a Kubernetes cluster. It could even be a rack in a data center. So a failure domain, which is basically a group of servers and network infrastructure that is isolated from failures in other cells. That's what a cell is. And if you think about this, this is the representation of physical infrastructure, which means that there is a physical location associated with it, which means that you can locate this in a jurisdiction. So the mapping between the geographical location of your data is sort of represented by a cell in a Vittes cluster. So keep this at the back of your mind. The next concept that I want to talk about is the key space. So a key space in Vittes parlance. So as you know, Vittes deals with sharded databases and a key space is a logical database. If a key space has only one sharp, then it's exactly identical to a my SQL database. But if you're using sharding, a key space is multiple SQL databases, my SQL databases, all with identical schema, right? In either case, a key space appears as a single database from the standpoint of the application. So application can think that it's talking to a single humongous database. It doesn't need to be aware that it's talking to a sharded database. So that's what a key space is. Now let's talk a little bit about what a key space ID is, right? So key space is sharded by key space ID ranges, and each row is assigned a key space ID. So key space ID is basically a street address of a row, right? And it's used to decide on which are a given row lives, right? If you are familiar with no SQL databases, which are basically key value databases, then it's an equivalent of no SQL sharding key. The reason that I have stricken out that row is because we do not save this key space ID anywhere, right? It's Internet of Hit S. The application does not need to know anything about it. And it's not stored but computed. So we'll talk about this, how this is done in the next slide. But key space ID is this each row has it. And depending on the value of the key space ID for that row, you decide which shard that particular row goes to. It's not stored but computed. Those are the things that you need to keep in mind about the key space ID. Now let's talk about another concept called V index or Vindex, right? And it's basically a way to compute the key space ID for any row in a table, right? So a Vindex for a table is defined by two things. One is a column name. And that's the column that you want to use to shard data in that table by. And the second is the name of a sharding function. So there are six or seven sharding functions that are predefined in BitS. And you can add your own. So we'll talk a little more about sharding functions in the next slide. But this equality holds where the key space ID for a row equal to the value computed using the column value for that row as an input in the sharding function. So key space ID equal to f of the value for row in that column. Okay, so for example, if you have a table called customer and a sharding column that you're using this ID and a sharding function is hash, then for a row where the ID is one to three key space ID would be hash of one to three. Right, straight forward. Now let's talk about what a shard is. So each shard represents a key space ID, an ID range. So each shard has a begin value and end value. And if begin value less than key space ID less than end value for a particular row, then that row belongs in that shard. Right, so that's what a shard is. In terms of key space ranges, each shard has one master and multiple replicas. And the thing to remember is that a shard can be located in one on one or more cells. So these replicas and masters that constitute a particular shard can live in one or more physical cells. And we talked about, we said that a Vindex is basically a combination of a column and a sharding function. And so these are the sharding functions that are predefined in Vitesse, binary md5 hash, numeric, numeric static map, unicode loose md5. So if you basically have a int or a long int as your column on which you want to shard, you would probably want to use hash as the sharding key. If it's a name or a var care or a var binary, you probably want to use the unicode loose md5. Right. There are some people who wanted reverse bits, you know, they just wanted to reverse the bits of the primary key. And so we built that, that for them. So these are the predefined sharding functions. So you can choose a column, you can choose one of these functions, and that allows Vitesse to calculate the key space ID for a given table for, for rows in the given table, and that's what decides which row that particular row, which shard that particular row is going to live in. Right. So that's what sharding functions are. So now, having understood all these concepts, let's go to the problem at hand that we are trying to solve. Now, before I jump in here, let me once again ask Chris if there are any questions. You're doing just fine. All right, since there are no questions, I'll move to the problem that we are trying to solve today, which is building a jurisdiction aware cluster, a database cluster that stores data for a given country in its jurisdiction. The client application did not be aware of jurisdictions, right. And this is important. So you need, you don't need to rearchitect your client applications, while you rearchitect your database to comply with data locality requirements. So for this demo, we are going to have four jurisdictions and eight countries. The four four jurisdictions are Americas, Asia, Africa and Europe, and the countries that are two countries that I have picked corresponding to those jurisdictions, right. So eight countries for jurisdictions. And these jurisdictions map to cells that we say are located in that jurisdictional area. So the cell that is located in the Americas is called US West. The cell that is located in Asia is called Asia one. The cell that is located in Africa is called Africa one and the cell that is located in Europe is called Europe one. And what we are going to do is we are going to create a key space with four shards, and we are going to instantiate one shard in each cell. And then we are going to use a custom geo V index. So this is a V index that I wrote. It took me about six hours to write. This is the cool thing about with us that once you index is an is an interface.  So it took me about six hours to implement this. And so this is a custom V index that I call custom geo that distributes data correctly using the country field in each row. And this is how we are going to design the cluster. Our V index will use a country to jurisdiction map. That basically maps countries to jurisdictions, and you can see this map. It's a JSON. So this sample data that we would be using Philip Roth and Gary Schengart are two of my favorite authors.  And so we have two, two rows per country. And without further ado, I can actually show you how this looks in practice. Again, I assume there are no questions. Otherwise, Chris would I interpret me. Correct. Yes. All right. I have SSH into AWS and I have, you know, four or five posts here in which I'm going to spin up this cluster.  That means that first, first we will start a zookeeper quorum, which will act as the topology server. Then we will start a VT CT LD, which is the control plane. Then we will start a VT gate, which is the stateless proxy to which our client will connect. Then we will start the VT tablet my security combinations for the key spaces that we will use. And once all this is ready, I will show this to you in more detail. So I'm going to run this shell script called start cluster. And this will do everything that we had talked about. It started the three zookeeper clusters. It started the VT CT LD. And then now it's starting two different key spaces. There is one key space called lookup, which is a single shattered key space. And then we will start the actual shattered key space in which we will store the data that we want stored. And if you see the V schema object passing by here, V schema is how an operator tells Vites about how their key space is shattered. So it tells you which for each table in the schema for that particular key space. You have to tell Vites which column to use for sharding and which sharding function you want to use for sharding the data in there. So all of these, you know, this is where we started the sharded key space. All the shards. And this is the V schema that we applied to this key space called user DB. And I would like to point this out to you here that we first define a V index that we are calling GDPR VDX and it's of the type geocustom. So geocustom is hard ported in the new code that I wrote saying that that's the type of the new index. So we are creating a GDPR VDX of type geocustom here. And we are, this uses our data from this GDPR config.json for mapping the countries to jurisdictions. And then what we are doing is that there is this table called customer, which is the only table in this particular database. And we are saying that use the column named ID and the index named GDPR VDX that we defined up here as the sharding function. So this is how the V schema works, right. So now what I'm going to do is I'm going to show you all this through the VTCTLD interface. So you can see that this is the witness control panel. You can see that there is there is one look up key space with one shard, which lives in US West. And then there is one key space called user DB, which has four shards, which lives shard zero to 40 lives in Europe shard 40 to 80 lives in Asia. Shard 80 to C zero lives in US West. And shard C zero to dash lives in Africa. Right. So there are four shards. Each of them are instantiated in cells that are in different localities. Right. Now let's take a look at how the schema for lookup looks like. So there are two tables in schema. One is called customer country map. And here we are keeping a map of the ID to country. And we are also saying that this ID is a sequence that is so witness has this cool feature called so basically in my SQL in a single sharded my SQL database you can use auto increment to get a unique ID for each row. That doesn't work if you have a sharded database because each each of the shards can have the same ID if you just said auto increment. So witness has a feature called sequences, which you can use to say to create an ID which can be distributed across multiple shards. So this customer country map. So that that is the that the table that backs those IDs is also lives in the lookup piece base. Now let's change to user DB, and then there are four different shards and each each of the shards is going to have the same schema. So this is the schema for the customer table. There is the ID, there is a full name, there is a national ID, and there is the country so national ID is basically my, I just thought that we need to put some PII personally identifiable information so national ID is the current or social security number. So this is the country itself in the primary keys ID. So this is the, and if I go to any of the four shards associated with this key space you will see that they have identical identical schema, and you can see that there is a V index called GDPR VDX of type in that is applied to this particular key space. So now that you have seen this, and you can see that there are zero rows in all of these right now, right, because we haven't added any rules. So let's go back and let us let us look at what data we are going to insert. And if you look at this data, it's basically this ID will not be used, we will be generating the ID, but the name, there is the full name, there is the national ID and then there is the country. And this particular Python script for client GDPR.py is going to read this file, and it's going to insert it on the VT gate that is running on port number 15306 on this machine. So I'm going to control C, control V, and it connected to this VT gate here, and it inserted all these 12 rows. So we are going to read these rows out from the database. But before we do that, we can also see here, if I refresh this, you can see that customer country map has 16 rows in it in the lookup database. And if the user DB, each shard now has four rows. The question is whether they got into the correct rows or not, right. So, for that, what we are going to do is I'm going to actually connect to my using the MySQL client. I'm going to connect to the VT gate, and I'm going to run this SQL script called showJDPR.sql. So let's take a look at what's there in that script. So Vites allows you to either access a key space as though it's a single database, you can see here where it's saying use user DB. So if I say use user DB and select count star from customer, it's going to show me 16. But if I say use user DB dash 40, which is the dash 40 shard, and then say select star for customers, it will show me only the four rows that live in this particular shard. And if you remember, the first shard is associated with Europe. So you would expect for the first four rows to be from Europe. The second four rows to be from Asia, the third four rows to be from US and Canada, and the fourth rows to be from Africa. So that's what we are doing here. So let's run this showJDPR.sql. And as expected, you can see that France and Germany rows have gone into the shard, which is located in a cell in Europe, the cell called Europe 1. Then the Asian rows have gone into a shard that was located in a cell in Asia called Asia 1. The North American rows have gone into the one called US West, and the one in Africa have gone in Africa. And if I instead just treat it as a single database, I can see the count star as 16. So I also want to just connect here to MySQL and show you how this looks. So this is just standard MySQL client. This is a lie. The server version underneath that we are using is different. You can, there is a VTGate parameter that you can use to tell VTGate what it should return the version number as. This was the first MySQL version that we supported. So that's what a Vites server response as the version number, but you can change this. We are actually running on MySQL 5.7. So if I say use customer, and I say select star from no use, I should say use user DB. I should say select star from customer. You can see all 16 rows. At the same time, if I say use user DB-4D, select star from customer. It just shows me the four rows in that particular shard. So just wanted to show this to you. And what is interesting here, there are a few other things that I wanted to show is that the code for the Geo custom, yeah, I first wanted to show you how the client looks like. So basically we are inserting into the customer country map first, and then we get the ID that was generated and then we insert that row into the user DB customer. So this is all, it's very straightforward logic. There is no mention anywhere of the cells or there is no mention anywhere of the jurisdictions. But that mapping comes from here. Here is where we are mapping the country to jurisdictions. So given a country, we figure out what the jurisdiction is, and then there is a range that is spanned by that particular jurisdiction. So depending on the offset of the country in this list, we use the byte corresponding to the range in that particular jurisdiction. And this particular shard uses that byte as the first byte, and then the user ID, and then hashes it to create the key space ID, so that all the key space ID is generated by any country have, you know, they fall in a particular range, and that's how they end up being in the same shard. So for those of you who are interested, this is how it is done. I could have just had, you know, a byte associated with every country, but this gives you some notion of adjacency and so on. So if, for example, UK was in Europe, and if UK decides to leave Europe and get into its own jurisdiction, having it at one of the ends of the Europe's ranges would allow you to create an own jurisdiction for Great Britain. And that's why I have this, the jurisdiction ranges and so on rather than hard coding a byte for each country. And as you can see you can do similar things with zip codes, if California, as California's data locality laws go into effect, you could have a California jurisdiction and rest of the US jurisdiction and do the mapping based on, say the zip code, zip code field for your data. Any questions? Yes. Chris asks, can Bethesda be configured to support multi tiered sharding, for example, each jurisdiction may contain a set of shards defined by an alternative key space. Absolutely. So the each if so only the first byte is used to decide the jurisdiction and the rest of it depends on the user ID. So if you have hundreds of millions of user in a single jurisdiction, you can, you can split that shard and you can put both of those shards and the same jurisdiction and the rest of the logic just continues to work. Any other questions? There are none in chat. That was the only one in Q&A right now. How would you have, like you mentioned California, how would that work if like how aware does Bethesda have to be of the underlying infrastructure or how do you define that, I guess. Correct. So the way you do that, let's say that you would create a cell in Sacramento, right? And let's say that you create another cell, say either in Oregon or maybe New Jersey. Right. Right.  Right. Oh, OK. And then you'll write this JSON file which maps, you know, it allows you to create a shards for California zip codes which get located into the shard that you have started in the Sacramento cell. Cool. Yeah. So it's about about a day's worth of work and we are making this as general as possible.  So I also wanted to show you the go code that does all this which fits into the rest of the test. And that's like 180 lines of code. Wow. And a lot of it was boilerplate, but the guts of it are, you know, so this is, this is, this is all boilerplate. But the guts of it are, you know, here I'm calculating the byte for country. So given a country, this returns one byte that I am prepending to the user ID. So here is the name that we used to define it. And if I look for, yeah. So I first convert ID to country by doing a lookup in the mapping table. I get the country. ID is the input to this map function. I get the country, then I get the byte for the country. And then what I do is that this is the hash of the ID itself. I use seven bytes out of that. And then I append the country byte with those seven bytes. Cool. And then I hash it. And that means that so similarly, if you're using, if you're using one byte for California and another byte for the rest of the country, you would do the same thing. And then just as long as you, yeah, you make sure that your shards are located. And if you remember how the shards are named is 00 to 40 so that that's basically a hex, yeah, slices of the whole key space. So, and you can keep splitting the shards as, as, as you need to. That's a cool stuff. I mean, congratulations on all the work put into, you know, push for tests over to the graduation side of the CNCF project framework to by the way, the, the, the amount of scale that I've seen people use for tests to handle is rather impressive. This, this to me is just like next level fascinating handling all the data locality rules, just with, you know, boiler pray code to kind of out of the box is awesome. Thank you. Yeah. Any questions, not right now know there are no questions 15 people on the call, besides all of us, you know, presenting and helping talking and so forth so there's anything else you want to talk about, you know, keep calm your party anything. Come party with us everybody.  Where is it again sorry I missed the, the location. Yeah, let me go here and let me give me one second. No problem. Here's a question. What is happening. Sorry to make you change gears. What is happening if jurisdiction configuration change is it supported for example, if it's has will migrate the data in the background. So like if you added a new data locality said go here. How does that like affect the data that's already there. So you will, if you added a new jurisdiction. So there are two ways in which you can change it right you can either add a country to an existing jurisdiction, or you can add a new jurisdiction. Right. If you add a new jurisdiction you have to keep in mind that you are using bytes. So let's say that a particular jurisdiction that the way the way this has been set up is that of the 256 each jurisdiction gets a quarter right so 64 64 values for each jurisdiction that that's the way it has been set up correctly. So if currently, so let's say that a jurisdiction has 10 countries in it. When you add a new jurisdiction the beginning the start byte and end byte you have to make sure that they start after 10. So maybe out of 64 you say that the previous jurisdiction gets 32 and the new jurisdiction that you're adding wherever it is gets 32. So in that case you don't need to change anything. Similarly, Great Britain I mean the thing is that the so in that case everything works as expected. You don't need to copy data from one place to another, but let's say that Great Britain leaves European Union right and gets out of the jurisdiction. That's a tricky problem to solve unless you had the forethought to put Great Britain either at the beginning or at the end of a particular jurisdiction. Okay, so you would have to do some kind of data move of some sort to call us it maybe. Right, I mean, if you kept this in mind and you as the first country you can just split it up as a different as a different jurisdiction without moving in the data because the byte associated with that country will not change. Right. So if you split the existing shard, we just have that key range be in UK and the rest of the key range in Europe. Would you think it'd be like a good practice to if you have a data center in that country like just go ahead and set it up as a cell whether it's in a jurisdiction or not just so you could predefined it potentially as a place. I think that's it. That would be a good idea. Okay, yeah, that's, I think that's good advice. Okay, cool. Thank you for the question said. Yeah, so let me just quickly. I think I'm looking at the wrong. Yeah. So Kim just said if you're okay with it like we can open the question like open the the voice up for all participants, like if just 15 of us like I'm pretty sure we can manage that. If you're okay with it. Sounds great to me. Yeah, let's do it. All right, Kim hit it. Cool. So talk and they just have to go ahead and unmute their mic. If you are muted, and you want to talk just unmute and ask away. Again, the the Vitesse AWS planet scale Q com kickoff party is Monday, November 18, 6pm to 10pm at We don't have the address here. It looks, oh, it says meze. I can look it up. Hang on. I got it. I will pull it into the chat. Just give me a second. Yeah. So if you have a question, you know, this is the test knowledge base with you here for next, you know, 15 minutes or so please ask away. There's the website and address is. So may say Greg fusion is the name of the place. Yeah, that's pretty cool. Let's see their address is and have to click on the Google Maps link to find it. And of course, Google Maps is probably on Google infrastructure. So it's going to be raining a little slow today. Let's see. No offense to Google. All right. There we go. No problem. So, there's the address dropped in the chat. The website is gas lamp mezee.com. If you have any questions about menu or anything else had there. You know, we're live right now. So if you have a question on mute. And, you know, we could talk about nice QL I'm assuming the tests in general CNC a project graduation cube con the whole nine yards. That's right. We're here. Really excited to have graduated. Yes. A question the chat from Eric, I may have missed it. But were you able to describe how a schema change gets handled. So schema change basically gets handled independent of how the jurisdiction of clusters work. Right. It's just stand with us has commands that allow you to apply schema changes to multi sharp key space. So it it sends the same DDL to all shards in parallel. Some people in production also use tools like GHOST or PTO SC to do online schema changes. And there are plans to build support for online schema changes in the test itself using the new verification feature. Nice. Does it ask a question. Eric, does that answer your question. You can say yes by hitting on mute. Also, yes. Thank you. Awesome. Cool. Any other questions? If not, I guess we can wrap this up. Okay, cool. I think I will close out if you're okay with it. Sounds good. Yeah, thank you. Thank you so much. Great presentation. Lots of information. Lots of demo. So thank you to the demo gods for being thoughtful there. If you have any questions, you know, feel free to contact Shatana myself. And, you know, we look forward to seeing you at a future CNCF webinar and we hope you have a great day and a great week. Thank you everybody. Thank you. Bye everybody.


For you, the day you graced your subreddit was the most important day of your life, but for me, it was Tuesday. Welcome to r slash dank memes from site 19, the bi-weekly show where I review, I guess the reviews the right word, I look at memes from the scp r slash dank memes from site 19 subreddit and give a little bit of commentary about what I think about them as a scp author myself. So let's get started learning about scp's from the wiki small brain learning about scp's from youtube regular brain learning about scp's from containment breach big brain learning about scp's from r slash dank memes from site 19 galaxy brain see this is the funnily enough I don't know that learning about them from any particular sources really matter as long as at some point you read them I guess that's the important part wherever you learn about them who cares learn from them where you enjoy learning from them you're not going to lose anything by learning only about them on scp containment breach as long as they're accurate of course scp containment breach is problem is that they just have the popular like series one ish style I know they've started to incorporate other ones but mostly at series one stuff so I guess the best way to put it is it doesn't matter where you learn about it it only matters what you're learning about foundation discovering something that doesn't exist mtf any scp after a round of amnestics there's nothing here scp amnestics the overused story trope me has a small cough my parents you are fine doctors you're fine web md zombie plague that see this is inaccurate web md would not tell you have a zombie plague it would tell you have cancer or heart disease or cancerous heart disease but that's the trouble by the way web md is a lot more accurate than people give it credit for the problem is that when you find out that you know nothing is wrong you keep looking until you find out something might be wrong there aren't any wholesome scp's scp 4999 2295 348 3786 3540 999 there's more than just those even but those are good examples I suppose you know I don't know that I've written any holy wholesome scp let me think about that for a second well I guess 2913 is sort of sort of wholesome doesn't involve a serial killer but there's just a sentient severed hand that really wants to sit down and watch some TV and not be left alone I want to join an anomalous organization Google scp foundation being chaos insurgency excite dado doesn't say that would have been kind of funny what are the excite actually I don't even know if excite still exists Jesus being exist for sure I keep getting these ads for this like use our search engine so that you can plant trees on YouTube and I'm like but I what if I don't like trees what if trees are bad what if I think the trees are going to kill us all not that they are that's a privileged information don't tell anyone that use use a search I guess maybe advert that same way Google probably makes money advertising or whatever eat either way is like I love you Spicer and I love you fly very well let us go my love to pastors unknown and log approximately 10 seconds after this yeah the approximately 10 seconds after this is let's just say a trope for the scp wiki it's like something happens something happens something happens and then stops happening forever and it's like ah see but you didn't expect that the problem is that now we've come to expect it as a trope so unfortunately it doesn't work anymore I guess it works for some people worked for this guy me after any or slash there's a subreddit for that oh no really of course it's I'm saying look I don't get it man I don't get it I just don't get it the time for talk has passed the Lord's work must be done I mean also I don't I don't get this reaction to it either I mean I don't get why people want to see porn of scp articles but also like let people be what they want to be let them do what they want to do it's not my business I don't have to understand it to be like you know what do you do you I mean I'm don't get me wrong by the way don't get me don't get me twisted I'm gonna be judging you all the way down the line but forever and always but you know what don't mean anything you gotta do what you enjoy the bodies in the water this is just this is just by the way there's a lot of there's you do not recognize the bodies in the water memes that I found when I was doing my or compiling my list this week that's like I was in the break room at the time of the breach Dr. Olasky there is no break room on the site see this this I want to tell you something this doesn't make any sense to me never did make any sense to me Dr. Olasky's got to know first of all there's no break room on the site it's a big site there's a break room somewhere second second of all there isn't a break room and Dr. Olasky either doesn't know that or doesn't think anyone's gonna notice that is such a hmm let me put this way the twist in that is well thought out but the execution of it is so haphazard because sure finding some sort of serious inconsistency in the story that it wasn't accounted for makes sense but the the break room is such a stupid choice how are you not gonna know that I was in the break room at the time of the breach you didn't eat this man thought out things down to the fucking like atomic detail but his in his plan in his backstory his alibi I was in the break room at the time of the breach I come on bothers me more than it should probably but see this is another bodies in the water bodies in the water exist me I don't even know who you are I love that scene though which is amazing and I love they made her super powerful even in the MCU like she's a god tier character in the comics like literally alters reaction she's a reality bender in the comics she's a little bit less powerful in the MCU but damn she definitely could have taken on this is the best that part is though she could take it on Thanos at any time and just straight up killed him and just didn't do it because of plot reasons but yeah you want other than that and still that's the problem with having a character that is as powerful as she is that's like how do you uh yeah how do you how do you not just have her solve the plot every single time let's have her be preoccupied that in the last movie had her try and destroy the minor like uh guard uh vision and keep the mind stone and maybe try to destroy it or whatever she could have just you know destroyed Thanos anyway what's wrong Pucci you want to reset the world r slash s e again I will make the same comments as before let people enjoy what they like judge the fuck out of them but let them enjoy what they like licensing staff going through the wiki no pictures just the way I like it that's not true that's not true I used to be licensing staff I'll tell you right now it's not that we want to have no pictures you just need to make sure that your pictures are compliant with the license and you're like well that's too difficult no it's not too difficult you stupid motherfuckers just like it's you know how many images you can find on creative common sites or even site pay for them if you want to pay for them pay for them if you don't want to pay for them because you you know you're broke or whatever then find there's so many free images or make your own second scp I ever did for the scp wiki which I think was the very first image I ever put up was just a picture of my own hand whatever there's no Easter Bunny there's no Tooth Fairy and there is no foundation timeline there is just differs from article to article there's no definitive foundation timeline come on corporate needs you to find the difference between this picture and this picture they're the same picture that's not I remember the the title of this one was I look just like this scp you don't look just like this scp that mask is far more handsome than you do I recognize the bodies in the water you do not recognize the bodies in the water you do not recognize I wish Google actually did that that'd be pretty great censoring stuff the Chinese government the scp foundation oh no I'm gonna get banned in China no I don't think I'm important enough to get banned in China probably the definition of attractiveness has changed a lot 1915 1945 1985 dr. Powers you know what for the core of this meme though like I don't get this what do you mean the definition of attractive this is just an attractive woman in three different outfits and I assume the original picture it's four different outfits I wonder what the original image of the 1985 was when I was born Jesus yeah this is just an attractive woman in three different attractive outfits I guess the first one's a little frumpy but still the second one's fine and the third one's fine it's like let's pretend like our standards of change that to be fair our standards have changed but not over this time period there was a time I was that then in the Canterbury tales there's like a line that I got really confused by where like he talks about it's and this is like what 1600s I think Canterbury tales 1600s either way it's really it was a decent time ago and the guy talks about how attractive a woman is because of how large her forehead is and I was like the and I looked up that kind of thing is like yeah beauty standards like the concept of what is attractive has significantly changed over time but just changing people's clothing that's not the same thing I'm sorry and yes I've somehow gone from I didn't talk at all about Dr. Powers because I actually I don't I don't know this SCP I think it's probably one of the newer ones but yeah this is just that's the reason why that stuck out to me SCP-2913 and this is this is the uh this is I've talked about this a couple times already and the reason why is because I guess it was on my mind uh but SCP-2913 is actually my second SCP article it's the one with the sentient severed hand uh aha well that's a long story I used to belong to this guy called Jimmy Jimmy was a real weirdo and for the longest time my only friend was Joe and who is Joe Joe mama no I don't think no that's not fair rare disc a heavy black disc studded with peculiar redstone when you hold it you're overwhelmed with a feeling of dread it's SCP-093 in Stardew Valley that would that would be great this should be like an SCP Stardew Valley mod how would that work what would you do oh my god thinking about it now I guess you could make it so that the cave was actually an SCP side like you down to like floor 13 or something like that and it's like there's just a mysterious door and you go in it and you're like oh hi and then the mts come and shoot your ass I get to see that being a thing that'd be interesting either way thank you for watching let me go through all of these SCP memes and talk about them if you liked the r slash dank beams from site 19 video or if you like any of the other videos I do on my channel scroll down hit the subscribe button if you don't I will activate the failsafe nuke at site 88 I'm not joking I am joking I'm not joking I am joking either way you'll know and after you've done that head on over to patreon.com forward slash decimarian and pledge at any level like the people in the screen here already have join them in helping support the channel and letting me know that I'm not alone out here I'll see you all again on Thursday


In today's video, we're opening the new Yu-Gi-Oh! product, Speed Duel, Duelist of Shadows. What's up, guys? We're back with another new Yu-Gi-Oh! product video. This has the Speed Duel GX Duelist of Shadows box. So, first of all, I don't know a lot about this set. I just wanna say, I'm so happy that they shrunk these because they used to be like this long. They were massive. This is much better.  I like it. But before we continue opening, I'll be giving away one of these, I guess you call them mini boxes, just like the video, be subscribed, turn on notifications. Let me know what you think about Speed Duel in general and this Speed Duel product. Seems like they don't do a lot of Speed Duel releases, but they are usually pretty cool. Yo, I'm back. But, Ruxin, where are you back from? If you guys don't know, I streamed on Whatnot back in 2021 and I've made my return to Whatnot. If you guys don't know what Whatnot is, Whatnot is a streaming platform where you can sell stuff live. And if you're a viewer on Whatnot, you can show up to the live stream, watch the stuff get open, watch stuff get sold. It's really fun to see the auction prices and people going back and forth trying to get different stuff. And you might think, I don't really wanna spend any money right now. Well, that's where I come in. Use my link down below and you'll get $15 for free when you sign up for Whatnot. It doesn't have to be on my stream. It could be on Team Samurai stream. It could be on Simply Unlucky stream. It can be completely unrelated to Yu-Gi-Oh cause this isn't just a TCG platform. There's also a lot of other cool stuff like shoes and other collectibles like that that are being sold on Whatnot. And I will be streaming live Thursday, March 30th at 5 p.m. central we'll be selling stuff like Epic Yu-Gi-Oh singles and old school Yu-Gi-Oh packs. And I will be giving away over $500 worth of Yu-Gi-Oh cards in this stream. So you guys wanna make sure you're there. And some of these giveaways will not require a purchase at all. You can simply be viewing the stream and win. And don't forget to use that link down below to get that $15 off when you come to the stream so you can save yourself some money. Thanks again to Whatnot for sponsoring this video. Let's hop into this. Normally they do have like the secret rare reprints in here. So I don't know what the secret rares are in this one if they even did those. Secret rares of all three secret secret beasts. That's, I mean, that's pretty awesome. So let's open this up and see what's inside. And of course in this video we have more than just this one box. We'll be opening a bunch of these to see, well, assuming there's different secret rares I guess we're gonna find out. If there's not, then I guess there's no reason to open too many more of them. But okay, here we go. So first of all, we have the pack. So I guess you always get the three secret beasts maybe. We'll come back to that because that's the really exciting part. Then the rest of these look like we have four decks which I don't remember. Do we have more than four last time or was it like, or was it four? Might have been eight or so? I don't really don't remember.  So we got those. All right, I'm gonna open these up real quick and then we'll get back to like what's actually in the decks. Cause we don't, you know, this stuff's impossible to open. We don't need to show me struggling. Okay. All right, the first deck we have has all the skills cause this is basically dual links. You have skill cards. So they give you like an extra effect or whatever or an extra. Basically it lets a lot of decks that are not playable become playable which is what I like about speed dual and dual links. Doesn't really work with master dual cause that's just straight up regular decks. There's no like effects like this. So maybe they'll add, I don't know. They probably want to add speed dual to master dual cause then they cut off their own game dual links. That wouldn't really make any sense. So, oh wow. This is just all the different cards. Probably all the skills you can get here. Then we got dark scorpion chick the yellow. Okay, dark scorpions. We got pharaoh servant. The old school booster box in there. We got the pharaoh protector. So it looks like an old school pharaoh deck which is pretty cool. Gaga Gigo. All right, this is pretty interesting. It's so interesting to see like, wait, rainbow overload of dark world. Okay, so dark worlds are in here now to see like what they add into speed dual. Cause like some stuff is like really old but it's like really good in speed dual somehow. Oh, elements of shining flare wing man. We didn't have this. I thought we already had these from the other GX boxes but I guess not. Terror king, salmon, arm dragon, cyber dragon. That's pretty big. Vampiric, koala, allure of darkness. Good card. Book of moon, counter cleaner. Flash of the forbidden spell. Chaos trap hole, okay. That's a pretty good card. Nightmare wheel and Zoma. Zoma was already in dual links. I know a lot of these already were but Zoma was pretty big for a while. All right, next up we have another deck with the helios, the golden homunculus. I remember these guys had like the special shiny print laminated foil. I believe if you ask you through Yu-Gi-Oh. We have disc blade raider. We have the DD survivor, DD scout plane, the different dimension stuff. Oh no, banisher of the radiance. DD crow. Oh, it's getting crazy here. We got tripe shocking virus, galaxy cyclone. There's just, man, speed dual is getting a lot of good cards, macro cosmos as well. I don't know if that was already in there or not but that's pretty big, D boys. A lot of old school stuff but there's a lot of good stuff coming into this game. It's getting pretty crazy at this point. This is such a weird card. This is the one where, yeah, you gain 3,000 life points but it leaves the field you lose 3,000 life points. You really gain like zero but you kind of gain some. It's very weird. Final attack orders. The cloudian deck, interesting cloudian deck in there. If you're interested in playing that it's your speed dual deck. Let me know in the comments. Do you guys actually play speed dual? I don't know a lot of people that do play it but like the people that play it really like it. And I played it at my very first event in Charlotte last year and it was extremely fun. Like, I mean, I played dual links so it's similar but there was a lot of fun there. I opened this, I thought. All right, then we got some evil heroes going which makes sense because this is the duelist of shadows from GX. So we gotta have some sketchy heroes. Then we got the regular old heroes and frontal prodigy you gotta reprint in here. That's actually probably a good thing for that and like the real version. Dark fusions in here. All these by the way, if you're playing super poly that's good for super poly. These can be played in regular duals and speed dual but regular cards cannot be played in speed dual. So that's how it works.  They're just basically reusing cards from back then and making their own format. All right, last deck before we get into the secret rares we're gonna be opening a bunch of those secret rares for fun to see if we can get all the secret rare cards in the set. Oh, this looks like a fun one. Goblin, zombie, plague wolf, Zambina, Ryukoki, call the mummy. Yeah, this looks like my kind of speed dual deck. I'm liking this. Vampires in Amazonus. Ooh, Amazonus are really good but they're very annoying to play against. They were very good in dual links for a while. This is basically the full dual links deck right here. So that's pretty tough. And then you got the Sculve Archfiend of Lightning.  So I could be wrong about that. I could be speaking out of turn. Battle's card, but let me know. Maybe they haven't been good at some point but I don't think in dual links are speed dual. Could be wrong though. All right, now it's time for the secret rares. Let's see what you get in these secret rares. So I think you get the Sacred Beast every time and then after that it's kind of random. So there's normally, there is a randomized feature to this. So I don't know how many are random. Let's check. We have Uriah Lord of Steering Flames. First of all, looks really good. The speed dual secret rares always look awesome. That's a good one. Oh, we got a Ruxen special right past the Haman. Sorry, Haman. A lot of times you can't see these very well because like, you know, it's behind a lot of the artwork but they actually put it over the artwork which is cool on the wings. And then the Ravi-L, very cool. Then we have an Amazonus swordsman. Speaking of Amazonus, this is a nice card for that deck. A DD Crow Secret Rare Print. That's nice. So we have one of these in, I was gonna say Legendary Collection Kaiba but that's actually an Ultra. We might not have that. It might be the first print. Fog King, this is an Ultimate Rare but it can also be used in Edison which is cool. Evil Hero, Malicious Fiend. It might be playable in this. And then Helios, the Primordial Sun. This is another one that has the Laminated foil. All right, so I have 10 more of these. We're gonna open up all the secret rare packs to see what we get inside. Hopefully we can get all the secret rares that you could possibly pull. That'd be pretty fun. I'm gonna open these up real quick so you don't have to watch that part. All right, we have 11 or 10 actually. We already opened one. We're gonna get the Sacred Beast a lot. We know that. But we are trying to pull all the other cards in this set. So first of all, we have Uriah Lord of Steering Flames. Very cool. We have the Haman Lord of Striking Thunder. Raviel Lord of Phantasms. Then we have a Dark Fusion. So this came as a comment in there but there's also a secret upgrade. I feel like they do that a lot. They give you the card and then they give you a secret upgrade which is actually pretty cool. You can get the Hollow version or the regular DD Crow again. Very cool. Amazonas Queen for those Amazonas decks. This is just filling out those really good cards. Macro Cosmos as a secret rare. That's actually, these are good prints for the TCG actually. And then an ill blood for zombie decks. Wow, these are really cool secret rare choices. I feel like Speed Duel's got a lot of cool decks coming to it. I'm gonna try to see if we've pulled all of them. I think last year it was like 16. I could probably check and see what it says to how many there are like different secret rares besides the three beasts. We have the three beasts of course. Then we have an Arm Dragon level 10 iffy but it's in speed duel so maybe it's decent. Vampire Genesis. Too bad we don't have our Vampire Genesis mat anymore. We do still have it but I'm not using it. Elements of Hero Shining Flair. Wingman, beautiful secret rare print. I mean, look, if you can't afford the ultimate rare for 1500 bucks just buy this for like two bucks. That's great. We have the Aqua Chorus. Wow, that's a weird secret rare. Maybe it's good. If there are monsters with the same name on the field those monsters gain 500 attack defense. Maybe that's good in speed duel. Who knows? And then Dark Fusion again, okay. Oh, here's some background stuff that I probably should have said earlier. We have eight different characters and they all have different decks. So it was Evil Hero, Red Eyes, Vampire, Amazonus, Archfiend, Macrocosmos Deck, Sacred Beasts, and Cloudian. And then we have the skills for them. So there was eight different decks in there. So it was, I knew it was eight. That's what I thought it was. There was just four packages because they're so small you could fit two in there. Okay, so what you get is three Sacred Beasts. Then you get five secret rare upgrades from a set of 21. So there's 21 different ones. I don't know if that includes these. So there could be 18 plus three. I'm guessing that's what it is because 21's a weird number. And then 20 skill cards. Okay, that makes sense. We probably should have checked that before we opened these. All right, back to it. Let's figure it out. So it looks like there's 18 cards that we need. I guess we could see if we're gonna get all of them. So first of all, we have these three. Then we have a Vampire Genesis again. We definitely have gotten that. Legendary Jujutsu Masters probably actually pretty crazy in speed duels. So that's a good card. A super poly. Wow. That's unfortunate that there's a reprint here because like Raio the Mega Pack already had one Legendary Collection GX. So that's definitely gonna hurt those values. But I guess that makes it easier to get the nice secret. And also the speed dual secret just hits. So maybe it is not terrible. I'm dragging level 10. Cool. Let's see what we can get. Man, these secret rares just look so good, man. They do such a good job with these. I don't know why the regular secrets don't look this good. Like they just have like, it's almost like a wetness. I don't know. I don't know if you know what I mean. It's like so reflective. It's like almost like it's wet, if you know what I'm saying. Maybe I'm sounding crazier, but I feel like it almost looks like you're looking into like a pool because this thing reflects so much. It looks so good. Jujutsu Master, we got Dark Fusion. Okay. Allure of Darkness. Whoa. That might not have a secret rares until now. I think it was Ultras and Supras and Ulti, and that was it. So wow, that's a big upgrade. Very nice. Malicious Fiend also really cool. And then a Fog King. Okay. So Allure of Darkness. That is a nice hit. All right. Still a lot of reflective cardboard left. Let's see what we can get. Will we be able to get all the cards? I think we have most of them already. Golden Hamunculus as a secret rare. Okay. We got the level 10. Vampire Hunter. Okay. Wait. I just looked and I think we had 17. A Skull Archvene of Lightning and the Primordial Sun. So that was three different ones. So I'm pretty sure there is actually 21 besides the Sacred Beast. I don't remember if we'd pulled the Hamunculus. So there's, I think another one that we haven't pulled still. So that's pretty cool. We're getting a lot of different ones here. We've got the Uriah, Haman, Raviel, Aqua Chorus. We've definitely gotten that one. We have the Amazon Swordswoman. Very cool. Skull Archvene of Lightning. Allure of Darkness. And Cyber Dragon. What? They got another Cyber Dragon secret print. Legendary Jet Collection GX and Raviel Omega Pack are getting killed. All their cards are getting reprinted in the same rarity. That's pretty interesting that they're choosing to reprint those. Are they like running out of reprints? I don't know. It's kind of weird that they're doing them all. I mean, I guess Secret Rare. A lot of cards have a Secret Rare print. We have Haman, Raviel, Golden Hamunculus again. Cyber Dragon again. Was just Veined, Diddy Crow and Skull Archvene of Lightning. Those are beautiful. Three more packs. Let's see if there's anything we missed. I feel like maybe we've gotten them all, but I could have missed one potentially or miscounted.  It's, yeah, I don't know how to describe this, but if you guys have ever opened like one of these little foil packages, they're very frustrating. They're just, you're shaking your hand all the place, welcome on. Okay, we've got Super Polly, very nice. Vampire Hunter. I don't even know this card. Started damage to have it battle as a dark monster, destroy that monster. So it's like Ally of Justice Catastor, Cyber Dragon, well, that's a non-dark monster. And then Diddy Crow, very cool. And then two more packs. Two more packs for the Secret Rare's. Five Secret Rare's per box is not bad or extra Secret Rare's. We have these three. Now I shouldn't say extra Secret Rare because that's an actual rarity, not the rarity, just another Secret Rare. Golden Monkulous, the Allure, Macro Cosmos, Swordswoman, and Dark Fusion. I feel like they kind of crushed it with a lot of these upgrades. And Dark Fusion was actually one, there was like a three and a half dollar card in Ultra Rare out of Legendary Collection GX. So probably a good one to get a Secret Rare. Final pack of the Speed Duel. Let me know what you think about Speed Duel in the comments. Make sure to subscribe to the channel if you enjoy the content and the new stuff. I was love opening the new stuff even at Speed Duel and I don't play a whole lot of Speed Duel, but I like to see these Secret Rare upgrades look beautiful. Vampire Hunter's at it again. Red Eyes, darknessy. There was one we didn't pull. Red Eyes, darkness dragon. So this and Vampire Genesis are now Secret Rares. The Amazon Swordswoman and Arm Dragon level 10. All right, so we ended it off with a card that we didn't on half. So we ended up pulling all of them though, which is nice, I think, unless I missed one. Shout out to Tonefo Show, Daxter JTChill, Puffins of Doom, Ernesto DeAndre, Dizzy Hoppus, Choice333, Missycle James Jantz, TCG Trust to Cards, America Deutzer, Supreme Sage 21, and on the Thai Show, Ian Lusa, Junior Barting, Mimic Gecko, and Thomas McClane. Thank you guys for supporting the channel. I'll see you guys in the next one. Peace.


So, we're here at the ID Tech Act show. Hi, so who are you? Hi, I'm Rachel. I'm from ASTAR. Accelerate the commercialization of ASTAR. And today with me, I actually have my Singapore ecosystem. So we have ASAHI, and we have KPP, with our ASTAR licensee. We have SYNTEC, which is the ASTAR Research Institute. As well as KAHA, our collaborator. So what is you doing over there? Okay, here's my colleague. Rick from SYNTEC. Hi. So he's showcasing our new latest application in smart wearable. So now, he's going to demonstrate. There's a machine there. This is a wearable, this is a module that is invaded into the smart apparel. And the smart apparel that I'm wearing is something similar to this. Where he has the smart electrodes and he captures my ECG, he captures my activities. He also has a module like this to transmit the information to the phone. So from there, it captures my heart rate, my VO2 max, which is my fitness, my steps, as well as the calorie that happened today. Is this what it is? Yes, essentially. Oh, this is one of the IOT end-to-end wearable platform that we are utilizing to showcase the software. So the data analytics is actually from this particular software platform. It actually incorporates our technology, which is the printed electrodes. Let me showcase what the software analytics will actually show you through the app application. Maybe Rick, do you want to do a... So what is this one, for example? What's going on here with this button? Okay, so this is a thermal vest. So what it does is there's actually three setting. Oh, there's actually three setting. So there is a low, medium, and high. So the temperature can goes up from 35 degrees to 55 degrees via setting. So if you can touch, we actually have printed heater hatch. So it's very... Yes, it's non-optimistic. It's heating up. Right. We have it in the pockets left and right, as well as the back. Yeah, it's actually made to be very light transportable, easily rollable on the luggage. So it's a fashionable mid-layer piece. So is this printed electronics? Yes. This is what it does. So we print this in row-to-row. This is a... Yes. And we incorporate the electrodes into here. So you can see that this is in row-to-row. And then it pass on to the apparel guys who can incorporate that into smart apparel. And it catches all the vitals from your body and transmit it out through this Bluetooth module into the mobile apps. So what is this material? This is carbon-based electrodes. And it has good connectivity? It has good connectivity. It can survive more than 100 washes. And it is easy to integrate. So we are providing modular concept where it's an open platform where a lot of smart apparel guys can come in to work with us to incorporate our electrodes into their apparel. And we have an entire ecosystem here to support them to create from design concept all the way to Mass Pro in the shortest time possible. So it's an ecosystem? Yes, it's an ecosystem. Like several companies? Yes, we have a value chain partner with us. So that's why today with us we have supplier Asahi. Hai. So who are you? I'm Ray. I'm in the business development team for Singapore Asahi. So we are positioned as a material supplier for the for the printed hybrid electronics space. Specifically we provide a range of inks. For example our silver nanowires. So this is for the fabrication of transparent electrodes. For example applications such as printed heaters, printed touch, screens, printed lights which you can see in the front. So this is using your material? Yes. Is this OLED light? No, it's printed lights. Printed lights. So we have a few layers of materials in there. And next we have our copper range of copper inks. This is our copper complex ink. This is our copper complex ink. And this is purely chemical based. There are no solid particles in this ink. So how you can apply it? You can inkjet it. You can screen print it. And to do this kind of stuff? Yes, so here is an example of what we can do. So these are traces which have been inkjeted onto a substrate. And then after going through decomposition, you form these copper traces. Nice. And what you can do with this? Oh you can form traces such as these. This is just a very simple demonstrator of a secretary with LED lights on them. So it is to serve as a drop-in replacement for the incumbent technology which is the silver paste. It's a way cheaper alternative. And companies like RFID firms are highly interested in this kind of material. What is this? Oh this is again a printed circuitry with our silver paste. And just a bit paste? Yes we do. Is the best in the market or what? Oh definitely. Definitely. Alright. And how's it been the ID Tech Actual for you? It's been great. So we get to see how the industry is developing what trend it's moving into. For example wearables, sensors, what kind of application methods they are going into. And of course to check on your competitors what they're up to. Yeah. Yeah they're also doing stuff right? Yes of course they are. Yeah. But you're also working very fast? Yes we are actually. Working with universities be it local or overseas. And where is local? Local in Singapore. So for example working with NUS, NTU and of course our partner SimTech. Is this some of the best universities in the world or? Oh definitely yes. Yes. Some good students coming out of that right? Yes. Good students, good professors. Alright. Cool. Thank you. And who are you? My name is Melvin from KPB Packaging. We are a printing and packaging company that we do printing in the printer electronics arena. Things like this heat transfer variable material. TPU. So what we do is that you know using the conductive and functional inks and print them out and transfer into the fabric. Yeah. And then others like you know like this one. What is this? This is using the EL so on very specific that using an EL panel and then we you know print and then to enable to light up certain area so that the product showcase what it wants the direct consumers to know what is it all about. Another example like this for advertisement. Yeah. So this is like the magnum ice cream that lights up? Yes.  How does it work? It's more for branding. It works with an EL panel and then you have an inverter and then to power up and then it comes with a sequence of the animation sequence and how probably the clients want them to light up to be. Alright. Cool. And what else do you have around here? Well that one Hi. Hi, who are you? Hi. I'm Loc here from Simtech. Simtech? What does Simtech do? We do manufacturing technology especially here we are doing roll to roll manufacturing. If you look at this so this this is a film that we manufacture in roll to roll and multi layer and we can do as wide as the 1 meter and up to 10 meter long we have deployed this onto a double deck bus in Singapore. Wow. So then it lights up with the ads? Yes. Just like this. So for this we do printing on the touch sensor that you can the thermal form into the ship and integrate into like an infotainment panel of the automotive. So this is molded electronics so what's it called? Yes, it's emol electronics. Emold. Yes, we are right. You do a lot of that? We are working with several companies on that. Alright. So that will be a typical flexible hybrid electronics that will integrate the touch function. Can you hold that up? So what's special about this? So we print the touch function and integrate with the hybrid electronics to generate the different functions so that is a reasonable demonstrator that we're working on. Nice. That's cool. So that's a lot of stuff around here at the booth, right? Ya, they come together as a Singapore platform. Alright. So so how's it been the ID Tech Act show for you? How's it been? Hi. Yeah. Oh, it has been very insightful. So we are able to see what people are doing and then be able to validate our technology, our value proposition and we are able to find new collaborators in this event. So for example, hi, so who are you? Hi, my name is Pawan Gandhi. I'm the CEO founder of Cahar. It's a Singapore company as my colleagues have already informed you. What we've done is we've created an Intuance Smart Wearables platform. Essentially what that means is that we've got in-house capability for the entire value chain for smart wearables right from the hardware electronics to the application framework to cloud, artificial intelligence, machine learning and so on. Essentially we've become a one-stop shop for any company that wants to transition from a traditional wearables into the smart category. So what you see here is a range of products that we've created. Yeah. So like you work on these? These are all created by us. And this would include... Smart watches? These are all, you know, look at the watch that I'm wearing and a category of these watches. This one would look like an analog watch but it is nothing but analog. It's a complete smart watch. It's driven by the software and the intelligence that has been built into it by the software people. So let me show it to you. You know when I landed into San Francisco from Singapore the moment the phone was touching down the watch automatically ended up setting the local time. But it also knows that I've come from Singapore and time and again I want to know at what time my folks are. So when I press this button twice this is what the local time in Singapore is. It is right now 9am. So it's dual time? It's got dual time. It's also got NFC-based payment. So if I go to a Starbucks I just need to tap and buy my coffee. It also picks up my heart rate so it is telling me my stress levels and how to manage it. Can you take it off? So let me show you this one. So it has a heart rate sensor right there? There you go. It's got a heart rate sensor and there's a smart algorithm sitting behind it which is powered by artificial intelligence. And it shows something like this. So what you're seeing is my heart rate my min max. This is getting picked up by the watch that I'm wearing. Nice. Okay. And I can even take this to the next level of plotting my ECG. Now essentially what that means is once you have data like this you can write a lot of deep machine learning algorithms which can detect issues like fatigue, recovering from fatigue, stress level and more importantly there is the algorithm can even go go around to detect certain ailments like heart conditions that might be getting ignored. And you have different designs? These are all different designs. What's the brand? Yeah. So the important thing is that we are a technology platform so we provide our technology to brands. So today we have leading more than 20 brands around the world which have footprint in Europe, Asia and the US who license the technology from us. Nice. So how's it been that I detect a show for you? The show has been pretty revealing to us. You're trying to... No, no, it's okay. Okay. So the show has been quite interesting for us. We've met some very, very cool set of people from the US. We are now working closely with Enterprise Singapore which is a business arm from the Government of Singapore to help us network with some of these opportunities within the US. So it's been a great show? Yes, right. So do remember us. We are from Singapore. Do come to talk to Accelerate and SimTech if you have exciting variable application to develop, co-develop with us. Thank you. So Singapore is doing a lot of cool tech. Yes. Cool technology. Like world-leading technology stuff. It's a question, yeah? Yes. Of course. Singapore has been a smart nation. We will be the ideal place to test-bate your technology. The ideal place to validate our value proposition and use it as a stage to leapfrog to the international stage. Cool. Thank you.


Would you like to know my best advice I ever got from a woman? What's the best advice you got from a woman? I remember, God, I think I was like 26 years old. This girl said to me, I got to get the phrasing right. She, oh, she said, wrong hole.  There you go.  Woo! Damn! Shout out Larsa Pippin.