Good afternoon, everyone. Tadev here is going to talk to us about Debian org. So thank you, Tadev. Thank you. I figured I'd start by actually putting the number, which you all have memorized by now, I hope, on my slides, so that both so it's public, but also so that if anybody wants to do key signing, they know that that's my hash at least, so come compare. Also, I'd like to congratulate another team member of DSA today, Peter Paltrow-Prater. Visible, just passed his examination for PhD, which is pretty awesome. OK, with that, let's jump into dev.dev.dev.dev.org. Let's see. So what's the problem that we're actually trying to solve with dev.dev.dev.dev.dev. Well, and it's actually bigger than just dev.dev.dev.dev. It's a large mission of what Debian is trying to do. It's we're trying to distribute software to users, because it's great to run the software on our own machines, but it's even better if somebody can actually use it. Our current way of distributing software isn't very resource-affected. We push out a lot of binary, which nobody is ever going to see our use. This is because the way the current mirroring network works, we push out all the binaries for all the architectures, even though nobody might ever download that binary off that particular mirror. This becomes even worse now that we're adding debug packages, we're adding ports, we have large data packages, and this is problematic, because bandwidth isn't free, storage isn't free. In some cases, we have mirrors who stop being able to mirror architectures or mirror WN at all just because it grows too large. In addition, those existing solutions, they have a couple of problems with them, which are hard, they're either vis-à-visible or they're hard to detect.  If you run this from a script and you detect errors, your script has just broken. After-date mirrors are much harder to detect. You run after-get update and try to upgrade and three days pause and you see no updates, and you don't actually know whether that's because there hasn't been any updates, or because just the mirror is broken. That wasn't supposed to happen. Clearly, it is called for another standard, and in the grand tradition of how we fix this, we do add another standard, which is where WN calls content. It propends to be a complete mirror of almost all of our packages. I'm saying pretend because it's not actually true. Almost all, because it doesn't include Snapshot, it doesn't include the archive.wn.org, which is the historical archive of packages, and also it isn't actually a mirror. It's a CDN, a distribution network, which sits in front of FTP.wn.org, the ports and all of that, and whenever somebody requests something from it, it downloads it from the upstream, and I'm going to show the design in a second. The primary motivations were to fix the user problems that we were seeing, partly because DSA is one of the users of, we run a lot of machines, and we use Tevion obviously, and we were seeing those problems and they were annoying. But in addition, we had a couple of other motivations. I wanted it to be a kind of one-stop shop. If you look at the traditional systems, there is no single place to download all our packages. You find some on Snapshot, you find the port archive is somewhere else, the security mirror is somewhere else again, and so on. And it's slightly bad motivation, but I used to work for Fastly, who runs the CDN. And I had a tool and I wanted to see what kind of stuff I could do with that tool. So, the design. I hope you can actually read this. We have a client running after usually, it can be, it's HTTP, all of this is HTTP. It hits the local CDN node wherever that is. We're currently using Fastly, they have machines and pops all around the world, almost true, they don't have anything really in Africa, which is slightly annoying from here, but they have a bunch in North America, they have some in South America, they have five I think in Australia, Asia, Singapore, Tokyo, various places in Europe. So, you hit whatever node is closest to you, that again goes to a secondary node, which you never actually see, which is located close by to the upstream. And the reason for this, and not force having this local CDN node go directly to the back end, is that between those two, there are links maintained, which means that these are pre-scaled, so you don't have to open a new TCP connection for that. You only have to do the TCP three-way handshake here to a fairly local node, you don't have to do it all the way to security WNORG, which might be, you know, in Europe or in the US or somewhere. So it means that even though you're going through a secondary node, it might actually still be forced to go indirectly. The paths here for where the various archives are exposed are different, so there's logic here, which says that slash-dabian goes to FTP-dabian-org, slash-dabian-ports go to ports, and so on for the security and debug archives as well. It's kind of a pretty simple setup, and mostly works pretty well. We didn't really encounter that many problems, though of course we encountered some. The reason I'm listing security-dabian-org there is that when you do, when you run a CDN, the way you send traffic to a CDN is you do a C name to a name provided by them. Now for those of you who know DNS fairly well, you know that you can't combine C names with other kinds of records. You can't have an A record and a C name, you can't have an MX record in the C name, and security-dabian-org receives email. So what we managed to do is we got the app author to add support for looking at a server record, which is a way for where you can add service records for a given domain so you can say that the SIP server for Dabian-org is this machine over there. You don't actually have to look, so it's a more generalized version of MX records where you can expose any kind of service. So apt in Weezy, no sorry not Weezy, stretch, and your support says another problem we had is that the UI of managing this is really cumbersome. Fastly provides an API, so we fixed that by writing code to integrate with that. The third one is because this is infrastructure run by somebody else, it's inherently non-free, right? It's a service somebody provides, it's not, we can run, there's nothing here which means that we couldn't run, and the hard bit here is not actually in the software. It's that you need to have machines all around the world, you need to maintain this, you need to have agreements on peering with providers and all that. And that's a business is in a much, much better position to do that than we are. IPv6 support used to be a problem. Fastly's IPv6 support has improved a lot over the last month so it's certainly getting there. And on the non-free infrastructure we're actually also doing this for DNS already. And most of the time this doesn't, it's one of those, yes it would be nice if everything was free and you know, but I'm not entirely sure how an open source model would work for infrastructure. So right now it's working, it's kind of in testing mode, they haven't denounced it that widely before. But there's obviously some future. You can't really see that very well, can you? There's no way to turn down the front lights, is there? Okay. Yeah, so this is a rocket for those of you who can't see it. It hopefully isn't actually the future because this is a missile from the only it's an air and space museum just outside of Phoenix, Arizona. So it's from the 1960s sometime. We might, maybe we might replace HTTP reader and the mirror network eventually. That's not an explicit goal here because there's no namespace conflict between dev.dev.nog and the mirror network or any other kind of mirroring service. If it gets popular then the alternatives will go away, but if they do then they go away. If they don't then we'll have multiple services. It currently doesn't support TLS. It's something which is really, really hard to do on the traditional mirror network because we don't have a way of distributing the certificates and the keys to all the various mirrors in a good way. That's much easier with CDN because you're dealing with a single entity on the other side rather than however many mirror operators we have. I also want to add support for snapshot and archive. But who knows when that comes. It's not really that hard. It just needs doing like so many other things. It's possible to contribute. I'm working on getting it so that it's actually married on Alliot. It currently isn't. I think at least all DDs have access to cloning that URL. But it's going to be on Alliot's fellow suit. It's a small script which contains configuration for that controls the setup.  How to use it? It's pretty easy to just point your app repository at .wnoog instead of fdp uk.wnoog or whatever. Because it mirrors you can also go there with the browser and it will tell you what you can use for I think we include the information on the other for security and so on. It's Dabian security, Dabian port. Thanks to Fathley who sponsored the service and thanks to Fenn who are my new employer as of a month ago for actually sending me here and paying my airfare. It's almost it. Let me see. I was planning on doing a live demo where let's see get out of the pier but we can do that go away. So let's see if we can do it like that. I'm running out to get in a loop because it doesn't have very much traffic right now. We actually get live stats on how much traffic it's doing. We also have let's see there that has the historical data on how much traffic we're seeing. We have a little bit of traffic. We're mostly using it for for Dabian. It's public, it's sponsored so it doesn't cost us anything and I'd love to see more people use it. Feedback is always welcome both if it works well or if you have questions or it doesn't work well. Questions etc. Thank you. On the slide you mentioned the support for stretch you only mentioned main so I'm just checking back again to see if the other components are supported or not. They are. It supports everything. It doesn't actually have any knowledge of what the archive layout inside of that particular URL is. Whenever FTP master adds bullseye it's automatically going to be supported and all that. Thank you. During the DSA presentation you talked about using different CDNs but you've only talked about Fastly here so I was wondering if you were planning on using order services or what happens when Fastly just dies? If Fastly dies we'll use something else obviously because we want to so we're not using any of the advanced features of Fastly so I'm not particularly worried about being able to transition somewhere else the reason or about most of the reason I haven't done so already and set it up on others is that as an employee of Fastly it would be okay for me to go and sign up to other CDN providers. Other DSA members have more experience with the other alternatives and we already have sponsors from Planet I think runs on next CDN for instance and we have offers from others as well so while it currently runs on Fastly there's very very little which binds this to Fastly. And Africa support? I can I don't actually mean to advertise so much for Fastly but let's see if we can make that Yeah, so they have a list of planned locations I believe that either Cape Han or Johannes Brogis on that list I do not know what the timing of that is but you know That's a question Hi, thanks for the presentation I guess too many one question I want to remark one question is do you have a preference for this replacing the mirror infrastructure I actually kind of do even though I used to run a mirror and one comment is you mentioned that HTTPS is difficult in the mirror infrastructure world because you can't really give the private keys to like a bajillion mirrors but if you use HTTP reader then they can redirect to a hostname they do control and have their own cert That's correct HTTP reader has the problem that it's somewhat sporadically sporadically maintained so yes, if we had more HTTPS mirrors we could do it that way for all we can do it with a single single provider like this I would not be unhappy if the mirroring infrastructure was replaced with something like that but on the other hand I'm not going to go out there and tell people that stop working on mirrors if you enjoy working on mirrors so it's one of those if it turns out that way it won't make me sad but I'm also not going to go around telling people don't work on this thing you enjoy working on but if mirrors start bit rotting then they're going to go away We are out of time so thank you very much .f for your presentation Thank you