I think we can start. Yeah, perfect. So hello, everyone. Thanks for joining me this morning. I hope you're having a great time so far. And today I'm going to talk about Agro-Rollertat scale, or how we brought automated rollbacks to more than 2,100 microservices at Monzo. So a little thing about me and why I'm doing this talk here in the first place. So I'm Joseph Paramedici, a picture of me with much more air. I'm a backend engineer working in the platform team at Monzo. I led this migration project where we essentially replace our default deployment strategy to basically use Agro-Rollertat and automated rollback canering by default. And I was a tech lead of the engineering effectiveness team, as we had a couple of discussions this morning during the keynote. So the basic team doing everything DevX, all the tooling that's engineer used to interface with the platform, and also deployment pipeline CICD and observability stack. And you can probably hear already, I have a strong accent. And unfortunately, I cannot do anything about it. So please bear with me. So first of all, what I'm going to talk about this morning. This is a study case of how we migrated all of our services to Agro-Rollertat. As I said, it could become like the default deployment strategy at Monzo. Or we did it safely because we regulated bank in the UK. And the name of the bank is Monzo. And that we are actually quite happy with what we got and what we achieved with Agro-Rollertat. And there will be a lot of emojis because it's part of Monzo branding. So yeah, a little bit of scene setting, a little bit of context. So what is Monzo? It's a regulated bank in the UK. It's seven years old. It has more than 5 million customers. It's one best British bank this year, I believe. It is mobile only banking app. As you can see, what the basic app look like. And if you've been in the UK, you've probably seen those very flashy hot coral cards that we're famous for. And that basically what they look like. From the tech point of view, which is the tech stack point of view, the more interesting bit, is Monzo really brought into the microservices philosophy. We have more than 2,100 microservices of our application, depending on what you call them. We have over 200 engineers, most of them back-end engineer. And that's where the interesting number come in. We did last year 27,000 prod deployments. And those are actual prod deployments. This is not conflict change. This is actual code that we push to production. Everything is going. And the other interesting bit is everything is running in one mega-cubanetic cluster of like 300 nodes, 20,000 pods, massive scale. It's a really tricky community cluster. And our entire platform is quite centralized and quite homogeneous. Every golden services look the same and basically use the same libraries. So as I mentioned, basically deploying at Monzo is a better than better. We made that process like incredibly efficient, like as efficient as possible for engineers to be able to deploy. And basically our deployment pipeline is unruled. And you can deploy in less than two minutes. And that's including building the images. So it's very efficient. But we kind of realized that it's something we do so often on a daily basis. Our deployment strategy was kind of simple. We still rely on community native deployment and use a rolling-up-date strategy, which basically means that the engineer, when you deploy stuff, because we do it so much, we are like alerting, genocadeting, and also business-level alert. But engineer kind of had to babysit deployments, look at Grafana dashboard, which is, as you know, not super interesting. And also, with our current setup, every time you do a new release, you basically release it directly to 100% of the customers. As I mentioned, banking is a very good industry. Dine time is very costly financially and in terms of reputation. I mean, for all of us in this room, you don't definitely do not want downtime. But for us, it's also like the risk, some regulatory risk. If we're down too often, or if some very important payments system are down for long enough time, that might actually trigger some regulatory action. And those are quite unsavory. We kind of want to avoid that. So in order to do this, we kind of try to stick to best practices. And we have this Swiss cheese approach to change management. So we try to catch bad changes. Bad changes are narrow at multiple layer. Could review to our CI pipeline having a staging environment that is actually quite close to the production environment, didn't get alerted. But we kind of realized that we did not add anything after you deploy.  So we kind of want to implement our deployment safer. We wanted to reduce the connectivity of deploying and also to allow for more automation, such as continuous deployments. In short, that can make deploying safer and quicker. So we're onboarding this progressive volat initiative. And the big idea at the beginning where to move gradually, move traffic gradually to a new version of the code. And also automatically rolling back a bad change if that happened. I need to do a bit of introduction about Agorollout. So this is stolen from the project documentation. So Agorollout is a Kubernetes controller and set of CID which provide advanced deployment capabilities, such as Bruggrin and Canary deployment. And what Agorollout more or less is, it can be used as a replacement of Kubernetes native deployment objects that basically essentially provide more feature and those advanced deployment capabilities, such as doing staged rollout through Canary and things like that. But in essence, it can be used either as a replacement for Kubernetes native deployment or alongside them by essentially referencing an existing deployment object. So this slide is a bit like there's a lot of things going on. What I really wanted to convey here is you have the volat controller that basically manages a different volat object. So the central thing of Agorollout are the volat objects. Like I say, it can be used as a replacement for Kubernetes native deployment. That's where you can define your pod template spec, annotations, but also where you define what the deployment strategy should be like. Like for example, canary, shift traffic 10% for the first five minutes and then 100% after that. And that's where you define all those things. That's also where you can define some analysis. So an analysis is essentially a condition using some metric provider, some metric provider, in our case, primitives. Or you cannot define rules, such as a rollback if that metric is above a certain threshold for a couple of times, this amount of time, and so on and so on. So in essence, that's basically what Agorollout look like. If you've been to the workshop yesterday, you probably played with it, and you potentially played with it a little bit. And that basically all there is to it, which is great. So great and simple tool. And what did we decided to choose Agorollout in the first place is because it was easy to understand and experiment with, very well maintained and well documented. And it kind of adds this plug and play quality to it as you can just install it alongside your existing cluster. Oculus has a very big, very complex, with a lot of business logic at the community level, especially everything related to security. We have a custom network isolation capability built into the community layer at Monzo. But even through all of that, you can just easily add Agorollout. And you don't need much requirements for it to be working fine. It works better if you have a service mesh that allows you to do final-grained traffic shaping. Yeah, perfect. Final-grained traffic shaping. But even if you don't have a service mesh, it still work. Traffic shaping still work. It's just a bit more like calls. And it's highly customizable. You can basically define custom rule for each of your rollouts for each of the services, which is what we were looking for in the first place. So now we need to talk a little bit about our journey. Or we did get to automated rollback at Monzo. The first thing is a bit of a bait-and-switch. So I talk about canarying a bit. So canarying is a canary deployment is when you shift traffic progressively to a new version of an app. And that's what we're looking for in the first place. But then we ask ourselves the question, is that really what we want? Is that really what we needed? Because we're going from a really simple process of engineering and not having to think about things that could be rolled back. You only have a couple of assumptions. You only have one version of the code running in the cluster, and so on, and so on. So something much more complex. And there was a clear need to educate engineers of the org and of the consequential tooling. And we're like, is that really what we want? Is that really what we need right now? By the way, this illustration is made with Dali, which is a game changer when you actually need to put illustration in your presentation. And as you can see with this kind of ronky giant canary and giant cage. So reviewing canary deployment where actually what we care is reducing the impact of a bad change. And what we're really actually looking for is automated rollbacks. So that's what we decided to implement first. We're kind of using a blue-green-like strategy at the moment. It's still use our go canary strategy, except you still go straight to 100% of the shift, the new version, to 100% of the traffic. But you have this analysis period where you basically run this check to know if you need to rollback or not. And we did that to basically allow for us to move to real canary in the future because we got most of the infrastructure and most of the work in place to be able to do that. Basically going instead of going from a rolling updates strategy to just having automated rollback and later on move to canarying. And this is what's the deployment workflow look in this new world. It's essentially an ingenious target deployment that updates the relative objects, the new version of the app become live. Then for a short period of time, there is this analysis step where the different rules, like we call them rollback rules, are evaluated. And if it's successful, the engine is notified of success. We do that in Slack because we also like, we use basically a lot of Slack Slack ups. We have a lot of processes in Slack. And if it's not, if there is one of the rules actually evaluated to false, we automatically rollback to previous version where we notify the engineer and the team. This workflow allows us to choose a strategy, the deployment strategy, per service and per deployment. So for each individual deployment that you make, that allow you to potentially change what type of deployment do you want? Do you want automated rollback? Do you want canarying? Do you want something more fancy in the future? And that's what this flow allows us to do. Then again, this is probably not super interesting right now. It's more like if you're reviewing this talk later on and you want to look more about the details. This is what it looks like from an architecture point of view. So I don't mention everything here. Our deployment pipeline is enrolled. So we have a service that basically generates a manifest and so you say engineer use CLI to link all cheaper to trigger the deployment that basically make one of the service create a new manifest for it, put that in a GPO and then this is applied to another like services, another service that applies that to Kubernetes and updates the world object. And then creates a new replica set and so on. So basically this is just to prove that we actually use GitOps. And yeah, there will be a complimentary blog post about this talk with like all the nice gritty technical details about all the aspect of the work that we did for all of you if you actually love the animal. So that's what the tooling is a look like. So at Monzo we have this philosophy of backend engineers should not know about the platform. They should be able to interact with it but they should not actually need to know things about Kubernetes like networking or even auto scanning. So we have this one tool called Shipper to deploy and the president super interesting here again here but what we just wanted to show you is you basically just deploy by calling Shipper, giving it a commit hash, a number of service and deploy it for you. And we basically completely abstracted away our goal out. Our goal out itself provide like great tooling and great dashboard but we decided not to like expose that to our engineer and basically abstract all of that away which basically required us to required us to integrate with our goal out in a slightly different way. Same thing here that basically what our slack notification look like when you start the deployment basically tell you how you started the deployment with this strategy. Here's dashboard to end the logs if you want to see what's going on and the notification kind of success of successful deployment or like failed deployments. So I talk about rules and rollback rules. So we actually started with only one rule and we try to keep our rule as simple and generic as possible at the moment all the services in our back end use the same rollback rules. So they have to be like quite generic and we started basically with if the total error rate of a service is above a certain threshold then you need to rollback and about the rule themselves we decided to basically tweak them and add them based on our incident handling process. So basically every time we have an incident or something that look like an incident we have to have the question should that be caught at deployment time or could that have been caught at deployment time? And if yes, would have we made a rule to actually catch that issue? So issue is like rules when you come to like honoring an automated rollback is a false positive is annoying for your engineers because the change our rollback so deployment is rollback. Customers should not seem much problem hopefully but it's really annoying and it's not great gross UX and you don't definitely want to avoid like a false negative where you're not rolling back where you actually should. So the takeaway here is basically if you're planning to move your entire organization to something more complex like our rollout do it in step start as simple as possible. You don't need to go from like a simple process to something like state of the art who's like very complex canary strategy just start with what actually matter for your org and in our case that was automated rollbacks. Argo is relatively easy to integrate as you can see with our custom tooling because it provide a lot of things out of the box like it will trace notification on the line as a Argo rollout as a Argo controller like metrics like the rollout metrics and it's CLI and dashboard are quite useful when you're like in development phase in the development phase. Now I'm going to talk a little bit about how do you actually migrate to Argo rollout? And as I mentioned before, Argo rollout allow you to do it in an incremental fashion where the rollout objects essentially reference an existing deployment. So the rollout objects essentially define the strategy and a couple of other parameters but the potential aspect and everything related to the actual deployment is still on the deployment object and the rollout just reference it. And this allow to basically migrate to Argo rollout in a safe manner that avoid downtime because you essentially just bring a new rollout object that reference the deployment object. At that point you have two replicas sets one managed by the deployment resource one managed by the rollout resource and then you can safely decide to set the number of replicas of the deployment to zero. So you just have like a replica set put managed by the rollout object and doing this at all for like a no downtime migration which is great. And also doing it this way allow you to easily reverse the process if you don't want rollout again or you're just in an experimental phase. So that is great. That is super easy to do. Cube CTL command probably like two, three minutes relatively safe to do except as I mentioned earlier we have 2,100 application. So we need to do that a lot more which kind of make us actually there was a clear need to up to the automated process and through this automation we also decided to like on soft to enforce even more safety because as I mentioned earlier we are regulated bank who cannot really allow downtime and also part of our like company motto making money work for everyone and there is nothing more like scary for our customers than having that card not working at the supermarket because you know produce down. So we wanted to make sure like we extra safe and that the whole process is repeatable, observable and in the patterns. So that's what basically what the migration look like for us is we first notify the service owner on stock we create a new rollout object for that savvy that we're migrating we freeze the deployments because we want to limit the amount of change we'll be doing this process which is not very long it's like on average two, three minutes per service and then we scale up the rollouts and about three of the rollouts if your service is an HPA will you do the migration we wanted to make sure that we scale up the rollout to the max band of the HPA just to be extra sure that if there is like a surge of loads will we do the migration there's not gonna be any problem and then we check if all the pod that we're supposed to see are actually alive and well in our cluster and only then we decide to scale down the deployment to zero replica because we know that we have enough rollout replica managed by the rollout object and we then check that we don't have any deployment pods managed by the deployment object left and then if the service is an HPA we switch the target of the HPA from the deployment to the rollout and for the deployment notify people of success or not of success that is great so that basically what the process look like and we decided to like execute on it by going before like a migrator pattern where essentially all the step I just described are independent because that migrator pattern that migrator service that we're using will retry all the steps and that's weird about and also like allow for like fine grained scheduling we had so much services to migrate we only wanted to do them like during business hours we want to spread the migration across like the entire day because we want to avoid adding too much pattern in a short amount of time for example and just you know to basically control the amount of risk and we did all that through like immigration service where the user decided which service to migrate and then we had a cron every day that triggers immigration for those services and spread them out through the day we did it by using your service tiering so we went from like the lowest tier services to the highest tier services to the less important service to the most important service and by the time we actually got to migrate our tier services including the ledger service which is not a big deal like just managing like four million quid so you want that won't not be down because if it's down all the payments all the payment system are essentially down by the time we got to those tier services we already immigrated with that process more than 1,700 services so we were quite confident about it being quite safe so take away if make it safe, automate it like if you automate it you can enforce some additional safety rules obviously if you have like 10 or 20 application you can still do it by hand, it's probably much quicker but as if your backend is big enough that might be worth actually adding some automation the migration is actually what took the most time the act of migrating is actually what took the most time because we had to do it slowly and carefully so anywhere to invest in tooling and monitoring to make sure that we could do this process safely so in terms of like deployment probably like half the time was just actually running the migration after we developed everything so what did we learn like moving to our go-rollout and running it at this scale first of all some numbers because everyone like numbers so we have in our production cluster right now 2,143 rollouts so 2,143 services running in production probably a couple more since I did the slide like a week ago we have seven rollback rules that cover some things that are very specific to us but are generally enough to catch large classes of problem and we've been running a go-rollout in production for essentially every services for like four months now which mean we did 7,000 production deployments with our go-rollout and 22,000 production deployments in staging because I told you we did 27,000 production deployment per year but the truth is we did like more than 100K deployment staging last year so yeah we have like we built like the confidence that our go-rollout is actually capable of operating at this scale and we are like 63 rollbacks since we basically start using our go-rollout so a little bit like less than 1% a couple of them are like false positive that's why there is a need to clean it to like tweak your rules so what about operating rollout at scale which is like the whole point of this presentation right actually there was very little thing for us to do it just kind of scaled really well with our kind of like unconventional architecture like it's very unlikely that we have like one Kubernetes cluster that big and that many rollouts running but the go-rollout controller did not have an issue with our scale of like operation at Monzo which was quite surprising we even had a small incident in staging where we did like 5,000 deployments over the course of like three hours and it worked which is great all the stuff broke like primitives but because of the putram but other than that it actually worked quite well at like this very large scale there's a clear need to like share the knowledge about our go-rollout because it becomes a new critical component that your ops and platform team need to understand quite well and as well as the rest of the organization that need to understand that the process is a bit different now there was still one little performance issue about leftover like analysis for an object but we're on the case and we hope to be able to push a PR like soonish to actually fix that issue and yeah and it's important to invest into our lighting and observability for when operating our go-rollout in general and especially the scale and the project make it easy because it already provide like most of the metric you can think of that you would want to monitor would we do it again and should you the answer to that first part of the question is yes and the answer and for the other question, yes as well so for us it was a clear positive outcome we reduced the connective load of like deploying we catch loads of issues across like tons of deployments we're quite happy with what we got in return and also with how easy it was to actually set it up in given the amount of customization and you know like business specific stuff that is running in our cluster and it's a nice addition to our like Swiss cheese approach like deep and deep strategy to control for bad changes and the most important thing is like engineers actually love it so this is one feedback that we got from backend engineer that's basically got paid for an issue due to deployment and was automatically rolled back in I think less than a minute and that would have been like a rather nasty nasty problem with basically the MasterCard 3ds processor on our end that would have rejected like every request which basically means that a customer would not be able to pay with a Monzo card online for the duration of that incident but that was automatically reverted within one minute and yeah, thanks you very much for listening to me we have some time for like a few questions if you have any and like I mentioned earlier like we will do like a complimentary block pass with all the actual technical detail of how we enabled the different aspect of this migration and our setup and if you have any question in general about the gritty details please grab me after this, grab me during the conference any questions? Yes. I got the essence of what you were asking about so how about the migration process when we scale up and down the difference? Exactly, yeah, so when you're scaling up and down the deployment versus rollout I think sometimes what I've heard is that there can be some kind of cost rep precautions because either you have two workloads that are both scaled up fully or you have to have downtime to scale one down to roll out the rollout did you guys, was that ever a conversation in terms of that process especially with the scale that you guys were rolling out these microservices at? Sorry, so the question is like was it a problem for us in general? Yeah, I guess was it a topic of conversation like a concern about maybe like lack of resources in duplicating these workloads or something? So we kind of like didn't add that much of the problem because due to the nature of what we are like a bank, we basically run with a lot of overhead in our cluster so we had like a load of like capacity leftover like the whole point of doing this migration of like spreading out the migration across like Zontair Day is to make sure like to control for those kind of problem where yeah sure you're gonna run double the workload for like a short amount of time for a couple of minutes and if you spread them out across the day for example for low tier services we did batch of 200 like spread out over like eight hours which basically mean you have like one migration every five, 10 minutes and the migration process itself only lasts for like two, three minutes. So that's how we control for it and that's why we decided to go slow to be extra safe but yeah indeed that's something to take care of because you don't, and that's also the whole point about doing this migrator, having this migrator pattern and checking that we are the right number of parts that we're expecting because for very, very large deployment with like 500 pods sometimes you would take like a couple of future eyes to it would take time for pods to actually come up due to scheduling pressure. So yeah that's why we, yeah. Okay, thank you. 17 second, any question? Okay, cool, perfect. Well, thank you very much. Thank you very much. And.


Good morning guys and welcome back to our Guinness World Record attempt to become the youngest pair to circumnavigate the globe by motorcycle. We're here in the city of Turkestan in Kazakhstan and we're up early this morning. The sun is just rising above the buildings behind us because it's going to get really hot today. Yesterday we had time to explore this amazing city here but today we've got to make some distance north into Kazakhstan. So let me show you guys the route for today. So we are here and today we're aiming for a big day. We want to make it past Kislauda all the way up here to a road house somewhere around here. It's about 380 miles and Google Maps says it will take us 7 hours. It's already 7.30 so better hit the road let's go. Okay already you can feel the sun right? When it hits you you can just feel the heat from it already it's crazy. Man it's going to be a hot day today. Yeah I can tell you that I was dead yesterday after our city tour. I was just knocked out. Yeah we weren't even wearing our motorcycle gear I think it reached 40 degrees yesterday. Hence why we set the alarm early this morning to try and get a little bit of cool riding in the morning. But I'm really positive surprised by this town here. They put a lot of effort in it to make it really nice. Yeah that central area around where the mausoleum is it's just beautiful. It's just beautiful. It's this massive open green space full of parks and monuments. I mean Turkestan is one of the top 10 most visited places in Kazakhstan. You can see why. Beautiful place. You speak English? Yes, yes. Oh cool. Where you learn English? Okay okay okay. Okay. Three, two, one. Cool. Motorcyclist? Yeah, Yamaha. Yeah cool. Yamaha, Suzuki, all under. Kitayski. Kitayski. Kitayski. Kitayski. Japan. Sisi. Novo in Shimkent. Shimkent, yes. This one and this one. This one, Novo in Almaty. Almaty, yes. Anglia, France, Spain, Africa. Africa. Brazil, America. America, Canada. Canada, Korea. Korea, Vladivostok, Russia, Mongolia, Russia. Kazakhstan, Kyrgyzstan. Kazakhstan. Yeah, finish. Finish. It's strong. What have you got for us? Cappuccino. Cappuccino, lovely. Look at this. We needed to stock up on some water. And we needed some breakfast. So apparently here is cabbage inside. Okay. And I don't know what's inside here. Okay. Let's give it a go. All right. Some undefined pastries. I love it. Yes. They stopped us. Okay. Hello. Hello. Hi. Good day. Good day. Thank you for this. Okay. This is me. Yes, yes, yes. Good day. Here. This is Lorda. And... Aiteki bee. Yeah. Ah, okay. Okay. Okay. Me sitting. Okay. Okay. You have a shade there. You have a shade this much. Oh. How are you? How are you? Thank you. Photo. Ah, photo? What? Rahmet. Rahmet. Rahmet. Okay. Bye bye. Okay. That was a bit strange, wasn't it? That was funny, yeah. He was like... He just wanted like a picture for evidence that he was like saying something to us or something. Yeah, I think so too. I don't know because he was like... He didn't really say anything. He was like taking a picture and he was just like pointing like da-da-da-da-da. Yeah, yeah. No. There you go. Pleasant police stop in Kazakhstan. Yes. Nice. So we just crossed the east-west time zone boundary in Kazakhstan. Actually it's interesting because Kazakhstan actually spans four time zones. But the country just decided on two. I guess to make it more simple, they just have east Kazakhstan time and west Kazakhstan time. And the boundary we just passed put us now only four hours difference from the UK. Yeah, I mean this is as well a really interesting point about travelling the world to cross all the time zones. You know, normally when you take a plane, I don't know, from England to Australia, you just skip all the time zones because we are riding every time zone back to England. It's always like, oh, we got one hour more. Oh, we got one hour more. Now it's instead of 8.50, it's 7.50. Oh, that means we woke up really early this morning. That means we woke up at like five o'clock this morning. But yeah, just four more time zone boundaries to go and we'll be home. Yo! Oh, a whole bunch of camels right next to the road. Wow. Amazing. Cool. Hello, guys. Hey, guys. They are huge. Wow. There he is. Hello. I mean, we have seen some camels, but that big. They are some pretty incredible animals. Hey. Wow. And so many as well. That's awesome. But I'm confused about them now because some of them have two humps and some of them have one. Like this definitely has one hump. Look. Yeah. What about this one? Hello. As well just one. Yeah, this has one hump. But some of the ones I saw back there had two. So maybe they're just mixing dromedaries and bacteria and camels together. Such beautiful animals. Totally. Oh, we made him a bit scared. Okay. See you later, guys. Cow in the road. Oh, man. What is he doing there? No idea. He's not even with a herd. He's just on his own. A little bit of a rest up here. Stretch our legs. We decided that as we're riding just under 400 miles, we're going to take a break every 100 miles. So we've done our first 100 miles and it's only 9am in this time zone. Yeah. That's going to be a new record. Yes. There's like a road house here and we got some pomegranate, carbonated drink. I don't know. Never had it before, but give it a go. And some crisps. I've never seen this flavor of crisps before. It's like, I don't even know what that is. It's like, I don't know, lamb and potato flavor or something. I don't know. We'll give it a go. What is it? It tastes like lamb. Is it lamb? It tastes like lamb. For me, it does. They're quite meaty, aren't they? We're just throwing away our vegetarianism now here in Kazakhstan. Yeah, completely. Bye bye. We've got a whole massive herd of camels right on the side of the highway. Look at this. Wow. They are big. Wow, and it's crazy that they are just like right on the edge of this highway. I mean, this highway has a speed limit of 110 and you've got like giant trucks flying along and then you've just got a whole herd of camel wandering down the side. That's mental. That's crazy. So we're on break number two of the day, 200 miles down and it's not even midday. Yeah. But our navigation says it's still three hours, 45 minutes to our destination. The road is quite easy, but it's really fast paced and it's very hot as well. So I think we will get there, but it's not the most comfortable, let's say. Hello, America. So the highway is turned into a single track highway now and there are just loads and loads and loads of trucks, a huge queue. And it's really funny because 90% of the trucks are actually all trucks from Germany. So I can see like really old advertising on there. This one is Schreiner, Schreiner Transporter. Probably when they fail their MOT, they probably just like sell them to Kazakhstan. Yeah. Okay. Yeah, we've got a lot of overtaking to do. Let's do it. Man to the front. So we're just having our break and we're just seeing this massive line of trucks that we overtook. Well, they've all overtaken us again. Look at them. One, two, three, four, five, six, seven, eight, nine, 10, 11 trucks. That looks so fun, hey? Yeah. Another overtaking session, hey? Well, we've done pretty well so far. This is break number three. We've covered just under 300 miles, 291 miles and local time is only 220. I don't think we've ever covered nearly 300 miles by two o'clock. Never ever. That's crazy. Yeah, it's always a surprise how other people can ride like 700 or even a thousand miles a day. I have no idea how they do it. Yeah, I know. It's impossible. Yeah. I mean, we're close to 300. I'm already like super tired. I'm already like, oh my God. Anyway, I think we can manage one more leg, 100 miles. Yeah. And then done for the day. Let's do it. 38,999.9. We just hit 39,000 miles on the road. I hope a petrol station is coming soon because we are dangerously low on fuel now. I passed the last one like 50 miles ago and I was like, oh, it really hasn't been very much at all out here. It's like almost a desert. So now the bike's saying that we've got about time for fuel. Is this a petrol station? Yeah, I think so. Maybe it is. Gasolina. Oh, that's amazing. Oh, OK. 10 km or? 5 km. 5 km. OK. Soziva. OK. Yeah. I can see something on the map here coming up. OK. 10 km is not too bad. We can probably make that. OK, cool. The estimation is that I've got 27 miles in the tank. OK. So absolutely plenty. Yeah. Oh, look. Here's some more camels. Look, a whole herd of them. Oh, nice. Wow. That's incredible. Honestly, I just find that crazy that they can just chill there like that. OK. We might be onto something here. Yes, this one looks like he's got some cars in front. Looks promising. Oh, yes. I think we're golden. Yo. Look at that. Nothing to worry about. A cool 25 miles left in the tank. Stressing for nothing. We don't know if they got fuel yet though. It's true. So look, 92. OK. Yeah. Black in the UK is always diesel. OK. So I'm always scared. But it says 92. OK. Yes. We have petrol. Awesome. Look at that. Stopped it on 3,200 dead. Oh, I'm quite proud of that. Yes. Yes. Back in business. Yes. He was just asking to put a sticker on his door. OK. So if you want to do that. Yeah. Let's put a happy and alive sticker on his door. Hello. Hi. Oh, hi. Nice to meet you. Yes, on the door. Cool. Look at this. Nice. Hey, horizon's unlimited. This is cool. Long way back. All bikers are brothers. I like it. Me, Ollie. Her, Lavi. Here. OK. Lavi and Ollie happy and alive going on the door of fame. Rock me. Rock me. Rock me. OK. Bye bye. Cool. Finally reaching the place where we're going to try and stay for the night. Nice. Let's pray that they have a room for us available. Yeah. That's the other thing, isn't it? Because there isn't many options around, to be honest. It's mostly just a desert. I know that they have like camping here, I think as well. In case we can't get a room. Maybe we could camp. OK. Yeah. Let's check it out, hey. Yeah. This is like all in one center. So hotel, cafe, restaurant, all kinds of things. Probably we have to go to the reception, don't we? OK. I guess. Yeah. I am tired after that many hours, I'll tell you. I can imagine. Oh my God. Let's check it out. Yep. So I just spoke to the guy. He's really, really friendly. And we will just pack up the bike behind the building. He gave me the room keys. And the room was only 10 pounds. That's pretty good. Incredible. Nice. I saw the room already. Nice air condition. Yes. Beautiful. And there's a restaurant inside? Yes. Nice. OK.  The green gates inside here. Perfect. Rahmet. What's your name? Daribek. Daribek. There you go. Home sweet home, hey. Yeah. Nice, hey. I really like the design. Look at that. There's rooms in Kazakhstan, there's something else, hey. I mean, they're always nice and clean and beautiful. I just feel so homely. The details are nice. Yeah, it's pretty cozy. 10 pounds. 10 pounds. 10 pounds. So that was a pretty hectic ride today. Very, very long. I think we rode 390 miles pretty much the whole day. I'm really exhausted. I'm really tired and I'm really happy to be here. Happy to life. Yay. So we will just pack up Bumblebee now and then we have a nice rest. We will have a nice dinner here in the restaurant. And that's it from us today. We hope you enjoyed the video. If so, please give us a thumbs up. Subscribe to the channel. Share the video with your friends and family. Comment below and we will see you next time. I don't know what you're supposed to do. Just eat, I guess.


I am just really, really delighted to be able to introduce Julie to you. Julie Brill is one of the commissioners at the Federal Trade Commission. She has held office there, I believe, for about five years now, which is a lot of time and a lot of hard work on our behalf. During her time there, the FTC has done some really, really illuminating and thoughtful work on issues surrounding privacy and data brokerage and things like that. That, of course, is just one piece of an enormous portfolio that the FTC has. I'm really grateful that she has taken some time out of her very, very busy schedule to come and share some thoughts with us here. She's going to talk for a while, and then she tells me she will be happy to field some questions. Without further ado, you can find Bio in the web and things like that. I'm going to just invite Julie up to take the podium and share her thinking with us. Please welcome Julie Brill. Hi, everybody, and thanks Clifford for that very nice welcome and introduction. And thanks to the Center for Networked Information for inviting me to address this meeting. You know, the vast array of topics that you've covered, I've looked at their brochure, and so has my staff, and you know, it's truly inspiring. The scope of what you're looking at and thinking about highlights the profoundly beneficial effects that digital technologies can have on access to information for research, education, cultural preservation, and other scholarly endeavors. I greatly admire the work that you're doing and that you have done over the years to connect researchers and educational efforts, and its ongoing efforts to make scholarship more accessible and collaborative. So thanks for all that you're doing. Now at first glance, CNI and my agency, the Federal Trade Commission, would seem to be focused on quite different issues. Yet I think it's clear that CNI and the FTC, when you look just underneath the surface, we're wrestling with many of the same questions about the benefits and risks of an increasingly connected and data-driven society. One broad issue in which we share an interest is the Internet of Things. We are connecting nearly everything from cars and buildings to clothing and light bulbs to the Internet. Network equipment manufacturer Cisco reports that there are 25 billion networked devices in the world today and predicts that there will be 50 billion by the year 2020. These sensors, along with our smartphones, tablets, and computers, generate twice as much data today as they did two years ago, and this trend is expected to continue. Sensors that are so small and efficient that they can power themselves with ambient radio waves are becoming a reality. But the number of connected devices and the relentless accumulation of data are only part of the story. The other part of the story is big data. Data is becoming cheaper to collect and keep, and our ability to analyze it is improving. This development holds many promises. Cities can better maintain their infrastructures by developing sophisticated early warning systems for gas and water leaks. Medical researchers can enroll patients in large-scale research projects and collect streams of useful data that in the past would have been a mere trickle coming from surveys and patients' own reports. In classrooms from preschools to universities, online learning resources are enriching students' experiences and helping teachers tailor their instruction for students' abilities and interests. And online courses and the communities of students that grow up around them are making it possible for people all over the world to learn and earn degrees from the world's leading experts and most prestigious universities. Some significant risks go along with the potential benefits of connected devices and big data. As we add devices to our homes, classrooms, and clothes, much more sensitive data will be collected. User interfaces on devices will shrink or disappear, making it more difficult for consumers to know when data is being collected or to exercise any control. In fact, I expect that the Internet itself will disappear because connectivity will just be part of the way things work, as electricity is today. Now these developments pose difficult challenges for privacy, security, and fairness in our society. In sensitive settings, these challenges are particularly acute. Let's take education, for instance, where privacy violations and security breaches can cause a wide range of harms, and inaccurate or unfair data processing can have a big ripple effect on students' lives. More generally, the data that will be available as a result of these connected devices will be deeply personal, and big data analytics will make the data more readily actionable. Some of these devices will handle deeply sensitive information about our health, our homes, and our families. Some will be linked to our financial accounts, some to our email accounts, and the devices themselves will be more closely connected with our actions in the physical world, making data security and device security critically important. But some fundamental aspects of our world are not going to change. No matter how connected and data-driven we become. Most importantly, we, as individuals, will remain roughly the same. We will not suddenly become capable of keeping track of dozens or hundreds of streams of our data, peering into the depths of algorithmic decision-making engines, or spotting security flaws in the countless devices and pieces of software that surround us. Faced with a world of uncertainty about which devices are safe and whether consumers are getting a fair shake in the big data world, consumers can use some help. Now I'm optimistic that consumers will be able to navigate and benefit from this complex and uncertain and exciting world. The key, I think, is keeping our focus on the ever-arching value of trust. Trust is the flip side of risk. It is the expectation of favorable reciprocity from others in situations that are uncertain or risky. Trust does not depend on knowing everything there is to know about the person, company, or thing with which you're interacting. Instead, trust becomes an expectation in which we ask, will this person, company, or device do what I expect it to do and not something else? So there's a close connection between trust and transparency, and it's this connection between trust and transparency in the context of the Internet of Things and big data that I'd like to discuss with you today. Now before I get into that discussion, let me take a moment to first describe the role that the FTC, the Federal Trade Commission, my agency, plays in privacy, data security, and consumer protection in general. We are the nation's leading consumer protection agency, and we share competition enforcement authority with the Department of Justice. Under authority given to us under Section 5 of the Federal Trade Commission Act, the FTC is responsible for protecting consumers from a broad range of unfair or deceptive acts or practices. Now it's under that authority, that Section 5 authority, that we have brought nearly 100 privacy and data security enforcement actions. The flexibility of Section 5 and our broad authority to obtain remedies that protect consumers have allowed us to keep up with rapid changes in technology. For example, we have brought actions against companies for allegedly collecting information inappropriately from consumers' mobile devices, for making unwarranted intrusions into private spaces, for exposing health and other sensitive information, for exposing previously confidential information about individuals' networks of friends and acquaintances, and for providing sensitive information to third parties who in turn victimize consumers. We have also brought hundreds of cases vindicating consumers' rights under laws that protect sensitive information. Those are special separate laws that deal with information about children, financial information, medical data, and information used to make decisions about consumers' credit, insurance, employment, and housing. Now the FTC also maintains a pretty busy policy docket. At the beginning of this year, we published a report on the Internet of Things, which emphasizes the importance of data and device security, as well as the applicability of established privacy principles to connected devices. Before that, we published a detailed study of the data broker industry, as Clifford referred to. And frankly, as far as I'm concerned, the data broker industry has been in the big data business long before the words big data became part of our daily lexicon. We also held public workshops on so-called alternative consumer scores, and the potential for big data analytics to be used in ways that discriminate against consumers. And just last month, we held a public workshop on cross-device tracking, which refers to companies' efforts to correlate consumers' activities, any consumers' activities as she moves or as we move, from a smartphone to a tablet to a desktop computer. So let me now turn to the connection between transparency, privacy, and consumer trust. Consumers want to know, and should be able to easily find out, what information companies are collecting, where they're sending it, and how they're using it. This kind of information is important to consumers' decisions about whether to use digital products and services in the first place. But transparency is also important in situations in which consumers are not making choices about whether to interact with the company, but instead, where the company is operating behind the scenes in ways that can significantly affect consumers. Data brokers illustrate this point well. Data brokers are companies that assemble individual profiles on consumers by collecting information from far-flung sources, but typically they don't interact directly with the consumer themselves. Through these profiles, consumers can end up in marketing segments drawn along lines of race, ethnicity, financial status, health conditions, and other sensitive characteristics. With all of this data and the inferences that data brokers can draw from it, they put consumers into categories or segments that had labels such as, we talk about this in our report, and it's been talked about in other reports, financially challenged is one segment. Bible lifestyle, another one. Diabetes interest. Metro parents, and that was a list of single parents who are primarily high school or vocationally educated and are handling the stresses of urban life on a small budget. And another segment that was uncovered recently is called Timeless Traditions, a list of immigrants who speak some English but generally prefer Spanish. Now, while consumers might benefit from some of these segments by receiving more relevant advertising, for instance, consumers should have some choices about where their data ends up and how it is used. Transparency is enhanced by giving consumers what we call at the FTC just in time notices or information. That's giving them information at key moments when it is most relevant to them, such as when they're deciding whether to download an app or whether to allow that app to collect geolocation information or whether they want to make a purchase on a connected device. But transparency should also include helping consumers navigate a complex ecosystem of data, devices, and big data analytics operating behind the scenes so that consumers understand the practices that can affect them and exercise choices about these practices. Now think about the alternatives just for a second, the alternatives to being transparent with consumers. One reasonable response of consumers is that they'll harbor suspicions about a product or service and may choose to avoid it or use it less than they would if they trusted it fully. Another response, particularly when an entity that consumers don't know about is collecting or using data, is to react angrily when the truth about the company's data practices come out. Comes out, sorry. And I think it's wise to presume that the truth will come out eventually. In either case, the result is the same. Consumers or customers lose trust in a company and the results for the company can be devastating. Now many companies and organizations understand this, particularly this important connection between transparency, privacy, and trust. But being transparent in the data-intensive age is challenging. With the Internet of Things, many connected devices do not have a user interface to present information to consumers about data collection. Devices, as I mentioned, are becoming more numerous, adding to the mountain of information that companies present to consumers in their privacy policies that one researcher has found can be as long as Hamlet. That's true. As devices become integrated into homes and other physical spaces, there are also questions around who should receive disclosures about data collection and use practices. How will consumers who buy a device and the innocent bystanders around that device know when the device is recording images or taking down audio and recording audio? And there are other questions like how can consumers choose to avoid having their data collected? For how long will their data be kept by the companies who are collecting it? And how will these companies keep the data secure? Now, companies that provide connected devices should recognize that providing transparency will require some creative thinking. Visual and auditory cues and immersive apps and websites should be employed to describe to consumers in meaningful and relatively simple ways the nature of the information being collected. The same signals should be used to provide consumers with choices about whether any of this information can be used by entities or persons who fall outside the context in which the consumer is employing the device and in which the consumer expects her information to remain private. Now, there's another promising tool for providing information to consumers in this world of connected devices, as well as allowing them to exercise some meaningful control. And this is known as the command center that companies are now developing to run multiple household connected devices. The driving force here is obviously convenience, but these command centers could also provide an opportunity for consumers to understand the information their devices are generating and to control where that information goes.  Now, let me turn to the issue of transparency and fairness in Big Data Analytics and how it relates to consumer trust. Big Data Analytics, sorry, employed both by data brokers and within companies themselves, are being used to segment consumers by interests and traits and to make an increasingly wide range of decisions about consumers. Now, some of these analytics projects could create questions about fairness. For example, a company might analyze its own data in an effort to identify good versus troublesome customers. The good ones will get to the front of the customer service line, you know, on the phone or on the computer, and the troublesome ones might go to the end of the line. But what if this analysis ends up tracking individuals along racial or ethnic lines? A Harvard Business Review article argues that this kind of result isn't just possible, but it's inevitable. Transparency for Big Data Analytics, both within companies and through third parties, is necessary to engender trust by informing consumers about the significant impact that Big Data Analytics can have on them and clarifying their choices with respect to some kinds of data collection and use. Transparency is also a helpful check on potentially troublesome data practices, since, as Lewis Brandeis famously said, sunlight is said to be the best of disinfectants. Without more transparency in Big Data Analytics, questions will linger about the role that Big Data Analytics plays in the marketplace and whether consumers are being treated fairly. But the question is how to present information that is meaningful to consumers along these lines. Now many of these kinds of decisions are going to be based on some kind of score, a number that's generated by an algorithm that gives some indication of what a consumer is likely to be interested in or how she is likely to behave. A familiar example that I'm sure all of you know about is a credit score. Credit scores are basically predictions of how likely a consumer is to pay back a debt. The higher the score, the lower the credit risk. In their early days credit scores were used strictly for credit decisions, whether you'd be able to qualify for a mortgage, for example, and the interest rate that you were going to be offered. Over time the use of these credit scores expanded to other major decisions about consumers, such as whether a prospective employer would extend a job offer to an applicant, or an insurance company would charge a higher premium on auto or homeowner insurance. Now we know a great deal about what information is in our credit reports and what our traditional credit scores look like. And we know a lot about it for one simple reason. Congress has required some transparency in credit scoring. In 2003 Congress instructed the FTC and the Federal Reserve to study whether one type of popular credit score used for auto insurance employed factors that serve as proxies for race, gender, or other traits that could give rise to unlawful discrimination. In addition Congress required credit bureaus to make consumers credit reports available to them for free and credit scores are increasingly becoming available for free to consumers. Now these transparency requirements and practices have been good for consumers and I think they've been good for credit bureaus and for companies that rely on credit scores to make business decisions. With the transparency provided by free credit reports and increasingly free scores consumers can more effectively exercise their rights to dispute and correct inaccurate information. And the thorough analysis of one critical type of credit score by the FTC and the Federal Reserve made users more confident that this score was not discriminatory. Today we're seeing a proliferation of other types of scores being used to make eligibility determinations covered by the Fair Credit Reporting Act. While these scores are subject to the same obligations of access, accuracy, security, and other requirements imposed by the FCRA they haven't been subject at least not yet to the same kind of scrutiny that Congress and the federal agencies brought to bear on one traditional credit score. The use of these new sources of information including information that goes beyond traditional credit scores to score individuals and to score consumers raise fresh questions about whether these alternative scores may have disparate impacts along racial, ethnic, or other lines that the law protects. And an increasing range of algorithmic scores and decisions fall outside of the framework provided by the Fair Credit Reporting Act entirely. The FTC identified a few of them in our May 2014 report on data brokers. We highlighted so-called risk mitigation. And these are services that service sources to make significant decisions about consumers that are not subject to the protections of the FCRA. These services answer questions like, is this consumer who she claims to be and is the purchase that this consumer is attempting to make likely to be fraudulent? While some of these risk mitigation scores may fall under the FCRA, a vast bulk of them do not. Now transparency is important across the full range of decisions that I've just been talking about. But it's not realistic to rely on the approach that the FTC took to understand one type of score that was used for auto insurance in order to gain an understanding of the full spectrum of scoring models that are used today. It took the FTC nearly four years to conduct the study that I've been talking about. The FTC and other federal agencies for that matter simply do not have the capacity to study every new score that is out there. This approach simply would not scale. Moreover, scoring algorithms and other forms of big data analytics rely on statistical models and data system designs that few on the outside understand in detail. And even if we were, even if we were on the outside, those of us who were on the outside could peer into the hundreds of scoring algorithms that could potentially affect consumers, what would we learn? We might learn which features of a data set are used in a given algorithm and what weight a company attaches to them. These details, though, might be so abstract and so rapidly changing that they would not tell government, consumers, or other concerned stakeholders much at all about what really matters. And that, I believe, is how the algorithms are actually used and whether they have discriminatory or other inappropriate effects. This suggests that the testing, that testing the effects of big data analytics may be a promising way to go, but this route does have some challenges, too. On a technical level, many companies will not have the data, the type of data that they need to definitively answer the question of whether they are treating consumers of different races or ethnicities differently. For example, an ad network might track consumers by using an e-mail address or a device identifier. The ad network might be able to combine this information with other data that is readily available to it, such as which apps a consumer is using. And all that data would be used to make inferences about more sensitive personal characteristics. But this is far from having test data in which consumers' race, health conditions, financial status, and other sensitive personal characteristics are known. Doing this kind of analysis from the outside is difficult. Researchers have done some proof-of-concept studies, but they required considerable work and involved efforts to tackle some cutting-edge research questions. This means that companies using scoring models should themselves do more to determine whether their own data analytics result in unfair, unethical, or discriminatory effects on consumers. In addition to scrutinizing their own practices, companies can provide consumers with creative user interfaces to give consumers more meaningful, usable access to their data. Now, ultimately, I believe we need legislation to address many of the issues that I've been talking about, but technologists have a key role to play, too. They have the skills to make data access tools that are easy for consumers to use and they have the technical insights that are necessary to determine whether specific analytics and specific analytic practices pose risks of excluding or otherwise placing at a disadvantage groups defined according to sensitive traits. I'm hopeful that companies will give technologists, including designers and user interface experts the support and resources needed to tackle these critically important questions. So although I've been addressing transparency, privacy, and fairness from the perspective of consumers and also talking a lot about trust, I hope that the challenges of the Internet of Things and big data that I've been discussing, as well as my suggested solutions, apply to the schools and institutions in which many of you work. Trust is every bit as important to students, parents, and researchers in the educational and research context as it is to consumers in the commercial marketplace. Despite the complexity of some of these problems, some simple questions and operating principles can help guide all of us. The more you can tell your stakeholders about what you're doing to keep their data secure, to use it fairly, and generally to meet their expectations for the kinds of sensitive data that you collect, store, and use, the better off everyone will be. Thank you very much. So, as Clifford said, I am happy to take a few questions if anybody has any. Or you all may be very exhausted from a very full day or two of talking about all these issues and more, but I am happy to take a few questions if you like. And I think there are mics. There we go. There's somebody. Hi. I'm David Rosenthal from Stanford. I noticed that you talked about data security for the Internet of Things, and I want to raise a number of problems in that area. The first one is that the reason that these sensors are proliferating is that they're extremely cheap. And that means that the business model behind that's delivering these things is incapable of supporting the necessary software, continuing software updates that keep the things secure so that one thing we know is true of the Internet of Things as it exists right now and it will continue to be true of the Internet of Things in the future is that it's catastrophically insecure and that there is no practical way of preventing these devices being subverted to harm the owner or to attack other devices in the Internet. We've just seen a major attack on the root servers which has apparently came from a botnet running on people's smartphones. The Microsoft and Sony networks were taken down by a botnet running on people's home routers. And the DMCA effectively makes it illegal for people to research and find these vulnerabilities and report them. And the trade treaties which the administration is pushing make it even more illegal. In fact, according to the TPP, if somebody uses their laptop to investigate a vulnerability of one of these devices, their laptop can be seized and destroyed. That's not very helpful. And the other thing that's happening at the moment is that the Supreme Court's just ruled again that companies can impose mandatory arbitration on their user and user license agreements which means that they can do whatever the hell they like and you have no chance of suing them about it because if you do, you'll end up in arbitration and the arbitration will rule for the company. So there are a number of very serious problems here and the result is we're proliferating the Internet and the world with devices that will undermine its security completely. So, please, go ahead and applaud. I've no problem, sorry. I couldn't agree with you more. I actually had a whole section of my speech that it would have been even longer about security and touching on all those issues and more, but I decided to spare you and focus instead on privacy. But you're absolutely right. In our Internet of Things report, which I do recommend and I hope many of you will go to our website and at least take a look at at least the executive summary, you'll see that we focused a great deal of the report on the security issues, many of which you mentioned, and I just want to highlight a couple of additional ones. So you're absolutely right. The business model is going to be very difficult to address security issues with respect to the Internet of Things. It is absolutely true that many of these devices are going to essentially be throwaways and it's not going to make economic sense for companies to push through patches. They'll fix their new device, the new iteration, but they're not going to push through patches and it's going to be difficult to communicate with consumers who are using the older connected pens or connected glasses, whatever it is, that they are dealing with a security threat, a threat vector. And as to the threat vector issue, Hewlett Packard, going to your point about how insecure many of these devices are, Hewlett Packard did a study of connected devices I don't know how many they studied, but they studied a lot of them, and they found that the vast majority, I think it was on the order of 90%, are collecting personal or linkable information and 70% of that information is flowing over unencrypted networks. So we're talking about some really, really serious security issues. So I commend to you and to everybody who's interested in this issue, all of those of you who applauded and others really take a look at our report because that is the very issue that we focused in on. Oh, and I should mention one other thing. We're not just talking about the security of data, but we're talking about the security of the device itself. And there have been proof of concept hacks, if you will, or studies done of taking over a connected car or taking over a connected medical device. And this presents threats not just to individuals' data and their dignity and the fairness with which they're treated as I spent my time talking about, but it also affects their personal safety. So there are lots and lots of issues around this. Thank you for raising that because you spared me from having to talk about that and put that into my speech. So thank you. Please. Hello, I'm Sue Gardner from University of Nebraska-Lincoln, and one thing I'm concerned about is accuracy of data being collected about me. I have a very common name. One of the administrators of Wikipedia is named Sue Gardner. She's been conflated with her, which is really funny. I have a friend named Sue Gardner. There you go. There's another person in my... She lives in Hartford. See? There's another in my town. I have to always give my address when I pick up my dry cleaning. And Google Scholar, for instance, conflates me with other people with similar names. My husband, in fact, gets confused with me, and he's very accomplished in writing. So I bring his H score down a little bit. He raises mine. And I've tried to disambiguate these things sometimes. And then I sat back and thought, maybe I don't really want people, or whoever is out there, to know accurate information about me. So I stopped trying to disambiguate all of that. So I just wanted to bring that up and maybe ask your comments about that issue. Sure. It's another great issue. So the accuracy of this information is a big issue. And it's one of the reasons why, outside of the credit reporting context, I have been calling for legislation that would require data brokers that are selling these profiles to allow consumers to access their information so that they could correct it if they wanted to. Let me just mention a couple of things here. So the reason I spent a little bit of time talking about credit reports in my talk is because they really are the progenitors of big data. And Congress decided back in 1970 to place rules around how credit reports could be used and how individuals could access their reports and correct them because of the fact that this data was being used for very, very sensitive purposes. Can you get a loan and for how much? Can you get insurance and how much is that going to cost? Can you get a job? As a tenant, can you rent? So Congress decided because this data was being used for very important and sensitive decisions, consumers should be able to access that information and correct it. I absolutely believe the same thing should happen with respect to data brokers. But I have had a similar... I want to just mention a personal experience that I had that's similar to yours. There is one data broker that has been listening to me. Actually, all of them are listening to me. But there's one that has taken action and has provided an immersive tool for consumers to use. It's an online tool that consumers can go to and look at the data, at least some of the data, that this company has. The company is called Axiom. It's one of the biggest data brokers that exists. And the website or portal is called aboutthedata.com. And you can go to it and see, with respect to your marketing profile, not your risk assessment profile, which I also mentioned in my talk, your marketing profile, that is advertisements that you're going to see and whatnot, how you're being categorized. So I went to, of course, I went to the portal and to see how I was being categorized. And there were several errors in it. I think they had my salary wrong and they had my job wrong and a couple of other things. I decided not to correct it for the very reasons that you're talking about. I kind of thought, do I really want advertisers to know that much about me? Similarly, I went to Google and Google actually does have a really good tool that you can see if you can find it. If I could find it, you guys could find it. I mean, it's not that hard to find. But you can go and you can see how you're being categorized by Google. And Google has me pegged as a 65-year-old man. And it's because I watch a lot of sports. And I'm also much younger than 65, but I watch a lot of sports. I watch a lot of sports late at night because my family isn't in Washington here with me and so I'm here by myself. And I just watch sports. And I love basketball. I love soccer. So they decided I'm a 65-year-old man. And I could have gone in and changed that too and corrected that, but I decided not to. I decided, hey, let's see what I get as a 65-year-old man. I'll walk in someone else's shoes for a while. So that's just... But I do think this issue about correcting information when consumers want to. You don't have to. But if you want to, I think it's critically important that consumers be given this tool or these choices. Yeah, I'll take one more. Hi, my name is Anthony Hellman. I'm at Dartmouth College. I really appreciate your opening comments about the FTC's role as a leading consumer protection agency. Right. And I just wanted to follow up some of the other security thread questions as well with the degree to which or asking about the degree to which the FTC is countering or trying to advise other departmental agencies who are advocating, for example, for encryption backdoors into consumer devices and how that seems to go at odds. And there seems to be a real fundamental lack of understanding how one backdoor for one agency might as well be a backdoor for anyone with enough gumption, as it were. Thank you. That's another great question. I shouldn't be surprised that I get great questions from a crowd like this. There's a lot of discussion underway now with respect to encryption and with respect to backdoors. I mean, you all undoubtedly have been following these issues. And frankly, these issues aren't really new. They kind of swing back and forth and have been swinging back and forth for a while now, at least since the 90s. You know, since the internet became such a powerful collector of data that could be used not only by commercial entities but also by governmental entities. I have stated publicly in speeches and elsewhere that I am worried about the magical thinking, and that's what I think it is, that creates this notion that backdoors can be exploited by some for good purposes, law enforcement, let's just say, and national security and otherwise. Really, it's law enforcement, we're talking about, though. And not exploited by people who have bad intentions, for instance, hackers, other malicious actors. So I'm deeply worried about our ability to do this. And I'm worried that the belief that it can happen safely is, as I said, a form of magical thinking. So I think we do need to keep in mind as we have these debates about encryption, the consumer's perspective and the desire for consumers to have their information held securely and to ensure that there aren't going to be these vulnerabilities built in that could lead to significant problems for them, not just with respect to law enforcement, but in other areas that we've been dealing with for so long. So, yes, we have been interfacing on that issue and it's a deeply important issue. And as I said, I think the dialogue and the discussion about it will probably continue for quite some time. So thank you very much. I really appreciate it and especially thank you for your questions. Take care. Bye-bye.


For professionals working in financial management roles, consultancy and advisory functions or international financial institutions, SOAS University of London's MSc Public Financial Management by Distance Learning programme will provide you with the expert knowledge in all matters of public finance, ranging from budgeting and revenue policy to financial reporting, audit and compliance. Taking a real-world approach to the application of theory and critical thinking, the programme examines where financial matters drive policy, such as in public-private partnerships or fiscal decentralisation. The course is comprised of six modules, three of which are elective, three of which are core modules. Against the backdrop of a global shift towards a more uniform approach to financial performance measurement, you'll gain an understanding of international reporting standards, including an introduction to core accounting concepts, a financial reporting framework and the guidelines currently available to international companies. Alternatively, you'll explore international public sector accounting standards, discussing and evaluating the theoretical framework and principles of financial reporting for public sector entities and developing your understanding of accountability, disclosure and governance. You'll deepen your understanding of macroeconomic policy and financial markets. You'll focus on the planning and expenditure side of public finance management, looking at budgeting, investment decision-making and how financial management, which is under increasing scrutiny, can be used to improve an organisation's performance. You'll also study the theory and practice of public finance with special reference to how governments raise revenue, specifically taxation, borrowing and aid. In addition to the core modules, you'll choose three of eleven specialist elective modules in areas of your choice, ranging from decentralisation and local governance to research methods and the International Monetary Fund. To find out more about how you can further your career through the MSc Public Financial Management Program, click here.


Hey everybody, it's Hari Swaminathan from optiontiger.com. Today I want to look at the gold charts. Gold has been behaving rather strange even when the S&P markets have been down, gold has failed to act as a safe haven which it generally does. So if you look at the S&P, we've been coming down over the past week or two and if you look at the gold charts also, we've been coming down from the past week or two. So that's a correlation that generally works the other way. So I wanted to take a look at gold charts. This is a one-year daily chart. Now I'm just going to draw some drawings here so we understand the levels. What we have is on the top side, it's pretty clear that there is some very heavy resistance going on at this level which is at around let's say the 1350, 1360 level is very heavy. You can see there's a lot of touch points there and let me just draw this here and you'll see the general area. We're talking about a zone here, not particularly at any one level, but this entire zone has had several touches here. So if we just freeze this line here, okay, so if we call that as the upper point, we can see clearly over the past one year, gold has had a very difficult time to break out. It's been at least one, two, three, four, five touches to that point and it's not been able to break out. Meanwhile, there's been severe bouts of down moves and also there's been one or two very strong moves like this one and once again here there's a strong move, there's a little bit of a pullback and then again there's another strong move, but then the down moves are also pretty intense and as if you can see this the number of bars which is the number of days is reaching pretty much its normal level at least in the recent past. So you can see there's a similar level here, there's a similar level here and even on the upside there's a similar level there and here as well. So here we see some larger moves but over the last let's say four to five months we've seen this a two week sort of a range being that for gold. Now let's look at the downside because right now gold, okay, so these are the gold futures, the gold ETF of course is GLD, GLD is a little more than one-tenth of the gold futures but if you look at the downside we are right now over here but and you can see that there is a pretty strong support level here and then the next one is over here. So I would say if we draw a line which is around which is around 12, let's say 12, 30, 12, 40 on the gold futures level and then of course the next level would be this the 1204 level. So if you look at these are the three levels that gold is looking at. Now the key is whether it can break through or get supported here at this level. So if it breaks through then there's another 40-point drop clearly that's possible because there's nothing in between but if it does bounce back it's got a pretty good room to at least come up to the 1313 level and perhaps even more than that up to the 1360 level. So the point of this video is I think if you're a gold trader you need to watch this level and see if this 1237 holds, if it holds then you're looking at a pretty big move upside if it does not hold then we're looking at going down to about 1200 or 1204 and which also would be a pretty big move and so there can be some good trades here but the trade doesn't happen today it may happen in the next couple of days. Thanks.


So with me, because I have a 30-year first look at the US Marine Corps' 130-meter-dreams body. I'm really why we're here. It's got to work violently with the Filipino government. I'm an officer, but with that, I'm on a regular scale. The local government, she kind of provides really what's both gone and says, gone out in the field, talked to locals, talked to politicians, talked to those that have other touchpoints, really gathered feedback in regards to what's needed, what would make an impact on the local communities. Now, obviously, I've just identified providing further schoolhouse for the group stores with an overall school, but what we're doing is evaluating it today. We're not identifying it, so that's why I was filled. So we're just back here going backtrack in and evaluating what you're going to actually get done and what was asked and what you all wanted us to provide. So the end state or the global front of the overall project, did it have an impact on the community? What's the positive? Was there anything that was missed? Can you keep following that on? Thank you very much. This is the 37th Valley TAN, and we're nearing 75 years of diplomatic relations between the US and the Philippines. And this exercise comes at a time where our relationships and our partnerships that we've had in this region are so important. We know that the better off of those partnerships are relationships that we've formed down to the local level of community. First thing, obviously, again, thank you for having us back here. Obviously, it's been a few years. It's sort of happened at the top of the whole economy over COVID. And then I put a little bit of a hammer on the spot. So I appreciate you making the time today. Thank you, sir. I'm going to first student. I, 26, maybe it's my classmate. I'm going to first student. Because they are not going to study under your children. They have beautiful buildings on their own, in their study. Yes, yes, going up. That's why they need more facility to accommodate our students coming this year. We're going to get that. Two buildings are very, very important to our learners. They are happy. And we find that they have interest in learning because of these rooms. Thank you. We are thankful for these two classrooms. We have pupils and our students, principals. And especially the pupils are very thankful to you. We love this school. Thank you.


Hubbard County authorities are asking for the public's help in locating a woman who has been missing since February 2nd. Kara Blevins was last seen on one week ago in Akeley and is listed as a missing person. It's believed she has no phone, car, money, or extra clothing. If anyone has any information, you're asked to call the Hubbard County Sheriff's Office. If you enjoyed this segment of Lakeland News, please consider making a tax deductible contribution to Lakeland PBS.