Howdy! My name is Katherine Scott. I'm the developer advocate for an organization called Open Robotics. I'm here to talk about ROS2, where ROS stands for Robotic Operating System, 2 stands for 2, and basically we're going to talk about free open-source software for building production-level robots. All right, so the classic overview in five parts. I'm going to start with some social proof, because I think that helps sometimes keep people motivated and gets the hook. Talk a little bit about the history of the project, the philosophy, and then what's in the box in ROS2 and sort of the community that surrounds it and what's up. All right, so ROS. Why should you care about ROS? So ROS stands for Robot Operating System. It is not an operating system. It's more like a layer that you drop on top of your operating system that runs just like Linux and makes it suitable for robots. And ROS is probably the largest, oldest, fastest-growing ecosystem for robotics, and it's also free and open-source software, unlike some of the others. So ROS2 in particular, in the past couple years we've built a ROS2 Technical Steering Committee. There's still a huge and growing community of core devs and people who are defining direction, but we've also been working with these very, very large organizations to sort of plot a future for robotics, and particularly robotics for things like manufacturing and autonomous vehicles and all of the things in life that we think robots are going to do and make our lives much easier, the Star Trek future that we all wanted. One thing I like to talk about with ROS, if you're not familiar with ROS, this will all be new news. If you're familiar with ROS1 or maybe where ROS was a few years ago, some of the stuff may come as a surprise to you. So I personally, this is hard information to come by, I personally maintain a list of companies that are using ROS. So for almost 300 companies on that list, there are Fortune 50 companies, there are small startups, there's everything in between, and this is just what I know about. This is the BSD license problem where you put something out in the world and you don't know who uses it unless they complain on a forum or like put it in a job solicitation or something like that. So these are the ones I know about and I've only been doing this for like six, eight months. A more robust metric that I think people would be familiar with is last year, 2019, we had 264 million ROS dev downloads, which is quite a few. We have 35,000 users on our own Q&A forum. We have over 5,000 users on our just general ROS discourse forum. I did a search the other day on GitHub for all the projects that are tagged specifically with ROS. There were over 2,000 of those and those are the ones that are tagged, like we don't even tag someone. So there's more than that. We keep a wiki and the wiki is actually the older ROS1 version. Mainly, there's a lot of relevant information there, but still that wiki gets 200,000 visits a month, which makes it a pretty sizable operation. And then we just wrapped up conference season. So ICRA, the big robotics conference just wrapped up. We're now at over 7,000 research citations for ROS. You maintain a website called metrics.ros.org that I would encourage you to go take a look at because it kind of distills some of this down. And if you don't believe me, that ROS is a big project, well, there's a lot of evidence to the contrary. The other thing that I think helps now is that one of the biggest complaints about ROS and taking ROS from, say, something that was used in research to something that is used in production or used in big manufacturing facilities and big industrial warehouse was, oh, we don't support Windows, so we can't use it. Well, it turns out now we do, Microsoft's actually helping out a little bit with this. So we now have a toehold for all these ROS systems into this sort of Microsoft world. And I think this is actually great. I think this allows us to start getting more and more open source out into these places where I think we want open source to be. So I wanted to give sort of an overview of the history of the project if you weren't aware. And I think, you know, with almost all technology, and I certainly found this when I was in school, that if you talk about the history of how something was developed, you can kind of really understand why things are a certain way and why they're put together in that way. So in the before times, I had to go find an image of, like, you know, robots manufacturing stuff in the 80s or so, or even the 90s, because it's big, huge, industrial, capital intensive manufacturing operations moved to robotics. It was very, it was either that or it was, you know, research student or research projects at university focus very much on the doll, the dirty, the dangerous, the studies of 3Ds and robotics.  The software in these organizations was like you'd expect for these large organizations. It was proprietary. It ran on Windows 95. It was bespoke for the application. It had weird hardware controllers that had weird daughter boards that go into your PC to do stuff. There were the system integrators that come in and would, you know, wire and glue and duct tape put all these things together. And you are very much leaning on your vendors to do things. And the vendor API that if it was on the web, it's behind a paywall or, you know, behind a registration wall in a password protected zip file. And that was the before times. So about 2004, so this is right when I'm getting out of college, there's something started changing, something kind of clicked. And in my mind, this may not be the only thing, but this is sort of the watershed moment is DARPA, the Defense Advanced Research Projects Agency for the US says, hey, we're going to put together this competition. We're going to put a bunch of money out there. And we're going to ask all these universities and organizations to give us your best vehicle that can drive by itself outdoors.  It didn't work great, but vehicles were actually able to go with a non trivial distance outside and not fall off the road and do what they were supposed to do. Sort of concurrently with all of those things happening in 2004, you know, Google wasn't as dominant as it is now. But in 2004, it was when Google had its IPO, so all of a sudden these people working there and working in Silicon Valley are flush with cash. And the reason I point out Google specifically is Google is one of these orgs that has at least had, you know, strong opinions about open source and actually using open source to do what it is they do. And so all of these things are stirring in a pot in Silicon Valley in, you know, 2004, 2005, 2006. And it's, we have these autonomous robots. We have money. We have open source doing all these cool Web 2.0, as it was called stuff. And at some point, somebody says, well, I'm going to leave Google and I'm going to try to take this whole idea of free open source software and apply it to these robots and see if we can really push robotics forward. And that organization is called Willow Garage. You may have heard of it. And Willow's, you know, core belief was like, let's use open source to solve the robotics problem and build all these cool robots that we were all promised when we watched TV in the 90s. And so from Willow Garage and all of this venture capital and money from Google come three great open software projects, right? There's Open CEV, which if you're not familiar, stands for Open Computer Vision. It's basically a collection of, you know, all of the primitives that you need to build a robust computer vision application. So it's like, hey, I need to find the lines and something or I need to like change the colors and something or I need to find a face. There's PCL, which is Point Cloud Library, Point Cloud Library. At the same time, you start seeing the first like Microsoft Connects and all of these cool 3D sensors. The thing that allowed in the urban and grand challenges that allowed people to win, right, to dominate was really these Vellinine LiDARs, these spinning LiDARs, and those generate Point Clouds. And you need tools to analyze this Point Cloud. So let's put them all in the library. It's called PCL. And finally, ROS, which stands for Robo Day Offering System. And that was the same thing. It was, let's take all of these things that researchers had built 1Z2Z at each university, and they built the same thing over and over again. Let's put them all in a standard library and start seeing what we can build. Sort of concurrently, Willow did a lot of work just to build robots that want products, to build robots, to build roboticists that could build those products, right? They laid the ground work for this sort of next generation of roboticists. And they built two things. They built a small robot that was really suitable for undergraduates, graduate students to get their hands on and play with and get dirty. And that was called the Turtle Bot. We still make them. And then they built this big, cool, does everything you'd ever want, like multiple arms, all the sensors, mobile robot called the Pier 2, Personal Robot 2. And they just kind of shipped them off to all of these different research facilities across the globe. And so they ship all these robots everywhere and they start sort of incubating and collaborating and talking. And all of these different research groups all around the world are talking and listening and building and sharing their code in what we would consider now very modern ways. And they start, you know, how do we build a project? How do we do, you know, build up the standard library of robot components across the globe? And so sort of from that ethos comes these sets of ideas that I kind of called like the ethos of Ross, right? The first one is sort of federation over centralization. I know, or, you know, decentralization, you know, and we're kind of coming back to that now and I think a lot of ways.  Let's figure out a way to do that smart. And that really, you can see that in like our build tools and how things are packaged. You know, and then the next thing is very much the Linux ethos, which is small, simple, composable programs that you can string together to do cool stuff. Third one should be self-evident. Don't reinvent the wheel. Let's build shared common tools. The other two are kind of hard to define, but the first is non-exclusive. Like Ross and all of the associated tools, they work really well in C++, but there are also, you know, you can build them with Python and you can interrupt them with other things like Java and JavaScript. And I saw something for Russ the other day, right? So we're not going to say like, we're going to build all these tools and we're not going to say you can't play with us because you're using a language we don't like. It's more about bringing all those people in and then sort of concurrently, I would say inclusive. And that's maybe not well defined, but you know, I think this is the biggest misconception about Ross. And it is that people say, oh, I can either use Ross or I can use this other thing. And the reality of the world is it's not an is or thing it is in addition to. So you can take from this library what you want, that what helps you and the things that you don't like, the things you don't want to use, well, you can find something else. And that's perfectly okay. And then the last one is freedom, which I think everyone gets is the non-restrictive licenses. Go do what you're going to go do in the world. Sadly, my great illustration here in 2003 or 2013 will have folds and the open source is kind of sitting there.  And we formed two new or, you know, one organization for us open robotics, which is sort of the steward of at least Ross. And then we have a couple partner organizations now that help us take that code and make it suitable for other applications. So it's still living on, it's still going and there's still an organization that's really, really driving this work. Okay, so what happened? So, you know, around 2018, we'd come through and we released a ton of different versions of Ross. Basically, we're releasing it every, you know, once or twice a year and having like longer LTS versions and then, you know, intermediate versions where we're just sort of working out new features. And 2018 comes on and we start talking with people and we realize that, you know, probably now we've learned everything we're going to learn. There's actually some things that need to change to make Ross better. And we decided to start working on Ross too. And so, you know, at 2018, we basically start winding down Ross one production and we start really ramping up on Ross to production to the point where like Ross one is basically, you know, there's not going to be another version of this. The community can keep doing as they please, but it's not under active development. So, anyway, working with the TSC, we sort of identified how we could make Ross better. Like, if we had to make a whole new version, what would we do? What needs to get fixed? What would make it suitable for all of these different use cases? Not just for researchers, but for everyone. And so, we've iterated a few times. We're now on the sixth version where Ross to Foxy Fitzroy. And this is LTS. It's got three years. It's actually going through the API docs. I think they're fairly, you know, there's still a little bit of work to do, but I think they've sort of stabilized and they're ready for people to use and it's got a bunch of new features. And I'm going to kind of go through and talk about what's in there. And I'm going to talk about also the philosophy of how we do things. And, you know, I think everyone talks a lot about Python and, like, the Zen of Python and how we do stuff or how Python does stuff. And I think Ross has sort of similar things. I'm not going to say this. This is the canonical version, but it's like I said before. It's like, don't reinvent the wheel, build common tools, defer to federate, you know, do federation versus centralization, build small simple composable utilities, be non-exclusive, be inclusive, and generally, you know, default to freedom, which is non-exclusive licenses or non-restrictive licenses. Okay, so let's talk about the parts of what you're going to find in Ross.  And your basic robot has three parts. It's got actuators, like we got actuators. It's got sensors, like we have sensors, which are things that take in data. And then it's got something to make sense of that data and do something with the actuators, right? So I have three different bits of clipart here. And so let's talk about the first sort of wheel that we're not reinvented or you're not going to have to reinvent if you use Ross, right? And the first is just multi-process control. And I, like, I know that sounds kind of weird. But let me describe what's going on in Ross at a high level. And then you can go take a look at the code yourself. So Ross nodes are basically processing encapsulation between languages. And what I mean by that is all of your Ross nodes are going to interoperate. They're going to be doing stuff together. One might be written in C++, one might be written in Python, one might be written in something else. They all have a basically a standard way that they're going to operate with Ross, right? They're going to have a standard way that they compile and be installed. You're going to have some tools to start them, stop them, pause them, query their status, and collect your CLI. So you're just saying, if you're going to build a Ross program, it's going to roughly look like this. And what we do is we gather these nodes into packages. And a package is just a collection of software. It could be multiple nodes, it could be one. And these packages come with their build instructions and you're testing and all, you know, all of the things like, it's a, for lack of a better word, it's a library, right? It's a package of things. And so the way I make sense of this when talking about Ross nodes is that imagine if it weren't this way. Imagine if we didn't build tooling around all these different programs that are going to be talking and doing stuff. Imagine writing the shell script that starts a bunch of C++ and Python programs that they all kind of need to come up concurrently. All of them are interfacing with different bits of hardware, right? So they're interfacing with some camera that does weird stuff when it starts up for some other system that does something when it starts up with lots and lots of configurable params, like the input of this thing needs to be renamed to the output or the output of this thing needs to be renamed. So it goes into the input of this thing. So all the pipes need to be managed. And there might be like some flow control in there. So you're just going to end up with this gnarly 5, 600 line, right? Shell script if you have a large robot. So we basically built some utility on top of it. So the nodes are this encapsulation of, you know, your process, right? If you're familiar with Python, it's kind of like argvars plus main plus a couple other things. And then the tooling to start all of them. So now that we got all these different nodes, we have to figure out how to wire them up. What's the glue that like takes all these process, you know, inter process communication? What is the glue that makes them all work together? And so Ross nodes communicate over what we call topics. Topics are basically a pub sub bus. If you're familiar with rabbit mq, zero and q, zero and q. Um, proto boff, like it's, it's the same thing. It's our spin on it and it kind of predates a lot of them. And it's all based on having very, very standard messages or messages with their types included that allow you to move data between, you know, different languages in different processes smoothly. And then it's a bunch of tooling on top of it. And so the, the messaging is really important to remember because the messages are sort of a lot of them are standardized around common things in robotics.  They're, you've already got this sort of standard library of types moving around through this system. Um, and what's really, really cool about topics, and I've talked about this is they, they help you with your code due to things. The first is really, really good at helping with loose coupling. So and I, you know, I see this when I work with younger engineers, not very good at decoupling things.  So you get nice modular code. You're just defining the inputs, defining outputs, and then you're building the small process modules in between. And it also kind of gives you a level of plug and play. Right? So like, if you see here in my little illustration, I, I yanked out the eyeball and placed it with a camera, right? And so in Ross, and this has happened to me in the past, right? You have your robot, it's got cameras on it. You don't like the cameras. You yank those cameras out, you put on a different kind of new camera. It's not a lot of work to do that. And the reason is, is that all of the cameras just publish camera messages. And to the rest of the system, it doesn't even see or care what's going on.  There's a lot of capabilities for introspection of these messages, what they're doing. You can actually define quality of services. And I'm not going to get too into this, but it's worth digging into later on your own. It's built, all of this is built as an abstraction on a system called DD, or it's not necessarily based on DDS, but more often than not, it's based on DDS. And these DDS systems basically give you somebody, you know, someone to talk to you about like, well, how secure is the system? What do we do? You know, we need it to do X, Y and Z. And it basically allows this online or the implement the underlying implementation of the pub sub to be pushed off into different implementations. And, you know, those implementations can come from a vendor or they can come from open source. As I said before, there's all this sort of cool introspection capability on top of it.  There's also visualization tools that are built on top of it. So in particular, it's, this one is called Arviz. So Arviz will let you see all these camera images. It'll let you see all this 3D point cloud data. It'll let you see the maps that the robot makes. And you're able to sort of pick and choose these different topics all at once and put them on your screen and decide what you want to look at. And you can also kind of look at them in context, right?  And it's very, very helpful for understanding what your robot's doing. So let me take a sip here. Another thing that I see people reinventing, and I really hope people don't reinvent, is basically serialization. When you have a robot, you're generating a lot of data, and you need to either save that data for machine learning. You need to save it because it's just a part of logging and you want to understand what failed. So with ROS, what we've done is we've got all these messages serialized, they're moving all over. All we have to do is kind of like tap into that message stream and build a utility that dumps it to disk highly efficiently.  And so these bag files are great for, you know, debug and failure analysis, logging, really, really good for collecting machine learning data. They're just all around a wonderful, wonderful utility and there's so much functionality you should never go in and rebuild that. At least I hope you never would want to, because it's really quite great. I mean, you can take a bag file, a short bag file from your robot, and as you're building your unit test, hook up the bag file and replay the data to run the unit test to verify that it works.  And it's just automatically packaged like that, which is, that's sort of my background. It's very, very handy. You can take a bunch of data, say you have, I saw this together, I thought there was a collaboration, like say you have this really cool sensor that's still a prototype, you can collect a bunch of data, and then package it up and send it to a collaborator somewhere else and they can use it and kind of use it as a simulated sensor on their robot. And then the other really, really handy thing is like bags aren't logs, but if you're going through a log file, right, and something on your robot blew up, and you want to know why that happened, like you can go through first, you see the log, then you go back through your bag file and look at when that happened, and what data was going in the machine, and you can kind of trace back into this input was what the robot saw that needed to do the thing that I don't think it should have done. So you don't have to take my word for it, like bags are basically starting to become de facto standards for data sharing among people building advanced autonomy applications and other sort of robotic applications. So four just released a huge data set from their autonomous car efforts. On the right I have another data set from like ETH Zurich. People just find this a really handy way to share and collaborate, right, because you just take your robot, run it, say bag this data, you know, once the robot's done, you just take the bag and copy it up to the web and it's out there and it's ready to go. It's very, very handy. Another thing that you shouldn't reinvent on your robot is basically, you know, how do you how do you build really good synchronous and asynchronous APIs for robot behavior? I've seen people kind of come from Don Robotics background and try to do this and they never seem to do it right. So we basically have two things in ROS that are your behavior primitives. The first one is your synchronous behavior primitive. So that's called, we call that a service and that's anything, it's just basically a framework for doing anything that would be simple on your robot like in an autonomous car, it'd be lock the doors, turn on the light, flash a blinker, well flash blinkers a little weird, but any of those simple things, there's an API to do that and then there's also an API to do complex asynchronous behaviors.  Unfortunately, it can't violate physics, it can't go from over here to over here instantly and there's a process of doing that and it's not just an asynchronous function call, it's an asynchronous function call that handles like negotiation of the request. So sometimes, you know, a robot can be very busy, it can't actually do it, so it'll tell you it can do it and then as it is performing that particular action, it will, you know, tell you its progress. So I'm going from A to B, I'm going to tell you how far along that path I am and where I am at any given time and then it's also going to support preemption, which is nice and all of these are done mostly through callbacks, so you don't, you know, you don't have to worry about how you're going to handle all that async, you're just sort of filling in the steps of that state machine and going to town. And I think this basically allows you to build very, very robust behaviors very, very quickly. I think the thing that people who are interested in robotics or coming into robotics, whether for work or just for education, the thing that's most intimidating are these geometry and kinematics and like understanding where everything is on a robot, right? So within our simple robot, right, the hands have a frame of reference I put right here, the sensors have a frame of reference over here, and the brain also has a sort of global view of the world. And, you know, the problems that people consider hard is where, where is this particular part of the robot, where is this thing with respect to the world, how do I get from A to B, you know, can I see this thing and grab it, how do I grasp it, and a bunch of other sort of problems, right? So in Ross, we already have all the utilities to handle, you know, the geometry of robot. And so the first one is something called universal robot description format. And Erdiff, which sounds Swedish or something, but in Erdiff is basically a file that defines the 3D geometry of your robot and all of its joints. And what you can do is you can take your CAD software and some of them allow you to export it. And you export this whole 3D model of the robot. Now you can see it in that Arvis thing I showed you earlier. Now you can do basically like what is the position of this thing with respect to this thing, all of that's already baked in. Along with this, there's something we call TF2 or transform2. So transfer2 will say like is a set of utilities that'll do all the fancy math, all the things that people consider hard to tell where one thing is with respect to another. So if your camera right has a field of view and it's looking out and your arm is moving, it's sort of, you can answer questions like is my hand in front of the camera? Or, you know, well, all these different sort of problems, right? All of the kinematic links inside the robot, it can tell you where they are and answer like where they are with respect to one another. So my little illustration here, basically it's just TF2 does the position from like say pixel back to your actuator. So now that your robot sort of knows where it's everything is and where all the positions of the joints are, the next thing that people want to do is they want to figure out how do I go from, you know, especially with like an arm, like a 5x's arm or 6x's arm, I see a thing, I need to go get the arm to get there. And so solving that problem is basically a library or a set of tools inside of ROS and it's called MoveIt. Sort of the subgroup within the whole ROS community. And MoveIt is the set of all the tools you would need to do kinematics. And it's very, very well done. It has lots and lots of different implementations. It allows you to build robust applications around picking and placing an arms without really having to know underneath the hood how all these things work. So it's quite a valuable utility. Along with MoveIt, there's also what we call the NAF stack. So the NAF stack basically does two things. It does SLAM, which stands for Simultaneous Localization and Mapping. So what happens is if you have like say a Roomba-like, you know, small autonomous robot, it drives around a room and it figures out where everything is and it builds up a really, really basic map and then it can do navigation around that area. And that's just a feature set that's available in ROS as a set of tools. This little video I stole from the NAF website, right? And you can see basically the map of the robot and the path planning around it. And this is all ready to go to be dropped under your robot as an open source library. Another thing we should talk about with robotics, and I think this is sort of the not as well discussed thing, but it's super, super important, is simulation. So once you actually get into like a big robot, like a robot that's an autonomous vehicle or is a capital good, right? They're expensive, they're dangerous. In terms of software as a programmer, they're slow, right? If you have a behavior you're working on, you may have to start your process, let the behavior start to queue up to get to the point that you're actually working on. And that all takes time. It's a very slow process, actually, software-wise. Another thing is these things are expensive, they may only have one, and they have to be shared between, you know, 10 graduates, students, or 10 programmers. That's kind of a pain. And another thing is just that whole testing paradigm that I said before. How do I, you know, I want to go run tests. Do I run tests on the robot when everyone's sleeping and they can't use it? Like you need a way to basically test behaviors and do regression tests and integration tests without having the physical hardware there. And so what ends up happening is we end up building simulations of robots all the time because it just makes our lives so much easier. Well, Ross has a sister library, Ignition Gazebo, that is just about simulation. And Ignition allows you to build a virtual robot. You can basically take that 3D model, that Erdiff and drop it into Gazebo. It allows you to simulate the physics, other robots. It allows you to simulate sensors. And all of this happens playing very, very well with Ross, too. So you can kind of take your code base, pick it up from the physical robot, drop it onto the simulation, and do all your work there. Do all your testing there. Do all your prototyping there. And then when you're fairly confident, then you can go back and drop it back on the robot and see how things are going. And this paradigm of doing things in simulation well before we do it in the real world, since computers are so powerful now, and usually robots aren't, is sort of the way forward in how things are going. I have a, this image on the right, this is a competition called Ariac run by the National Institutes of Standard Technology, where they just built this giant simulated factory floor, and people compete on performing different tasks and how they program them. In terms of what comes with Ross, it's not just about these utilities that are sort of like core functionalities that I've described, but it's also, you know, what is the power of Python? The power of Python is that it's got batteries included, and Ross really is sort of like batteries included for these robotics, for robotics applications. As I said before, Ross has packages, and packages are just collections of code that you can go and grab and use in a variety of different languages. And, you know, we keep an index of all of these different packages, or a lot of the different packages. And so you can go through that, that Ross index, and see all of these different tools that you may potentially need to go and build your robot. A lot of them are actually go through a fairly rigorous testing program. We have a whole build farm that builds and tests a lot of these packages and a lot of these packages in integration tests. It allows you to have all these, you know, cool parts of a robot that you'd never think about. So like one I always like to call out is this robot web tools. If you want to go build a web-based interface for your robot, it's just there and you can grab it. And all of the Ross topics that you, you know, are running on your machine, you can now expose through a web socket and start building a front end on top of it. So really, you know, like if there's things that you're good at and things that you're not good at, there's a chance that there's a Ross package that will like take care of all the things you're not good at. Along with these packages, you just have all of the hardware drivers, right? Which, you know, drivers always been a problem on Linux. But more and more and more, especially for robotics hardware, we're seeing hardware vendors build, you know, a Ross package.  And you just sort of grab the package, build it, and run it. And you can go and play with all of this cool data or all these cool sort of components. Not quite at Lego, like Lego Glandal yet, where you can just grab them all and they all kind of work, but we're getting kind of close. And it's not, it's not this monumental task that I think it used to be. Along those lines, one of the big innovations, I think in Ross 2 is, is basically we're, groups have been working on what we're calling micro Ross, which micro Ross is basically a Ross 2 interface to an RTOS. So, you know, on all these different embedded systems now, it's running in RTOS because it's doing real time control. And there's now in that code, you can actually just insert, you know, a library and have it directly interface with Ross topics, which I think is amazing. Like this has always been something that's held us back. And now that there is a very, very clear path to Ross interacting with some of these real time systems, it's, it's quite amazing. And I think the future with this is very, very bright. So those are, you know, I've talked about some of these features in, in Ross and Ross 2 and how it makes your life easier to go build a robot. But even on top of that, there's a fairly substantial community of people who build these things. If you're interested, I think the first place you kind of start is Ross.org. It's a little bit of an old website, but it'll point you in the direction of all of the community resources that we offer. There's also the Ross 2 documentation is in a little bit different spot. So it's an index.ross.org. But if you take a look in there, there's a fairly, I think we're like at the Ross core API documentation level, Ross 2 documentation is basically parody.  We also have our own community discussion forum on discourse. It's fairly active. This is where people come and make their announcements, post their cool new projects, talk about packages that they're building or want to build. It's definitely worth checking in on a regular basis. We also have our own Q&A website, which sort of predates Stack Overflow. So there's a lot of good and useful information on there. If you're having a problem with Ross, and I'm going to admit it's not always the easiest thing to use, you can go there and more likely than not, a core dev or somebody who's very, very familiar will find and answer your questions. You'll make some friends. You'll sort of join this large community. So just wanted to point out that we've released Ross 2 Foxy. We think this is sort of the release where we're saying, hey, if you're starting a new project, if you're going to upgrade, just go ahead and switch over to Ross 2. It's the first major LTS of Ross 2. So this version is going to have a three-year LTS. So we'll have three years of patches, three years of support. And it's not, should be fairly easy to upgrade. It's not a monumental task if you already have a Ross 1 application. If you'd like to learn more or get started with Ross, I can't tell you what you're able to do and not point you to a direction where you can get started and actually start building robots and understanding how they work. So DARPA has the sub-T simulator, which sub-T is a competition, sort of like the competition I mentioned earlier, where they have robots navigating and sub-training environments. So they have a very, very big, complex, simulated tunnel system that they have created. And you can actually build, they have preconfigured robots that you can drop in there, and then you can start programming them right away. And we've actually done a lot of work with them to help build out tutorials that should help you learn how to use that nav stack, learn how mapping works, learn how to do basic perception tasks. And it's a really, especially now where people can't leave their houses as much, it's a really, really good way to get started without having to deal with the physical robot. And there's a lot of things there that can actually help you learn and grow and understand how to be a roboticist. Along those lines, there's a sort of parallel community with Ross called AutoWare. So AutoWare is sort of the people building the next layer of shared resources on top of Ross, specifically directed at autonomous vehicles. So we've been working with them to put together a set of tutorials where all these sort of luminaries and autonomous vehicles get together and talk for an hour on a subject that they're particularly good at. They also have sort of this prepackaged Docker container with simulation in it so you can go and try it out too. So this is about halfway through right now, so I would go take a look at it. And then finally, as I said before, there's the Ross 2 docs. There's tutorials and there's just really simple basic tutorials that get these concepts that I've talked across. And hopefully those can help you understand at a more specific level of API and application development what's going on in Ross. And then finally, we also have a turtle bot that it's still made. We still sell them. If you want to pick one up, if you want a physical robot to actually go run around or play with the kids, they're still available. So thanks. Again, feel free to check out Ross.org. Feel free to drop me a line. I'm here to answer questions. And yeah, thank you. Go build some robots. It's weird. I don't hear anything. Thanks for giving this talk. I'm just going to read off questions. Hopefully that's working. Let me... I got the speaker mic here. Hold on. See if anyone's saying anything to me. Let's see. Stopping every 20 minutes. Should I start with Ross 1 or just completely dive in with Ross 2? I would absolutely recommend starting with Ross 2, especially if you're building something for a long haul. If you're looking for... You're going to get all the concepts like you should be able to translate back and forth. I think that's the best way to get started at this point. It's still a little rough, but it's definitely a time to start switching over. I got a really long question here. The barriers are actually... We think more... Know what the big plans and issues from Open Robotics are to help bring down the complexity of Ross, both the perceived and actual complexity. Also, please tell Open Robotics we can't wait to see them again at Ross. Yeah, so I'm well aware of the complexities working with Ross. So part of it is we just need to... I know this is not a not satisfying answer, but part of it is we first recognize that the things that we're doing are really hard, right? So no one says, oh, what do we do to make organic chemistry easy? And part of it is that it's hard.  Now that said, the way I think things are approached and sort of discussed, I think Ross already does a lot better than, say, a lot of the academic literature on describing robotics concepts. But I think for us, the challenge right now is to find resources and capabilities to really deliver content to people that gets them to where they need to be. And I think this is really important. I actually think this is a huge sort of economic issue in the sense of we need to train the people who are building the robots that build the future that we want to see. And so I think there's going to be more and more curricula sort of focused on not just building robots, but maintaining robots, developing with robots, etc. Everything's working okay, correct? Is Gazebo included in the Ross installation or do I need to install Gazebo? Believe you need to install Gazebo separately because it's a separate package. Let's see, other questions. Let's see, any, oh wait, is this more? So what hardware, okay, so there's this question, what hardware is required for beginners, especially kids? So slowly, and some of those things I recommended at the end, so we're really approaching a place where you can teach robotics and you can teach software fundamentals about robotics without hardware. And that's the power of simulation, right? Because it's expensive. And it's difficult to deal with and you have this problem in robotics hardware where kids get frustrated because they don't know if the software doesn't work or the hardware doesn't work and like that's an advanced debugging skill. So I actually, I really see a path forward for robotics with simulation being the way that you start teaching. And then once you kind of get good at software or figure out what you want to do software, then you drop it on the hardware and then you go there. So I actually think, you know, especially now with COVID like software is the way to begin. Don't even worry about the hardware, go build a simulation. And a couple of resources, the auto work course I showed you, the ROS tutorials, the DARPA tutorials, they all kind of get to that really, really quickly where it's like, here's a simulation, go play with the simulation first. Let's see, do we still have swag for donations going on? T-shirt sale just ended actually, like last week somebody said, yeah, t-shirts are over, but I think, you know, with the next release, there'll be more t-shirts. Do you hire it open robotics? Yes, we hire, we have to employ people. Are we hiring right now? I think we're a little bit stuck there. I think there's some might be positions at the Singapore office. Is ROS2 a replacement for ROS1? If not, ROS1 has its own use cases. I mean, I think ROS2 is a replacement for ROS1. I think, you know, I believe replacement is not the right word. Replacement, you know, is Python 3 a replacement for Python 2? And the answer is, it's the evolution and software doesn't remain static and needs to be maintained. So yes, let's see, I work on embedded projects that run on custom hardware for fancy PTZ cameras, been there. I'd love to convince our Python app developers how if ROS could be leveraged, where do you think I could start? There are consultants that could help, you know, specific use cases. So I don't have any, I can't actually recommend consultants just because of who we are and what we do. But specifically, but I actually think this is the place that we're missing a lot in and that's the building. So I take the broad, broad view on robots where robots are anything with sensors actuators and control versus the sort of narrow view that like a robot has to be like something you see in sci-fi. So generally, all of these people are talking about IoT sort of applications, kind of robotics is the same thing. So PTZ camera, I would start, you know, it's kind of if you can, if you're comfortable working with some of the micro ROS stuff, I'd give it a shot. I think as a framework for developing concepts around hardware, ROS is really, really good. And I wouldn't even think it was a robot framework more as like a hardware interface framework. Is ROS2, did I already cover this one? Is ROS2 a complete replacement for ROS1? If not, does ROS1 have its own use cases? ROS1 has its own use cases in the sense that you are interfacing with other people's ROS1 stuff and you are building things that are ROS1 that are sort of, I don't want to say legacy that are kind of the state of the world. What we're trying to do is we're trying to, hopefully I made this clear, we are, if you're familiar with Python, we are at, it is in Python land, there is Python 2.7 and everyone had to switch over to 3.5, 3.6, 3.7. You know, this wrapped up this year, people really started pushing about 2015, 2016. We are, the analogy is ROS1, ROS2, we are at 2016, 2017, you need to be thinking about it, you need to be building around it and it will be worth your time to do so. Is there a significant demand for parallel kinematics in gazebo? I can't answer that question well, I don't quite grok what parallel, what you mean by parallel kinematics, like parallel robots running and doing kinematics. I need more information to answer that and it may not be a domain I'm particularly suited for. New questions? Anyway, I would feel like you guys can get at me directly on Twitter or we, like I said we have ROS discourse, I'm always trying to get people to come over there, that's sort of where we try to direct discussion. This is kind of strange, I'm disembodied, I can't hear anything. I'm just kind of looking at questions. Okay, well I'll give it, how about we give it till, I have 1250, why don't we give it to 1252, any other questions? I hope I, you know, I'm not writing them out but hopefully you guys are getting your answers and feel free to, like, it's just cat at open robotics.org, just feel free to shoot me an email if you have questions. One once, going twice, well thank you for having me.  I definitely, having worked with ROS2 and ROS1 now, I definitely see why we want to do so and the advantages of doing such, oh, there's more questions, I can answer these all day. I've held ROS from the start and this seems like a big junk from the amount of tools that are exposed are big robotics companies like ABB, Moto Man, KUKA, et cetera playing with ROS today. Are there talks of them moving completely to ROS or is that still far-fetched stream? So this is CRT Mori, I wouldn't say, so the short answer is playing with using aware of yes. I wouldn't phrase this as like a all or nothing thing. I think there's room in the world for a diversity of manufacturing and just my experience with manufacturing, I think there's a slow march towards ROS. I don't think it's good, there's not going to be a day where all of a sudden everyone switches over. I think people are becoming more and more aware of what we offer in terms of making big strides and big improvements in robotics. That said, robotics is a very capital intensive long life cycle thing in manufacturing and so that process is very slow but I think there's a lot of curiosity now and I think there's a lot of realization that we're the set of people that are working on this domain of bringing high-tech interesting stuff out into the world. But Singapore roles require permanent residency at least. Do you hire people on the go without any of these restrictions? I'm not HR, I can't answer that question. That's a question that goes to our website. What is supporting effort to port a package to ROS 1 and to ROS 2? I don't think it's that I haven't done it myself personally, so I got to preface it with that. I've talked to people at ROS Industrial and a few other people who've done it. It's not that bad if you're good at keeping your unit tests up and being mindful and if you're on like a sea turtle, it's going to take you a whole long time to get from ROS 1 to ROS 2. If you are on noetic or melodic and you work slowly and you have good unit tests, the process of transitioning is probably not as bad. It depends on some of your packages and where they're at. There's the ROS 1 to ROS 2 bridge which I think helps that process move along a lot smoother. Zang says thank you. Well, thank you. Let's see, Gino, what is the state of ROS 2 for parallel robots? Are you talking, I guess I need more information. Is ROS suitable for small robotic systems like RPI? Yes, it is. Getting things to build on RPIs and stuff like that can be a pain but it's perfectly suitable. The other thing I was going to say about people asking about ROS 1 to ROS 2 transitions, just a couple of hours ago I found out Michael Ferguson who built the URB 1 which is basically kind of like a smaller PRT robot. He found one on Craigslist, bought it and basically moved it from ROS 1 to ROS 2 over a weekend and he's going to talk about it under a Zoom meeting like tomorrow or today or something which I think that's going to be super interesting. You need to move the page if this pops up. What is about certifications on industrial usage? I'm not super abreast of what's going on in that domain. I know there is some efforts that certain organizations are working particularly hard, particularly at like auto wear on certifying ROS in particular for certain applications. Right now those tend to be sort of an organization driven value add. Internally, I think if there were a little bit more push I think that could happen but there's certainly people doing it. We are helping them at times but we're not necessarily doing all of it. I think is roughly what's going on but I'm not completely sure about that. Okay, any more questions? Looks like there's enough page already. Besides be shared, just like should be shared. This one. Okay, I think I've seen all of these. They're numbered but they're not in order. Okay, any last questions? If no one has, I think I saw one more. Would the size be shared? Yes. ROS support for serial manipulators is very mature. How easy is model simulate? Like, okay, a Delta robot. I think five bar robots, okay. I see what you're saying. I think there's still some in Gazebo. I think there's still some limitations on like five bar linkages if I last I heard because I was kind of asking about that a couple months ago. That said, I will say that there's pretty active development on ignition Gazebo just in terms of like souping up both the performance and the capabilities but I don't know. I'm not as familiar on the Gazebo side and particularly like the erative side of if and when that'll be possible whether it's a ticket and where it's at. So that's a little bit, you know, closer to the maintainer down on the wage question. That's a like, that's a ROS discourse question because I don't know that but somebody else probably does. Not, I think that was 26. Here's another one, Karthik Poduval. ROS one, I did not allow camera metadata to support that. I don't quite understand the question. I mean, there's a metadata topic more or less. I don't understand the question. Okay, I'm not seeing any more questions. So I'm going to assume we're done. Give it. We got a question here, Eric, even if it's imperfect. Okay. Well, I'm going to strangely not getting any feedback or hearing anything. So I'm going to say I think we're done. Appreciate you guys showing up. Let me know everything. Most of the things are open. Check out all those resources I pointed out. I believe I sent over the slides. They should be able to point you to them. If not, I'm going to happy share them with you. Cool. Have a good day.


Okay, it looks like it's time to get started again. Thanks for rejoining us. We have two really interesting talks coming up in this segment of virtual meeting. And we're going to start with our colleague Ken Klingenstein from Internet2. Ken is well known to many of the folks here and has had a tremendous gift for identifying issues around privacy, identity management and security over the years. And is back with some really challenging I think questions. Ken has told me he really would like this to be quite interactive so feel free to raise hands, use chat, whatever, as his presentation goes along or as at the end when he calls for questions. And with that welcome back Ken over to you. Thanks Cliff. Yeah, my, my goal would be not to finish my slides today. Just to have it be that interactive. We'll see how it goes. So there's a number of issues on the landscape that are emerging, and they all need answers. And I guess my greatest confusion at this point is who's going to be able to answer them. Able in terms of knowledge, able in terms of where the people who could answer this sit in the ecosystem of trust and identity that's out there today. We have lots of players, many of them will raise their hand and say, ah, we can fix that. I'm not sure they're the right authorities to do so. I'd love for there to be high levels engagement on this so through the chat session or raising your hand. Please interrupt me. We're going to touch on the following points and we're going to talk a little bit about trust privacy and portals. That's the most vexing aspect right now in the landscape that I see in the ecosystem. Well, then look at course site tracking. This is a very active area with Google, among others, really trying to address the situation about minimizing cross site tracking while preserving business models attribute release control. The federated environment is built upon release of attributes from an identity provider to a service provider and that will enable access control and other kinds of capabilities. And then we're going to be in charge of the release of that information. Basic identity itself. You know, it has traditionally been anchored by governments increasingly in a decentralized environment. There's new ways of creating identifiers and vouching for the validity of the association of that identifier with a particular subject. So that's in question. The legal and regulatory safeguards that are out there are frankly a mess frankly a mess in the US. I don't know what the mess is and then I have a blank slide for who's going to fix that. And then I'd like to if we have time cover some of the things that I might have missed in this that I should be talking about the next time I do this presentation. So in the federated ecosystem middle things have emerged in the trust system, and they profoundly alter the end to end trust model for federated identity. The model itself was that an identity provider would trust a service provider and vice versa. Now we have all of these portals and proxies out there that intervene in those transactions and what's their role. In particular the research and education environment is right with proxies and portals. And sometimes to translate authentication approaches from let's say federated to IP address that's a common one from easy proxy to complex composite sites integrating distributed disparate entities. The dialogue on is a very important one in the science community. NIH is in the process of building a portal for the National Institutes plural of health. And one in particular has raised a number of concerns, because placing anonymous access sites like PubMed behind a portal which is oriented towards identity and strong identity has caused a lot of stir about. How do I get to PubMed with anonymity when the portal is asking for identity. Elsevier has science direct out there as a kind of authentication point for a variety of services, some of which support anonymous and pseudonymous access, some of which like pure and identity. Easy proxy, despite its name is real is is or is really a gateway to a lot of stuff where you have to provide identity and then those resources behind there might well not want to know that identity. How do we do that. The browser itself is becoming a middle thing I'll come back and talk about the browser specifically in terms of course I tracking. But the browser has become essentially the operating system for the web. And so we're seeing a move of functionality from external places into the browser. And that becomes very consequential for who are the power brokers, and this complicates privacy and security, the middle thing can sit there, see the stuff that's going by, and modify it. And so, how do we protect our lives and our privacy with in the presence of middle things, and most conspicuous leaf. We don't even know where to start the conversation. There is no reference framework. There is no model out there that says that defines the functionalities of proxies and portals, and perhaps categorizes which functionalities affect privacy which ones affect security etc. Who's going to answer this question. Who's going to build the model. Well, middle things are to some degree, a creature of multilateral federation in that bilateral world that industry would like to see. There really are very few middle things out there. But in research and education. Many, many services, many, many providers operate middle things again CI log on NIH. And so we might be the ones to answer it but we're not. We're not even building that framework. There are standards organizations that could build this a cantara is a standards organization that does levels of assurance and other kinds of standards is international. They have no framework. I just had a conversation a few weeks ago with NIST. And in a very different world, they have portals that they're trying to deal with. It's tempting to throw this question over to NIST. However, the lights are barely on in NIST. A couple of years ago, during the Trump administration. NIST got eviscerated and they're just now we rebuilding the resources to work on this. Typically marketplaces often decide these questions and I fear that the marketplace may decide this and the marketplace seldom favors privacy. Cross site tracking. I suppose most of us have had that experience of doing a presentation and pulling up a website.  And that's awkward. That has actually raised a number of sensitivities. And we can look at how to ameliorate that. Cross site tracking unfortunately has a variety of techniques that are used third party cookies are the most obvious ones but there's link decoration and bounce tracking. And what's awkward about this is that fine and upstanding services at least in my mind like federated identity winds up using link decoration to convey information about your IDP to the SP and vice versa. And so some of these techniques were actually invented in the RNA space, but were abused in the marketplace. And that perhaps happens frequently where we invent stuff for the most noble of reasons, without understanding that Oh my God, the marketplace and advertisers will work on that. And because some of this cross site tracking is about to be addressed via a number of initiatives. There are new companies now springing up that are doing analytics, and using a lot of artificial intelligence to begin to find ways around whatever solutions were about to invent. So who will answers. Well, browser manufacturers, since this cross site tracking is happening largely via the browsers that they're offering to solve this, a Google in particular, if you're not familiar with the browser domain, there's very few independently coded browsers in the world anymore. Fire Fox is one, I think opera might be, but many of the others are built on a single code base called chromium, which Google made open source, and then built chrome on top of. So a large degree I'd say almost 75% of the browser space is dominated by chromium, and it's its derivatives. And so Google feels like they're the right ones to address this. Since they have a web browser to do this.  So Google is offering to do this and there's a working group that's been operational now for about six months. Heather Flanagan who many of you may know is trying to facilitate it that group. The issue there is that the third party cookie deprecation that Google is proposing to reduce cross site tracking breaks, many other things that are unintentioned. And so the question is how do we ameliorate the damage that Google will do in trying to minimize cross site track. Who else what could answer it advertisers can answer it. Well there's Google again. Third parties, the ones that are doing analytics governments could also begin to address some of the cross site tracking issues. In the structured environment in the US that's not about to happen in the EU. You've seen a number of initiatives by the European Commission, all of us have experienced the new cookie paradigm. And the sites in that are based in Europe, we're given a choice of which cookies to accept. I wonder about that capability within this space as well. I'm attribute release boy I've stepped into the middle of this. Attribute release was again a major capability of the federated environment. We were supposed to have some consent modules developed. Some of us are still working on that. To some degree, the authority for that attribute release is passed to identity providers I'll come back and and and release and talk about that in a second attributes are really important for access control personalization and customization. You want to release sufficient information to gain access to the content you'd like, but you want to release no more than that so data minimization is an important feature for this. Metadata is the vehicle by which relying parties can indicate to either users or identity providers or two librarians, what data might be needed. And one particular gnarly aspect of this is purpose abuse users identity providers might want to know from releasing this attribute. What's going to be used. What's going to happen when you're done with it. The need for normative taxonomy for the R&E has percolated up for purpose of use. You've seen it as I indicated in the cookie paradigm that the EU has promoted. There's just for three or four different purposes of use. You can create something in the R&E space that might talk about your releasing information. Here's how we intended to use it in a classical taxonomy. And here's how we intend to dispose of it at the end. Who's going to answer this. Well, boy, identity providers answer it today. Your idea your federated identity provider typically says what attributes are going to be released about you. That's not how we designed it but that's how it's, and that's how it's rolled out governments can do this. We designed this for users to be in control with some kind of consent module. Even the word consent is a tricky word in that the way it's used today. It's kind of like someone else has made a decision about what's going to be released. Can you consent to this. How about control. What user control versus user consent. Wouldn't it be nice if we were in the driver's seat as users. When I mentioned this to librarians, a few of which are on this call. They say, well, you know users may not know what will reduce the friction that they will encounter in getting to the content they want. There's a group called FIM for L. If you're in a librarian who's interested in federated identity I urge you to do that it's a part of Lieber. We just had our FIM for L calls this morning. Very good people on there, largely Europeans we could use some us representatives. But there's been a perspective to date that librarians really know what should be released because users don't want to be bothered. I'd love some validation on that. Oh, I see. I'll have to come back and do that comment. The decentralized identity paradigms. We'll talk in a second about how technologies have their inherent challenges relying parties may want to decide what attributes get released browser manufacturers. One of the things I saw recently was a Google mockup that had your browser, presenting you with a list of options for which attributes could get released. That means the browsers watching things more closely than I'd like the browser to watch. Clearly, if you're doing encrypted transactions we can avoid that but most settings out there today are not around that. We have device manufacturers well there's a Google again because of Android. My God Google's all over the place on this one. No surprise communities of interest. This is a new idea communities of interest can certainly define taxonomies for purpose of use. And perhaps that's going to be an answer for this. This segues into the comment that Pascal just posted to the chat session about the distributed identities. So traditionally identity was anchored via an identity provider using a government document and the level of assurance that us geeks in security talk about is anchored by what kind of documents that we use to prove you are who you are and then we'll see that identifier will create a level of assurance around that all of the mumble jumble that we talk about. There's pluses and minuses with taking an identity that's anchored with the government and decorating it with attributes from a lots of different sources not necessarily from governments. So going out watching the terrible situation in the Ukraine today. A lot of people don't have governments. And so what is going to be the source of the official documents that would anchor identity. This threats, that's me do a great deal. One of the solutions would be self sovereign and decentralized identities. Sometimes these can be anchored to larger trust chains, large, large amount of the time it's anchored by a reputation system. I have 400 friends assert that I am who I am to the distributed identity provider, and this binding of that identity to me is anchored by a group of cohorts. Pluses and minuses. I would mention that sometimes technologies have fatal flaws, because of who's promoting the technology. I've been to a number of decentralized identity conferences. And what's tricky on this is that everybody has a different version of decentralized identity, and they don't necessarily want to cooperate, because it's decentralized and that's their nature. So these are individuals sometimes looking to make a market share, sometimes it's just their nature. I think the federated case that I've been a part of for 20 years, has the virtue that we wanted to work with each other. We weren't trying to create an independent standard, each in our own right, we wanted to federate and work together. So I mentioned that in terms of decentralized identities, a interesting concept, again can tie the reputation systems. But is there something intrinsic about the people promoting this that could be a problem. Who will answer this. Well, governments have traditionally answered identify the basic identity, but again, governments are sometimes fragile marketplace forces. And then I'll just point out that we have a lot of new data these days that we don't want to look at, like surveillance. And so the idea of people talking in the open field is no longer open. On the other hand, that surveillance data could be a great anchor for some kind of trust based upon, well, we saw this this person at that location. And so we have an association that way. So that's a way of taking surveillance which I think of is a great concern, and perhaps tying it into creating some identity systems that we wouldn't have had in the past legal and regulatory issues. So GDPR enforcement is now finally taking place finds are being leveled. Reciples are being developed by the Europeans that indicate the nuance that GDPR requires it's a very sophisticated set of requirements. And I'm not quite sure that the marketplace is up to the nuance that's embedded in GDPR shrimps to has added its own wrinkle in this and put the US in a kind of an awkward position because our trustworthiness is no longer obvious. We don't have the recourse that the EU would like to have, how do you begin to implement things like right to forget in our fractured environment. The UK has its own version once they left via Brexit. And it was actually the UK researchers who raised the concerns about PubMed being situated behind the NIH portal. In the US it's pure chaos. Most states are trying to develop privacy approaches, very different emphases, very different recourses, and many states don't need no stick in privacy. Somebody want to raise their hand at this point and tell me because it's not working. It's very clearly not working but I sure don't know. I don't see any, I don't see the federal government stepping up into this place with the polarization. The state governments have different masters. It's going to be tricky. Many transactions are interstate what state would apply to Google. Yes, Google. Thank you. So let me just close with this one and then hopefully invite some conversation. A lot of this stuff, a lot of security, a lot of privacy depends upon encryption. Encryption is reasonable at this point. Quantum computing will break most of the encryption algorithms that we have today. NIST and other places are hard at work at building encryption algorithms that might be resilient to quantum computing, but we've got to be careful that encryption which underlies much of this stuff is itself fragile. And then reconciliation of national laws and now state laws and where the data will be geolocated. I'm seeing a lot of work by a lot of different organizations to make sure that their data is being stored in a place that's benevolent to whatever privacy views that organization may have. Lots of problems. No answers. I'll stop at this point. I'm appreciating the comments. Oh, Joseph Glass. I don't know you Joseph but that's an interesting idea. So maybe you want to raise internet to Thank you Tara. I'll just mention that we we created part of this mess with the federated identity space. We intended it to be deeply privacy preserving. We had from the very beginning, the idea that there was going to be a consent module out there that was going to enable users. The consent module has been slow to develop. There's been a belief. I'll say it in arrogance on the part of identity providers that uses can't manage attributes. And part of this is because the error message that we have when you don't release the right attribute attributes to get to a resource or error message is the wonderful 404 resource not found. There's a few places out there that do the right thing and say, ah, we would have loved to give you this resource but you didn't release these attributes but for most relying parties. If you don't release the right attributes, you get a really opaque error message. We're working closely with Elsevier and I've got to say that Elsevier, we're trying to develop a notion we call agile privacy, and maybe in the fall. We can all at the next CNI in person. We can talk about agile privacy and and how well it works, but Elsevier gets it to a large degree about how depending upon what you release will give you anonymous services pseudonymous services which will give you a the ability to keep your search histories but without us knowing who you are, or we'll give you personalized service based upon who you are. So I would say almost, you know, we're involved because we create a part of the mess. I have an Instac grant that we're still working on many years after Instac has passed called scalable privacy and we have a wonderful consent module that came out of Duke University that we're trying to promote these very days to get it out there. Let's see Cliffs might enforce federal action. Anybody watching the congressional hearings for the new justice has got to be pessimistic about anything happening in that in the at least the congressional sense. So would the, what federal agency would cover this FCC Federal Trade Commission, it's not even clear what agency would do that many of the agencies are suddenly owned by commercial sectors so if it was going to be from the executive branch cliff I'd worry which agency, if it's going to be from the congressional branch. I would wait for a long time if it's going to be from the judicial branch. Ooh, enough said. Appreciate any questions. And on that happy note. Oh, I think we're at time and I know there are more questions that people would love to ask. That privacy agile work is very interesting and I do hope we'll have an opportunity to hear more about that as as it proceeds forward. Love to we have we have some stunning demos and you know it illustrates that even in the best intentions publishers like Elsevier. They're wonderful they they they give you anonymous access and then they say, Oh, but we need a place to hold your search histories give us your email address. We want to know how about something else, we have better identifiers so we'll we'll pick that up. Maybe we all get together in the fall. I'll leave it at that. I look forward to it. Can I know there are many more questions I'd like to ask there are many more questions I know that our members here with us would like to ask. We need to move on to the next presentation but I hope you're able to stay around at least for a little while and maybe field a couple of questions in chat from people. I think you very much for yet another very provocative presentation and really appreciate you joining us today.