Yeah, so hello and welcome everyone. So this session is about a generic multi-prog API and the first user for it, which is TCBPF. So I call it TCX, where we want to implement links for it. So this was like a longer term goal. And then as a second part of this session would be an update on the meta device thing or for those who haven't heard about it, what it is and why we want to add it to the kernel. So let's start first with a generic multi-attach API. So the goal would be to have a generic reusable multi-program management API so that this would be fitting for longer term, given we see more and more projects using BPF in the wild. And therefore like a single attachment hook in some of the cases is not really adequate anymore. So we want to basically have a way to attach multiple programs and to have the same look and feel for different consumers of this API and to be able to express dependencies between programs. So back at the Linux Plumbers Conference last year I gave a talk about Syllium's BPF kernel data path, ref-empt, and there's also a corresponding patch set on the mailing list if you want to take a look later on. Basically the TLDR in the patch set was that we reworked the TC BPF layer. We added a new fast path and also the management API for it to make it FD-based so it wouldn't use netlink and Q-disk and all of that. And with that, it would also be the opportunity to add BPF links for this because this has been missing for a long time. And we want to add it because from the Syllium perspective, we've seen that some of our users run into cases where Syllium's BPF programs got ripped out underneath by others by accident. So yeah. And with this rework, I basically also added the attach-detach query link creation API for TC so that you can basically use the proc and link file descriptor together with the priority. Priority was used back then because it was the same SVTC, but then the feedback rightfully so was, can we challenge the status quo, as Alexey mentioned yesterday, and can we come up with something better because priorities are hard to use? Users don't really know what to pick and so they might all pick the same, which is what happened in our case. Yeah. The other feedback was also to, just as a side note, that's why I mentioned that TCX here to basically name this layer slightly different. So it's called TCX, basically. So in terms of alternative directions to express dependencies, one of the areas we looked into was SystemD because they have this already for unit files where you have before, after dependencies that you can express and based on the unit file name to have this different ordering. So it would be nice for the BPF side to have an idea. This was basically one of the feedback from the discussions on that patch set. And especially also from talking to different people, for example, like Andre from Meta, there are multiple cases where you have management demons for BPF. So it would be super useful to be able to express the before, after dependency for this case. So the initial design with that that we converged on from the discussion was basically to have something like this, so that we add a couple of flags before, after. And you should be able to specify an FD or an ID. Usually when demons want to query, you get all the IDs back. So it would also be useful to have an ID that you can point to the kernel when you load program that you want to have this next to that given ID. And this should be useful. This should be working for programs and links. And in order to toggle between those two, there is an additional BPF ID flag. And in order to toggle between program and links, we have a BPF link flag. And then on top of this, also first and last, so that you can basically specify, I have my DDoS mitigation. This should be guaranteed to be the first in the whole processing chain, or I have my monitoring to see what kind of traffic, for example, is being pushed into the node, so this needs to go last. And then you should be able to combine all of them as well. On top of this, so the implementation would be for the proc-based attach and detach API as well for the links. And from the query, we should be able to have a revision counter that we can also pass in when we attach something. Yeah, that's a question. So that you can basically assert that when you attach it that this internal state must be that specific revision. Yeah. Just a question on the first and last. Is this a requirement, meaning the second one that tries to say it's first is guaranteed to fail, because they can't both be first? Yeah, exactly. So it's a first come, first serve type of well. And you can even come, and in what I will show later in the examples, you can also combine those. You can say it needs to be first, but it also needs to come before that specific program so all those flags are combinable. And when you say first, last, well, then there's a, if you do that, then it would be the only single one. And so before doesn't mean immediately before, you just mean anywhere before, is that what you mean? Or is it immediately before that specific program? OK. And so as you said, you just answered my other question. If you say first and last, that means I must be the only one of the others. Exactly. Or fail if you can't meet that requirement. And the idea would be like to have this whole thing as an layer or as an API that all the other consumers, for example, the TCBPF, but also future XDP, various others, the C groups that they can just use it. And it would be the same for all of them. Yeah, so basically it would look like this. And just to walk through some examples, like that would be the simple. So here is the implementation for LibBPF when you have the link creation API. So I'm just showing this here. So basically you have as an argument the specific program. So there are two new sections for TCBPF, the TCX ingress and TCX egress. You pass in an if index where you want this to attach. And then you have a flag and a relative object argument. And if both of them are zero, it would just be like an append case. You would just append the program next to like into the list of programs. If you would say BPF before and BPF ID, and you would specify a program number two ID that exists in that area, it would basically make sure that it attaches right before that. If that doesn't exist, it will bail out with an error. So that's basically where you can assert that is the requirement that is my intent that I want to add this. Or for example, you combine them, BPF first, BPF before. And this time a program FD and similar. So now it will make sure that the program that you attach to, like this from the skeleton, it will always be the first. So you said if it doesn't already exist, then you'd fail. So my question is, is there a race condition? What happens if the thing detaches at the same time as you're attaching? Is it non-deterministic then depending on which one you process first? So that is basically locked with the RTNL so that there's no such race condition. I didn't understand that. Maybe we can tuck it off of line, but I didn't understand that. Yeah. Now what? Let's say A says I must go immediately for B. That completes. B then says I'd like to detach B. What happens to A? So the first request, that is basically serialized, right? So the first thing. I understand it's serialized, but when B detaches, does that put A into an invalid state? Because A has to come immediately before B. No. And that is just for the point where you attach at the time where you attach it. But if you're saying A must come before B, and if you're not going to start up A unless B already exists, then you can still get into that state if you don't. So it seems like you're treating B before as a kind of a permanent condition. It's not. It's just like for that particular attachment. It's like immediately before that as of now. And if something changes after that, that's fine. That's sort of the idea. Because otherwise it's very expensive to keep maintaining this invariant, right? I understand that answer. It sounds like the model then is to say if it fails, it's because something must have just changed. And I kind of repeat attaching in a loop or something because after then reread what's the state and then see what I'm going to go before now, right? That's your intended model, right? Is that if there's a failure, it means that there's something that hasn't hit steady state yet and just keep retrying, or you could, right? You're not forced to, but you could keep retrying to say, well, then I want to be after it right before. The next one that would have been after that. Even better, you can actually specify expected revision of the attachment.  So that's the expected, whatever expected version or something. Yeah, yeah. All right, another example like you would go first before and then you specify a link in that. So given like in the sense of time, so like, yeah, I think it's quite flexible in terms of what you can express here. Then we have the first and last, which case it will be the only one in this case. And yeah. Then with the revision, I added like a second API to LibBPF, but we can discuss it, whether that makes sense or not from your point of view. I mean, I wanted to keep the other one simple in case people really don't need the revision thing. I mean, if we follow the API design of LibBPF, I would keep the program and if I have index as like arguments and then everything else is there opts as truck. Yeah, that's fine. I can do that. Then you just have like one API, which is optional stuff can go into a truck. So if I understood that right, an advantage of the revision stuff is if you're in a particular state and then two apps are trying to do an attach at the same time and they both say before the same one, right? Then there's a race conditions, which one goes first where revision would prevent that too. So maybe revision is actually the more recommended one. I don't know. I mean, if you care about strict ordering, right? Okay. So that is basically where you can program it. In terms of the UAPI extensions, so for the query UAPI, well, for networking, I added the target if index, then like there are two like two arrays that we add that we extend this one is like a link IDs and then the link attach flag. So basically when you would dump the internal state, you get the number of proc IDs. They're always filled when something is attached. And then you get link IDs in addition, which have the IDs of the links obviously as the name says, and they are being filled and otherwise zero if there's no link there. And then you have a proc attach flags and link attach flags. So that is quite flexible if you have proc specific flags only in the future. In this case here, like the first and last, like those two flags that need to be permanent. But yeah, so that is how that looks in terms of the flag extension. What I mentioned earlier, I think I can quickly skim over that attach detach UAPI. So that's quite similar. We add the if index for networking as well. And then just the union for the relative FD and ID and then the expected revision. So it's not that big of a change to the existing BPF attra, the link creation. So here, a lot of the parameters are already like from the generic common fields that we can reuse. And the only thing is given, yeah, here we need to add this to like a link specific section. And in this case, like the TCX because we cannot attach new fields at the end of it. But yeah, that's just what we have today. So you have to extend it in this case here. For the internals, so basically you have this BPF M proc concept. So it's basically like an array of items that you want to iterate to. I made this an area instead of lists so that it's more cash, so that you have better cash locality. And the internal state, what we store here is basically the proc pointer flagged for flags that need to be persistent and then ID in case of the link ID. And you always have a pair, an A and B pair, that then when you do updates to the internal state, you would populate the opposite pair for this area and then swap it out. So that you don't need to allocate memory on detach so that this doesn't fail. So that's the implementation here. And so yeah, like that is basically being added for TCXpress as a first user. And the way we run or like the way we execute this in the fast path is basically to just walk to the area whenever it says next as a verdict. Then we go to the next program and then at the end there's always like a null entry so that we exit out of there. And if you look into the sketch handle ingress or sketch handle egress, so that is basically something that I reflected a bit. So you have the static key where you run the TCX programs first, what you can see here. And then if we have a verdict where we don't stop there, where we stop the pipeline, we can directly go to the individual actions or if there's a way where you need to be able to collocate both like the new style TCX versus the old style TC programs, you can say from your TCX program to do the TCX next. So then it will jump into the whole Qtisk Sealers Act Qtisk. Daniel, Tok has a question on that. Tok, go for it. Hello, can you hear me? Yeah, we can hear. Cool. So about the Multiprack API. So first and last flags are permanent, right? Yeah. Why is that useful and what happens? How do you resolve conflicts of two programs at them? I think, well, why is it useful?  It just seems like someone will just grab it and lock everyone else out and then you will have the same problem again just in reverse. Yeah. We are thinking, I mean, this could be extended further. Like if you, for example, have just some monitoring use cases where you don't actually change the state of the SKB or whatever and you just want to have the inspection. Maybe we could add another flag where you have like a read-only and then you would also be able to attach into this first and last as long as you don't change the internal state. It could probably be something we could think of. Another option that we also considered for libHTP but haven't implemented yet is some kind of admin override where you have an API to clear the flag if something is misbehaving or override what the program says for these persistent flags. So I didn't cover it here but like for the replace case, that should be an option. Okay, right. And the second question was with this as far as I can tell, it's not possible to atomically replace multiple programs at once. Yeah, that's correct. I mean, well, you have to, well, yeah, you cannot. Like the only other thing is if you would expose the whole area but like when we discussed this, like, yeah, I think it makes things more complicated and error prone as well, in my opinion. I'm kind of picking up on that first last question that Toke asked. It's interesting because it ties into what Dave was asking with like, what does after or before mean? Like first and last are very different than after and before. And if we remove that, if we said first and last are also just a concept that applies at attached time, not always, then that wouldn't make the API much more consistent, right? And you're always gonna have a conflict in this thing of saying, oh, we need the API to mediate what's gonna be the first program or the last program, I think is basically the wrong approach. Like you need to do this in your user space management layer where you're saying, well, the DDoS thing is the most important and so I should start that first and then the other things come up or however you do it. So I think I kind of agree with Toke in saying that the first and last is maybe a bit special. So it's, okay, so two things here, right? Like first and last year they are special, but like they are important, right? Because like for example, when you think about LSMs, right? Whoever runs first can like reject everything, right? And like if correctness of like the overall system depends on like your BPF program to be run first, that's a hard requirement. And if it doesn't, that's not satisfied and like you cannot start and function correctly, right? So in that sense, first, last is actually yes, different than before and after. But I'm wondering if people are actually confusing the first and last as like a prepend versus a panned. You know, a prepend, a panned versus like really first and second because like for, if you want to add something as of right now to be the first one, but like you don't care if someone like attaches after that before you, then you just specify before and like object zero or something like this, right? And that means prepend, similar like for a panned, you say like after zero, zero means like no FD, right? But that means just like after or before anything, right? So in that sense, yes, first and last, I think are important for correctness and they have to be persistent because like if you don't persist them, right? And like don't enforce them then like what does it even mean? It's basically prepend, a panned only, right? Which you can express with before, after. Yeah, that was why I asked why it's useful. So I think this also comes back to the discussion we've had before with like the difference between a system where you control in all the programs and you can have this kind of policy and the system where you are loading third party applications that just pick flags for themselves and some random developer has said, I wanna be first and then they lock out everyone else. So like and that's gonna happen so there has to be some kind of override, I think. If they're useful for a use case like I'm okay with keeping them as long as you can sort of admin override them in some way. We also talked about having a hook that a policy demon can hook into and reorder programs but I guess that can be added later as a way of doing this. So I think I am somewhat partial to Toki's argument because we've seen conflicts and before and conflicts and after and other things not just BPF programs, right? You know, firewalls or things that oh, I need to be first and something I'm ready to BPF, the pre-dates BPF certainly on the Windows side and I think on the Linux side too and so it gets to, you're going to have collisions what are you gonna do and the answer right now you're giving is first come, first serve and everybody else loses, right? Separately I think there's an interesting thing to Lawrence's part that says, and Andrea you mentioned will people do the accidentally use first when they just mean I wanna be the first one on the list right now? Maybe there's a case for having a flag that's currently first separate from a flag that's always first, right? Because right now it is before with zero, okay. It's before with zero where the zero was the magic no more that just means I'm first on the list. Okay, that wasn't on the slide, thank you. Okay, so for the like, I mean we are talking about like end goal is like some production system where like you know maybe multiple systems try to coexist, right? So like yes, definitely there should be some sort of agreement like gentlemen's agreement that like if you don't really need to be first like you shouldn't specify the first, right? And like if you do because you don't know that's a bug you will report and someone will fix it but we have like for the force detachment that has to be kind of a human or like Capsis admin like enforcement and like detachment and all this stuff and like we have link detach, right? Like where you, it's slightly different because link detach leaves the link in place but it detaches the underlying program so maybe we can reuse that or like have similar approach here but like you can forcefully detach even if it like says like first or last, right? But it's more about like forceful detachment than like the ordering. You should be able to force detach any detach thing if you are admin, right? I think it's just what we're talking about. Okay, then a final point, maybe the first, last is a bit like as we said like people interpret it differently maybe it needs to be like always first and always last, force first, I don't know whatever yeah, Henry said that, I don't know. Maybe it's the hardest, yeah. Okay, always first currently before is what Dave Fahala says. Okay, just add a bunch of syllables. I have to move on a bit to not over shoot too much in terms of time. Sorry, I have some additional questions. I'm sorry. Like for this API you said you take both IDs and file descriptors and you take both like programs and links, let's start at the IDs, why does it take IDs instead of file descriptors? Cause all the other APIs we have as far as I know take file descriptors, right? So when we discussed this a bit, I mean like the rationale was like, well like that management demon that would consume this API it would query and all the time when you query you get back IDs, right? So then you have this additional hurdle that you need to get the file descriptor from the ID and then so you could just as well. Is that a hurdle? So yeah. I don't know, is that really a hurdle? But I mean that daemon that you want is not going to be kept. So Andri says getting the FD from the ID is capsis admin. So you need capsis admin to be able to specify the other thing, isn't that what you will want? Well the idea is that like it's not always like one centralized demon, right? Like it could be multiple independent applications that you just have like they know about each other potentially, right? And like we just don't have defined order like in which they start up, right? But like when they start up they can actually go and somehow discover they're like, okay some other program is already running and I want to run after it, right? And then they will just like query ID, maybe like ID is stored somewhere on the file, right? So like they don't need capsis admin if they can specify this ID. They can get these IDs through some other means, right? Okay, then the follow up is like do we need to add IDs to all new APIs or is this, I don't know. I'm sorry, but it kind of begs the question, right? It's like. Like if we do this for all APIs. Yeah, I mean like in the future like do we, like I mean like the idea was like for this API to be able to support this in different attachment points. So in that sense, yes. Okay. All right, moving on. So the meta device for BPF, the idea is basically overall for the Selium use case that we have, we really want to have the same performance for applications inside Kubernetes pods, AKA network namespaces compared to applications residing into host. And just because you moved them to network namespace shouldn't incur a performance penalty. It currently does. So we did some performance measurements. The first case, what you can see here like in turkeys is basically the reef and the upper stack forwarding. The next case is some of the improvements we did in the past with the, so we call it BPF host routing but it really means like to use the BPF redirect here and BPF redirect neighbor for traffic going in and out. And the last one in this comparison is basically the baseline, the host performance. Why does this suck? In terms of the, yeah, like when you go up to the stack the socket is basically, the SKB is often from the socket and that basically breaks TCP back pressure. And there were multiple attempts in the mailing list to remove it but it all boils down that it seems to break net filters T proxy, so it needs to stay there. But that is the, yeah, I think that is the main cause why the performance dropped so much. Like in this case, basically the socket is retained all the way to the physical device and it stays there in the Q-disk if it has to and then really only TCP inside the pod gets the notification and yeah. So the question is can we get to this point? And so yeah, so we added a new device driver as a ref replacement. We call it matter but I'm still open for other names if you want, like if you have good suggestions, good short suggestions, naming is always the hardest but I thought because it can mean a lot of things depending on what the BPF business logic in that driver would do. So basically the core idea is like for traffic going out of the pod to shift like to move the BPF TC programs that are currently attached to the on the TC Ingress on the Weave site into the driver itself so that basically that BPF is executed at the XMIT routine in the device. And then what we can do there, we switch network namespace immediately, then we can do the fit block up and when we see that it's going to the physical device we can directly redirect it from there without going through a per CPU backlog queue. So that is basically the core idea behind that. Question is yeah, what about XTP support because we first XTP do we also need to add it here now and I'm saying I really don't want this because I mean XTP if you look today into the weave code it takes up three quarters of the weave code. So it's really super complex. If you want to use XTP just use weave. But really the idea is like for the Kubernetes use case I mean like well XTP in the physical device that definitely makes sense and after that you get all the GRO batching for SKP and that is absolutely fine. And in terms of program management we would reuse the same API that I talked about earlier. It's still like a main and a peer device so you have two devices and the idea would be only the main device that you set as main device it would reside in the host namespace and only that can control the program management for both devices. So you can only update the peer device BPF program from the main device so that no other entity inside the pod can somehow detach it. And the whole device would be an L3 device so that you have it as a no ARP device. L2 mode could be configurable but it wouldn't be the default but it would still be useful I mean when we talk to some of our folks that do BGP with Scyllium I think it's still useful for testing.  And the idea would be to have compatibility for TC BPF so that you can then move the programs from TC ingress weave into this device type. And if you look here into the X-MID so basically here we switched network namespace and we execute the BPF program and if the BPF program says redirect you redirect directly to the physical device without going to per se pure back lock queue. And if you look into flame graphs so what you can see here on the right side how it looks it's all done in the process context so there's no rescheduling point as in the case for weave when you really have worst case scenario. And looking into performance this is basically where we get to so we get really on par with the host same as like that's for the TCP stream case 500 gigabit same as the TCP or R where we get the lower latency so it's really on par. So yeah in case of Syllium that's basically one of the last missing building block that we want to get to. So we basically here we have the bandwidth manager with FQ EDT where we would then also be able to support BBR from the pods. We do have this BPF host routing where we like for inbound traffic do this redirect peer for outbound traffic to redirect neighbor then we have the reef device replacement so that we really get to the same performance as I showed earlier in the graphs and then the other thing is like big TCP that we enabled but even with big TCP we get lower latency so that's what we measured here as well. So yeah that is the proposal and one small open question I wanted to discuss is that given that's coming as a module like I think that there are two options I think like as far as I've seen the net filter BPF basically has this Boolean to configure it when the system call is enabled we are able to create links for it or whether we should have some kind of registration API for this device so that then we can have all the logic around like this BPF Mproc handling inside the device driver and then when it loads it just registers to it and then it will do this NDO call to delegate but yeah I can, I'm leaning towards the letter and I would try out that for implementing it as a next step but as far as I recall, network namespaces is not a config option and these is so built in the kernel with net and s well built in the kernel with net includes net and s but then these, without these this is completely useless configuration. Yeah that's true. So it was sort of trying to do the same if you just make it depend on net and be a part like don't make it a driver like why does it have to be a driver? Like these didn't have to be a driver. All right that's fair yeah okay. It's like crucial part of like net and s net and s is not the same thing here it's like it's core it's not a driver. Yeah that's good point. Yeah that sounds good to me. Last slide so basically the generic multi attach API so that is pretty much all of it is implemented and working so I'm still adding a bunch more test cases and make it nice ready for upstream submission and then once that lands that's also the prerequisite for the meta device so that's like the next step after that. Yeah and then I'm thinking to look into adding this multi attach support for XTP as well so that we have this for both TC and XTP. Yeah. We want this for C group role as well and like maybe for like LSMs I don't know like pretty much for everything like even where we have multi attach right now like we need more control over like the order. One more point about like the what if someone attached was sorry going back to the first part the first and last right maybe we should add another flag like force and then if you have Capsus admin then like you can go and basically detach the existing first or last or something like that. And another question I have like when you go to the very beginning right like with Mproc right like you're using this Mproc at runtime to like iterate over programs. If you want to optimize for cache right like why would you collocate the program pointer with the flags like you don't need flags at runtime. I would split them. Just for management yeah. And also I don't know if that's good that like we just allocate the maximum lens of the array. We potentially will be wasting a lot of time right there. Yeah.  Yeah. I think that could still be. I mean yeah. I think we can still yeah we can it's not like you API where it's baked in forever. I agree. Cool. The question on the meta device. So you mentioned if fall to the you then you will you will directly do redirect without going to the bad log queue. So how about for the packet going up to the hoax. So that is already the packet going up to the host. Yeah. That would go to the back lock queue and that would take like the same path as the we foot. Cool. So I don't want to overshoot too much. So thank you very much.


The study investigates the transmission of bacteria from radish seeds to seedlings using individual strains and synthetic communities, syncoms, under in vitro conditions. Syncoms were composed of highly abundant, subdominant, or rare bacterial seed taxa. The study found that stenotrophomonas rhizophila became dominant on seedlings, but most strains had variable transmission success and some had detrimental effects on seedling phenotypes. The results confirmed that the plant core microbiome includes pathogenic taxa and syncom inoculation can manipulate seed and seedling microbiota diversity, making it a promising tool to better understand early stages of plant microbiota assembly. This article was authored by Simonin, Marie, Prevose, Anne, Marie, Coralie, and others.


When you look at a country like Iraq, Syria, when you see what people go through in a country like South Sudan, Central African Republic, or even Gaza, you understand that more than ever the idea that rules should be implemented in war is critical. And this idea is so much challenged right now. And it's critical to understand that 150 years after the Geneva Conventions were created, we need to fight for these ideas. And the we is not just the ICSE, it's all of you, whoever you are, in an armed group, as a president of a country, as a citizen, you need to make sure that rules are respected. The Geneva Conventions need to be alive, they need to be respected everywhere. This is critical. We cannot accept that civilians or health workers are killed, that they consider as, you know, cultural damage in a war. They are not. They are people. They are critical. This is what the Geneva Conventions are about. Think about it. Please, act.


So hi, everyone. My name is Ricco Carbagnani, and today, I will show you how to create an architecture to create a to a platform to create an efective human-robot interaction. I will start saying what our main goals are. What is the main architecture that we use. Then I will say, how do we implement this architecture for the now, the field of application for the robot. alguna. recorded video. Well, audio doesn't work. So it at the end without any audio so forgive me, I will show you all the problems that there are, next steps conclusions and then I'll show you Adam off the robot actually doing something. Only making? What the main goals? Our main goal is to create a robot able toanstulate and interact with human understanding, speech, gestures and facial expressions. The other thing is to build a modular software infrastructure so we have the possibility of of integrating different robotic platforms and integrate new software really easy and also remove all software. So this is really useful, especially for testing purposes and performance assessment. Also so we can test software and change it whenever we want. So how was it achieved? So we use high-level Python libraries for spoken language processing, sentiment analysis and vision, and also we use artificial intelligence applications. So the main architecture actually is pretty simple. Everything is based on Rosindigo. Why Rosindigo? Because this Mac library that we use for finistate machines runs only on Indigo. So we are stuck with that. Also the state machines actually run separate processes, so separate Python scripts, and they communicate with Ros. Now, what are the main benefits of this architecture? So thanks to the state machines that actually run as different programs, we are actually able to use asynchronous computing and also achieve multiprocessing. It's not threading, it's different, they're just different programs using different cores. So since its structure is modular, because that was one of our main goals, then soft code can be added and removed without problems. Also since it's already ROS-based, basically every component that uses ROS in any way can be added without problems. So, now, where do we actually use the now? Mainly in entertainment, education, field robotics, home and companion robotics, hospitality and robot assisted therapy. Now we would have been in the video, but too bad. Now, robotic platforms. The way that we design this architecture, and then it can be virtually be used on every robot, but based, but ROS based or not, because in fact, now we are not actually using ROS to communicate with the robot. We're just using ROS to communicate between the state machines. Also, if you want, you could also use the architecture to work on non-robotic related projects, but it's up to you. Why did we decide to use the now robot? Well, it's actually one of the best commercially available humanoid robots. There are actually not many other options. The other one is the pepper, which is its evolution. Also, since we actually need to interact with humans having humanoid robot makes the, with a humanoid robot, a human can actually connect to the emotional level so the interaction is a lot more effective. Now, how did we actually implement our architecture for the now? So as I already said, we use ROS just for communication between the state machine, and then we use the Noachi library, which is a Python based library to send all of the comments to the robot. Now, we use the PyIML interpreter for natural human robot dialog. So what is IAML? IAML basically is just a programming language that permits to actually create chatbots. This is at the moment kind of the only way to create an effective way for the robot to communicate with a human that actually with answers that kind of are human since you programmed them. Also, we actually heavily use Cosby solution, especially for voice recognition and also language translation, because the local-based one equivalent are not as effective and also the class-based one are a lot faster and a lot more accurate. Also, why language translation? Because basically if we add some kind of automatic translation, then we can just go to the config file and say, okay, I don't want Italian anymore, I don't want Polish and the robot now can speak another language. Also, we use neural networks at the moment we use it for object recognition and face recognition. Now, this is the actual map of the state machines. So basically the main Python script is called humanoid. When you call humanoid, it will launch these five state machines which are sensing, sight, thinking, acting speak and acting move. So now you might ask why is acting speed and active move connected to sensing? That doesn't make any sense. Basically, when the robot talks, if we didn't do anything, it will still continue hearing and go crazy more or less, wouldn't do anything. So we need to make it mute when it talks and when it moves. Now, this is the inside of the site state machine. Basically what happens is there are different states and basically you can just have one state at a time. So you don't have two working at the same time, just one at the same time. The standard one is idle and basically when I talk to the robot it will change to whatever I ask him to. For example, I ask him, oh, who am I? It will go to face recognition. And then when it's done it will go back to idle. Now, this is the actual brain.  I don't know, but for the sake of readability we cut them down. Otherwise you couldn't see anything. It would be just a mess. It's already a mess right now. Imagine with five or six extra machines. You can read anything. And it's pretty self-explanatory. Also we have movement control which is pretty simple, I guess. OK, now one thing. We can provide for a web-based solution rather than, you know, a local one. So basically we can reduce a lot the CPU load. So basically if we use something like a laptop then everything can run faster since it's on the cloud and who cares. Oh yeah, also you actually need a fast internet connection that might be a problem but if you have it then you're screwed basically. And also obviously the tools provided by big companies are usually the best available and the most reliable. OK, so wait, what? For some reason these slides are inverted. They should not be. So OK, what are the main problems with this? So basically Rosindigo is using Python 2 and obviously if you have a piece of code that runs on Python 3 you will have a problem with compatibility. Basically to solve this problem what we do is just, OK, I need to run a Python 3 script. OK, very easy. OK, when you go in this state just execute a Python script that somewhere else and then get the result and make the robot speak or whatever you need it to do. Also right now when we change robotic platform we have a really, really big problem because every time you need to go you need to go to search around on all the code that you wrote and said, OK, so here the robot speaks and use this kind of this kind of command so I need to change it and you need to do this basically on everything that interacts with the robot. Also then our robot for our needs is not really powerful especially in the sensor department for example the microphone that it has is really, really not powerful so basically we actually need to use an external one to make it work and also the actuator's department so basically its limbs are not powerful enough so if you ask him to I don't know pick up something it will fail most of the time also the grape is really basic it's just like the pinch basically it's just like if there were two fingers even there if there are three so what are the future steps that we will do to actually correct every problem with this so as you could understand this is actually a project that we are working on with a company so basically a lot of this stuff unfortunately is closed source so we can just show you how it works but we want to make the skeleton of the main architecture available to everyone I don't know if you want to make it open source so I don't know if you want to start a new robotic project say just download the architecture put in my robot commands in the config file that we will create and basically you can just add code really easily remove it and you could have a robot working pretty easily also obviously so it's not enough for our needs we need to change it to something newer and hopefully more powerful and also adding a config file so as I said when you need to change something on the robot specific task then you need to change all the software so we just create a config file ok here is the webcam here is the activator for the left arm you can do everything there you don't need to modify all of the software also we might add a companion robot to the now so this is just a temporary solution so it will actually house all of the necessary hardware so basically just mainly a desktop computer and also all of the sensors that we replaced the now ones because they just don't work, they're really bad also I know it's kind of early for conclusion but don't worry we also have a lot of more stuff especially with this guy and also we have a video so basically with this robotic platform we have been able to achieve the main goals that we initially achieved which is great, thank God and also we are actually seeing promising results with our now embodiment and also obviously Python is the art of everything and we are exploiting the batteries included philosophy so we try to integrate everything that is already available and now let's enjoy the demo and hopefully the video ok ok, now let's make it start actually ok can you see everything? yes, Rosco started so basically as you can see ok, this is I have little space to actually make you see everything but for every state machine there is like a log on an X term so you can actually see what it's doing also forgive the spelling mistake and here you can actually see what is actually doing on real time the problem is it might crash ok, doesn't also forgive it everything is really slow because this is just not a really powerful computer so speaker is not so important site maybe whatever something missing here it is ok basically you can see what actually it's hearing so one small problem is that the now for the moment can only speak in Italian if you are an English speaker I am really sorry ok, so for example I could say first let me see if it can actually recognize me whatever, it should work but it doesn't so too bad for example I could ask him for I don't know I could ask him something so heard a lot of stuff so as you can see basically it's trying to to understand everything that I said and it's now trying to recognize it I said a lot of stuff and it's taking quite a bit of time actually one of the main problems is the cloud based architecture the speaker cognition is actually based on the cloud so to actually send all of the audio and then to get it analyzed and get it back takes a lot of time especially if you have really big words yeah mi ascolti mi ascolti ti ascolti kome staj kome staj kam nene e tu Or for example I could ask him if he like broccoli ti piaceni broccoli obviously doesn't work any time so we still actually need to to improve everything still a pretty a pretty young project ti piaceni broccoli what? work and times? as you can see Zelo, da se predtahba in in in glasbena, zelo je prišljena, zelo je in glasbena, izgleda je saj sva, a potem je zelo v italii. Zatim, da se prišlo, zelo ne vsezv, zelo je počkaj ustal, Orfan Black je brokulj, kamikaj začneš zelo brokulji, zelo je, da bo začnit, ne bo začnit, nezelo se predtahba. The way it works. So let's try to ask him something else. It's getting too much stuff. The speakers are really not the best, so you can basically hear anything. Kje je George Bush? Doesn't really work as you think. Kje je George Bush? Come on, do it. Oh no, I used this. So that's the problem. This is like a pretty... OK, so as you can see, the performance is really slow because I'm using... It's I3 computer really slow, so it's a laptop. Usually we use a much more powerful desktop, so we have a lot of processes working and it's really, really slow. Also the internet connection, it's not really the best. Also it's hearing everything. This is a pretty sensible microphone. When I talk to the speaker, it can hear everything. He could ask him for some object recognition maybe. It doesn't really work because as you can see, the camera is not really the best. Let's try anyways. Let's try with a water bottle because why not? Come on, stop it. It recognizes everything. Kje kosa? Vimi di piu. Kje kosa? Dipensare. Vaj avanti, cerfi di pensare. As you can see, it's actually working. Siamo ancora su lo stesso argumento. Direi projettore oppure pettorale. AIGIS, AIGIS. OK, as you can see. Se devo scegliere, prendo pettorale. AIGIS, AIGIS. So basically as you can see here, it tries to understand what it is. And also there is like that light, which is really messing with his camera. And obviously object recognition, especially at this level, if you know neural networks, is actually kind of hard. We could try something else if you want, if you ask me, OK, making recognize, I don't know, a glass, a napkin, coin. If you want, you can ask me. Also I could show him, I don't know, dance maybe. Doesn't really ends well usually, but let's try it. OK. Oh, yeah, actually let me do this. Legim in Astoria. OK. Legim in Astoria. Pinocchio. So basically if you give him books, you can read them, that kind of actually it's pretty easy to do. It's nothing super hard, but why is it not working? No, it's heard lucho, not pinocchio. Why? There are a lot of problems, as you can see. Pinocchio, as you can see, it got to the red machine, and it's actually reading the book from a simple txt file. You can stop it by touching Z. Let's try Taichi, why not? Did you understand it? Or not? In this PC? No. In this PC is melting. Taichi, Taichi. Yes. And basically you can make behavior work. Probably it will fall. So don't worry. It's normal. It actually worked. That's impressive. And also I don't know. Dammi la mano. da mi la mano. Basically, if it's understood, in theory, it should follow me. Dam mi la mano. If you just put the arm just for the joystick, and it can follow me around. Yeah, it's slow. As you can see, as I told you, the actuators are not the best. And we kind of need to replace them. I can stop them. Okay, fantastic. Go back to eyeball. Faze, sun rays, it doesn't... Also, I don't know how well it's actually seen me there, so... Probably that gigantic reflector is not the best for its camera. I mean, it's really not that powerful. Come on. Is this the exact face recognized there or something? I think there is... Let me try this. That's right. I recognize you. At least it's the exact face that says, oh, I don't know you.  Obviously, since it's a demo, it must not work. Or... Oh, yeah, actually, it could also... It can actually also read, but... Maybe... This is big enough. I don't know. Can I try with this? Just that. It means they didn't recognize anything. Legi. Oh, it's actually saying it. Legi. Usually it works. You just need to have like really big letters. That's kind of the only problem. Basically, you need to have big letters because the resolution of the camera and also the focal point is kind of crap. So if you have actually a decent camera, you can actually use this with even like really small text. You can just take a book, put it there and say, okay, read, and it will read whatever it's on the page. If I had some kind of big text, but too bad. Also, you could do a lot more stuff at the moment. I could ask for example, domande, domande, ulivo, for example, ulivo, it understood WWW. Ulivo. I don't know. For some reason, at Google, I have no clue of why, but yay. Basically, as you can see, I could ask some really big problems. Think that this is kind of one of the best speech recognition software that you can actually use. Well, now it's not really the best because there is all of the echo from the microphone and it gets really confused pretty easily. So basically now you could ask everything. Microphone, for example, I could ask a microphone and it should ask, tell me what is the mic. It works actually. I don't know. There is more stuff. Stop. Stop, please. Start woofie. It's too intelligent. Oh my god. Stop. Fermo basta. Fermo la sua pazia. They understand what it wants to understand. Fermo basta. Fermo basta. Let's see. Materna. Oh, did it. Oh yeah, now it's idle. Thank god. Fantastic. Yes, it went out. So basically, to just make it go out of a state machine you say basta, fermo, or whatever you want, there is like a config file for that. It goes back to idle. I don't know. We still have some time, so do you want to see it dance a little bit more or not? Okay, so... Ballo sequenza. Ballo sequenza. Okay, this is pretty hard. Yeah, it's... Oh, okay, thank god. Pretty nice you should see the full manager kick in.  And it works as you can see. And there you have it. Good as new. I don't know, should I try to make it recognize something? Depends on what you prefer. We have time for a few questions. If you want demonstrations. I don't know. Tell me what you want to do. If you want, I can do Q&A. If you want, I can continue showing stuff that you can do. Tell me. Maybe one question, somebody? Yes. What? Behind, on the back. Question is, what happens if it falls on the other side? If you want, I can show you. You can do it softly. Any other question or challenges for the robots? Hi. Actually, this camera, I was wondering about, is it really... I mean, the color camera, what we are seeing is the same thing that the robot sees. Why don't you, for instance, use black and white and some sort of infrared light, so you are not dependent on... So the camera, the scene. Is it the same thing that we see right now, or you do some sort of black and white and infrared maybe to be not dependent on the environmental light? And the other challenge or question raise your hands. One question, can you pick up some objects? Can you pick up an object on the ground? Please speak in the microphone. I think it will be easier for everybody. So, as it is right now, it kind of can take objects on the ground. So basically, you can just create an animation for the robot and say, okay, when you are here in this spot, then you can just run the animation to pick up the object. But it's not actually a smart object pickup. It doesn't say, okay, I recognize that is the object and then I will pick it up.   We actually do it, not really that kind. If you want, I can kind of show just there are some things. Oh, yeah, let me try. Prendi. Yes, he does it. Great. Dammi. Dammi. Dammi. It looks hard. Give me back the napkin, but it's not. Oh man, this is so beautiful. Yeah, it's Dammi, no? Yes. Let's try again. Dammi. Why is it not executing it? Well, yeah. You're taking it out of his hands. It's stealing from a robot that you just did. I will. Yeah, basically you can just, it's kind of a cheap way to do it, but it works. Great anyway. Well, thank you. Thank you very much for showing us that robots don't like broccoli. And please give him a warm applause for his work with his robots. Thank you.


Hi, my name is Jeffrey. I'm a product manager at Google working on AMP. And I'm joined by Sandeep Gupta, a product manager on TensorFlow.js at Google. Our keynote is divided into two sections. First, we'll take a look at how the page experience and AMP works together. And then Sandeep will walk us through using TensorFlow.js to harness the power of machine learning to build novel experiences for the web. Speaking of the web, web is the truly open and distributed system we have. There are a few administrative things, such as getting a domain name or hosting that you need to worry about. But otherwise, nobody stands between you and what you want to tell to the world. From the early days of the internet, web developers have used this open nature to build compelling web experiences, sometimes of profit, and sometimes just for fun. As the user expectations have evolved, we as web developers are tasked with a great responsibility to build truly unique web experiences that capture people's imaginations, delight the users, and ultimately persuade them to have an authentic connection with you. This might be in the form of consuming content you produce or buying a service that you're providing. A useful way to think about user experience is using the four pillars of UX. They are loading, which signifies how fast or slow the resources of the page is downloaded and displayed on the user's browser. User annoyance, an important pillar that quantifies some of the web page behavior that gets in the way of user accomplishing a task. Security and privacy, a critical aspect of how safe, secure, and privacy-friendly a web page is. And accessibility. The World Health Organization's disability health fact sheet finds that over a billion people, about 15% of the world's population have some form of disability. We have a huge responsibility as web developers to build web pages that are inclusive to all users. This framework nicely lines up with the Page Experience ranking update that Google is launching mid-June. Page Experience is a set of signals that measure how the users perceive while interacting with the web page that goes beyond just the informational value. The Page Experience signal breaks apart into the four aforementioned pillars as follows. For loading, we have largest contentful paint and first input delay. The largest contentful paint, LCP for short, is a metric that reports the render time of the largest image or text block visible within the viewport, relative to when the page first started loading. First input delay, FID for short, measures the time from when the user first interacts with the page. That is when they click on a link, tap on a button, or use a custom JavaScript power control to the time when the browser is able to process the event handlers in response to that interaction. For the user annoyance pillar, we have cumulative layout shift, CLS for short. CLS measures the sum total of all the individual layout shift scores for every unexpected layout shift that occurs on a page. A layout shift occurs when at any time a visible element changes its position from one rendered frame to the next. No intrusive interstitials is an existing search ranking policy and an associated signal used in search that detects the presence and use of interstitials that are user hostile. Such interstitials are often used to build, to trick users to doing something that they do not want to, preventing them from reading or interacting with the page they landed on from Google search. There are a lot of great uses of interstitials, such as the ones required by the law, GDPR for instance, or an interstitial that provides updated business hours during the coronavirus pandemic. These are not affected by the signal. For security and privacy, we have the HTTPS protocol signal. Users should be able to confidently browse the internet without having to worry about man-in-the-middle attack or improper impersonations. And lastly, for the mobile friendliness, signal covers the accessibility pillar, which measures how effective the pages are on small screens, often used by mobile phones. The first three metrics, largest contentful paint, first input delay and cumulative layout shift, are the Core Web Vitals, a set of metrics that apply to all web pages and should be measured by all site owners and that will be surfaced across all the Google tools. Each of the Core Web Vitals represents a distinct facet of user experience and is measurable in the field and reflects a real-world experience of a critical user-centric outcome. They can be measured using JavaScript using these standard web APIs in your browser and provides an excellent product agnostic path for instrumentation. The Core Web Vitals are not just a set of metrics, but also a robust set of threshold guidance that map to user expectations. The Chrome team has done extensive research and come up with the guidance for what it means to be performing good, poor or somewhere in between. Let's take an example of LCP. LCP is where we measure the loading performance of the most meaningful content of the page. Values less than two and a half seconds means that the page is delivering a good page experience. Anything greater than four seconds means that the page is actually performing very, very poorly. Similarly, for first input delay, 100 milliseconds is a maximum delay users have to encounter during the initial input and its response. Anything greater than 300 milliseconds starts to feel like the page is frozen and leads to a bad user experience. Cumulative layout shift is a unitless metric. Anything less than 0.1, excuse me, is considered to be good or anything greater than 0.25 is great, is considered poor. AMP puts developers on a path to success not just by creating great pages, but also helping you maintain that great page experience over a longer period of time. This means the AMP runtime provides developers with the constraints needed to create great performing pages. And the use of AMP cache further improves this because this allows you to have the pages experienced by your users near instantly. However, like many other frameworks, AMP can't implement all the web development best practices into the runtime. This is why in the run up to the page experience ranking launch, we've been encouraging developers to take a look at your AMP pages to make sure your pages are performing very, very well on your domain. So the first step in starting to implement great page experience for your site starts with the AMP core. Luckily, the AMP core, which the AMP project builds, is comprised of extensions in the runtime, have robust set of constraints implemented to make sure that the AMP is performing very, very well. Over the past several months, we've invested heavily to making sure the core is super duper lightweight and extremely fast. If you're using the AMP framework, you are taking advantage of this automatically. The next building block is how the page is served. Core Web Vitals, as I mentioned, is determined for the real world outcomes. That means when the real users are interacting with the page, that's the signal that is actually powering their page experience signal. In case of AMP, pages could be served either from the publisher's domain or from AMP cache, and depending on how the users encounter the content. Many sites will see a significant portion of the AMP versus actually happening on your own domain. So to get the strongest possible user experience on AMP pages, we encourage you to take a look at how your pages are performing. The best way to get started is to use the AMP optimizer. This is a set of tools that we use on AMP cache, and we believe this provides great optimizations to your AMP pages. You can learn more about AMP optimizers on the link that you see on the screen right now. And that's how AMP project thinks about the page experience. We want to make sure that AMP allows you to create great page experience. At the same time, also continue to improve the page experience without you having to do a lot of work. I'll now turn it over to Sandeep to talk more about TensorFlow.js. Thank you, Jeffrey. My name is Sandeep Gupta, and I'm the product manager for TensorFlow.js at Google. So Jeffrey talked about some key concepts in UX that are driving next generation web experiences. I want to talk about another technology which has the ability to transform web experiences, and that is machine learning. Machine learning touches our lives daily with applications across many fields, such as healthcare, education, energy, transportation, sustainability, and accessibility. Due to publicly available large data sets, more powerful computing at our fingertips and research in new methods, we are seeing improvements in all kinds of products and services powered by machine learning. And you see some examples of that here on this slide, such as being used to power platforms for education in classrooms, helping detect diseases or predict natural disasters, and giving people new ways of communicating and interacting. And we're beginning to see how machine learning can help improve web experiences as well, whether it is to interactively experience products, such as this virtual makeup try-on web app by L'Oreal, or accessibility tools, such as the one shown here in the middle, where a person is playing a keyboard by moving their head, or apps that can detect body pose and use that in many useful ways. Machine learning has a lot of users for the web. Let's take a look at some specific examples of how web developers have been using machine learning. InSpace is a virtual learning and collaboration platform, and they use real-time toxicity filters in their web conferencing app. So when a user types something bad, it's flagged before it's even sent to the server for processing. It alerts the user that they may want to reconsider what they're about to send, creating a more pleasant and a safer conversational experience on the platform. Another example is from Include Health, which is a musculoskeletal care delivery tech company. They're using body pose estimation models to deliver guided physiotherapy at scale. So many people unable to leave their homes or travel these days, this technology allows for a remote diagnosis and treatment from the comfort of their own home, using just a web browser and a standard webcam that almost everyone has easy access to. This is an example for creative design. You can bring a character to life by using pre-made machine learning models that can estimate body pose and facial gestures. This pose estimator tool was created by the partner innovation team at Google, and it allows you to draw any SVG character you like and then use your body to control it in real-time, giving animators a motion capture solution to drive 2D character animation that anyone can use with just a webcam. And yes, all of this is running entirely in a web browser. Here's a core enterprise use case. So page load time is a very important factor for user experience on a website. And studies have shown that it can dramatically impact page view, time spent on the site and even conversions and clicks on an e-commerce site. Machine learning can help predict user's navigation patterns on your site, and by selectively pre-fetching the page assets can significantly improve page load time. Here you can see that the site on the right with predictive pre-fetching loads twice as fast as the unoptimized one on the left. So hopefully I've convinced you that machine learning can be a very powerful and useful tool for you. So now you may be wondering, is it easy to use? Do I first need to learn Python? Well, the answer is no, or I wouldn't be here speaking at the OpenJS world. So TensorFlow.js is a library for machine learning and JavaScript that can run in the browser or on the client side or on servers with node.js. It provides an easy JavaScript API through which you can use machine learning with just a few lines of code. By running in the browser, it enables low latency execution, lower server costs for your applications, and by keeping user data on the client side, it enables privacy-sensitive applications. Lastly, it is GPU accelerated, so you get great performance. With TensorFlow.js, you can write your application code once and use it anywhere. And running ML in the web has some unique advantages over running it in native apps. For example, there is zero install needed. You can reach an audience of billions of users instantly simply by sharing a URL with no complex environment setup for your users. Since JavaScript is a very versatile language that can run on a wide variety of platforms, there's a large list of environments where TensorFlow.js can be used. So you can use it client side in all the popular web browsers, as we mentioned, or on server side via node.js, taking advantage of the huge NPM package ecosystem. You can run it natively on mobile platforms via React Native, Angular, or PWAs, and even on IoT devices, such as Raspberry Pi via node. With TensorFlow.js, you can run existing pre-trained machine learning models. You can customize models for your use case by retraining them on your own data or write your own models completely from scratch, just like you may already be doing in Python, but now in JavaScript. We have released many pre-made models which are ready to use out of the box with an easy high-level API. These models range across many categories, such as vision, body, text, and sound for you to use. You can check them out on TensorFlow.org slash JS slash models, and you see them, and you can check out the demos and documentation, and we are constantly adding to this collection. So now let's take a quick look at what this looks like in practice. We will look at a text example built with our question and answer model. So let's say you want to build an app to help find the answer to a question in any piece of text that you present to it. Here we have a Chrome extension that does this on the text on any web page. Just type your question and the model provides an answer and scrolls to the part of the page that most likely answers the question. Examples like this are now possible with the new BERT-based question and answer model in TensorFlow.js. And to do this, you just need a few lines of code. So let's take a look. First, we import the TensorFlow.js library and the pre-mode model Q&A model that we want to use. These are conveniently loaded from our hosted scripts so you don't have to install anything. Next, we can define the text we wish to search. This could be some text on a website. Here we are just using a simple string. We then also define the question the user wants to ask. And this could come from some input query in a real app. Now we load the question and answer model itself with Q&A.load method. As this may take a few seconds to load, it's performed as an asynchronous operation. So we are using the then keyword here to wait for it for it to be ready. Once the model is available, our function will be called with the loaded model passed in. Finally, we can call our model.findAnswers method. We pass to this function the question we want to answer along with our search text. And when this completes, it will return an answers object from which we can get the most likely answer from the given passage of text. In this example, the model predicted cats as the answer to our question, which we proposed, which is correct given the text we had to search. And that's all there is to it, pretty simple. So give it a try. The exact same workflow works for using any of our pre-trained models. Here are some of our most popular models. Cocoa SSD is an object detection model which can identify 90 different object classes and show you the bounding boxes in the images where the detected objects are. MediaPipe face mesh is a high resolution face tracking model which can recognize 468 key points on the face. It's only three megabytes in size and it provides real-time face analysis for detecting facial gestures, lips, and eye movements. This has many applications in retail, entertainment, and accessibility. For human pose estimation, we have a powerful and easy to use pose detection API which supports three different models for a variety of performance and accuracy needs. We recently added two powerful new models in this API, MoveNet, which is an ultra-fast and accurate model that tracks 17 key points optimized for diverse poses and fast actions. And the second MediaPipe's blaze pose, which gives 33 key points and this extra granularity may allow better tracking for certain applications. So you can do a lot with our pre-trained models out of the box that I showed so far, but sometimes you need to train a custom model for your use case. Google Cloud's AutoML service lets you train powerful custom models on large amounts of data in the cloud, no machine learning expertise necessary. AutoML takes care of creating the best model for your training data and shows you how your model is performing on various evaluation criteria. You can also choose whether you want a model with higher accuracy or faster prediction or a trade-off between the two. And once your model has trained, you can either create a cloud endpoint or you can conveniently export the model in TensorFlow.js format and then deploy it in your web application. Another way to train a model easily in the browser interactively is by using Teachable Machine. This lets you train some common ML models directly in the browser and then exporting the trained model for your use. I highly recommend playing around with this tool. So this was a quick introduction on how you can use machine learning to give your web app superpowers. For more resources and getting started material, visit our website shown here and join the community discussion group on discuss.tensorflow.org. We are curious to see what you built. Check out other users creations and share your own work on hashtag made with TFJS on Twitter. Thank you so much for listening and enjoy the rest of OpenJS world.