Hey, everybody, what's up? It's Rob Dodson. Welcome back to Polycast. So today on the show, we're going to be wrapping up our tour of Polymer and Firebase by taking this project that we've been working on and integrating it with the Polymer build library so that we can build it for production. So follow me over here to my laptop. And we're picking up where we left off last time. So we've got this project that we created. We can go ahead and maybe start up our server so we can just preview this thing. OK, so we can go to our terminal here. We can run Firebase serve. I'm making this a bit so you can see it. Firebase serve. That'll start up a local server just so we can see what we're working on. OK, so this is where we left off last time. We had this kind of to-do list thing that we were building. It kind of looks like a mess right now because we added messaging. We added a bunch of other weird stuff to it. So the design is totally wacky, but whatever. It has the features that we want for our application. And today, we're just going to cover how we bundle all the code together so we can launch it up onto Firebase Hosting. In my project, the first thing I'm going to do is I'm going to create a Polymer and JSON file. Polymer.json. And inside of this file, we're going to tell Polymer's build library about the entry points to our application, about our application shell, and about any other extra dependencies that we want to package up for our build. So I've gone ahead and actually just saved this puppy out. And I'll bump this up a little bit so you can see what we've got here. So we've got this entry point, which is our public index HTML file that just tells Polymer build, hey, this is where we're going to start our app. And then we've got our shell, which if you recall is our MyApp element. So everything that we've been working on so far has been kind of contained within MyApp. Now if you were using something like the purple pattern and you had additional fragments, you could add those to this Polymer JSON as well. So those would be views that you were like lazy loading over time. In our case, inside of MyApp, we've actually just got all of our views just displaying right here. We've got my log in, my notes, my subscription, et cetera. So we don't have any fragments or anything like that. We're just going to tell it about MyApp, and it'll bundle that up. And the last field that we have here is our extra dependencies. So these are things which we're not referencing as HTML imports. So Polymer itself is not referencing these things. But these are files that we do know that we need to include in our build. We want to make sure they get copied over. So this is things like our manifest JSON, the Firebase Messaging Service Worker, which we talked about in our last episode, and our Web Components Lite polyfills. So we're going to tell it to copy these over as well. OK, so that will instruct to the Polymer Build Library what to copy. The next thing we want to do is create a package JSON file and actually install Polymer Build itself. So make a package.json. And we'll just pop some stuff in here. So I just called it MyProject for now. You can name it after whatever you want. Give it whatever version you want. We're going to include a little npm build step here. So I'm just going to include a build step that's going to run gulp for us. And we'll get to that in just a bit. The main thing we care about, though, are these dev dependencies down here at the bottom. So a few items that we're going to pull in. We're going to pull in this library called Dell, which will allow us to sort of clean up our build directory every time we're about to do a new build. We'll pull in gulp. And notice that we're using version 4 here. And then these last two, Merge Stream and Plylog. These are just for debugging Polymer Build as you're using it. Or Merge Stream is for merging your streams, Plylogs for debugging. And Polymer Build, we're going to pull in the latest alpha version, which is 0.6.0 alpha. Probably by the time you're watching this episode, 0.6 will be out already. But I wanted to go ahead and use it now because it's a little bit easier to use some previous versions of Polymer Build. So we've got that ready. I'm going to run npm install. And I've actually already got all these things installed in my node module, so we don't have to wait for it to download or anything. The next thing we want to do is create a gulp file. So make gulpfile.js. And what I'm going to be doing here is I'm going to be using gulp and the Polymer Build library directly instead of using Polymer CLI itself. Now, if we used Polymer CLI on this project and we just ran Polymer Build, it actually would do a pretty good job of building everything. But it's going to create some additional things, like a service worker. And it's going to put it in a weird place because right now, Polymer Build's CLI service worker is not programmable for the output directory. That's something that they're going to probably change in a future version of the CLI. So we don't have these weird vestigial files laying around and stuff like that. We're just going to use Polymer Build directly. We're going to tell it exactly what we want it to build. So we're going to do that inside of this gulp file. And I've actually gone ahead and pre-written this gulp file because there's a fair bit of code in here. We can walk through these piece by piece, just so you understand what is going on here. At the top of our gulp file, we're just pulling in a few items that we'll need for doing work like setting up our paths, pulling gulp, merging our streams, deleting files, and things like that. This is all typical boilerplate. What gets interesting is when we get down here to Polymer JSON. So we're actually going to require Polymer JSON. We're going to pull in our Polymer JSON file so that we have it in gulp. We're going to pull in the Polymer Build library, and we're going to pass it our Polymer JSON file when we instantiate it. So that way it has some configuration. And then we're going to tell it that we want our build directory where it outputs everything to be build slash bundled. So this is going to take everything that's in our public directory right now and put it in build slash bundled slash public. Then I have to find this little helper here, because the gulp file that we're going to create is going to be maybe a little atypical from most gulp files that you've seen. It's going to be basically one build function that's going to have a promise chain. And there's going to be a few places where we're going to wait on some streams from Polymer Build. So we've got this wait for function right here. All this does is it returns a promise, and it listens for a stream's end or error event, and either resolves or rejects the promise. This is going to be very helpful for our build function. And then inside of the build function itself, we're going to kick off Polymer Build. Now, the Polymer Build library, if you recall from our episode on writing your own generator using the Polymer Build library and the custom generator init function, Polymer Build library takes all of your HTML imports. It looks at the CSS in them. It looks at the JS in them. It looks at the HTML, and it extracts each of those into its own file stream. And you can run transformations on the stream. So if you want to do CSS minification, you want to do JavaScript like uglifying or something like that, you can after it has split apart those streams. And then once you've done those transforms, it rejoins everything for you. So what we're going to do is we're going to start off by calling this del function. We're going to delete the build directory. If there's already something inside of it, we'll just clean it out so we can start fresh. And then we're just going to work through a little promise chain. So we're going to create a source stream using the Polymer Build library. What it's going to do is it's going to look at our entry point for application. It's going to look at that MyAppHTML file. And it's going to figure out all the HTML imports that it references. And it'll treat all of those as sources. So we're going to create a stream for that. We're going to tell it to split the HTML. That's how we create the CSS, the JS, and the HTML streams. And here is where you could run any sort of transforms you wanted to. So for instance, if you were using something like uglify, you could then pipe one of those streams to uglify. You would use the Gulpith library to check the file extension. So if it's a JavaScript file, pipe it to uglify. If it's a CSS file, pipe it to CSS slam. HTML, pipes to HTML minifier, et cetera. So I'm going to comment these out for now because I don't have these libraries installed just yet. But I'm going to leave these here. And when I put the code up on GitHub, these will be here for you. So if you want to add your own Gulp transforms, you can. So that'll be all of our source files. And then we've also got a separate step where we work through all of our dependencies. So this is stuff that's living in Bower components or whatever. If you want to uglify those or CSS slam those, you can as well. So that's going to create two streams, sources, and dependencies. And then we're just going to merge the two streams together. We'll have a little console log here as soon as we start getting some data out of the stream that just tells the user, hey, we're analyzing these streams. If we want to do bundling, we can pass it to our Polymer project bundler. So we're going to pipe all those streams into Polymer project bundler. That's what's going to go through and basically run vulcanize on your behalf and start to put everything into the MyApp file. You could do additional things here, too. Like if you wanted to add prefetch links and stuff like that, you could explore doing that with your own transforms. And lastly, we're just going to take everything and we're going to pipe it to the build directory, which if you remember is build slash bundled. We set that up at the top of the file. And then we're just going to use this wait for helper function to wait on the build stream. So after it's done processing all of our files, this promise will resolve and it can move down to the very last step in the promise chain. We're just going to console log out to the user, hey, everything is done. We'll resolve that one last promise. And then that's it for our gulp test. That's our entire build process for our Polymer site. The last thing to do is just to set this as our default build step for gulp. And we should be off to the races. So let's open our sidebar here and let's give it a shot. So open our terminal up and we will run. I'll make this a little bit bigger. So we're going to run gulp or because I added in my package JSON file, I added a build script. We could also run npm run build. Either one of those should work. So we run that. And we should start seeing those console logs coming out. So it's deleting the build directory. It's analyzing our build dependencies. Then it tells us that the build is complete. If we go back to our text editor and we look at our build directory, you can see now we've got inside of build bundled. We've got all of our project files. It copied over things like our Firebase messaging service worker because we added that to extra dependencies. And if you look at myapp.html, you can see that it sort of munged all the code into this file. So it re-envolcanized for us, which is great. The last thing we can do is we can preview this using the Firebase server. The one change we need to make there is we'll go to Firebase JSON. For hosting, we'll tell it that we want to look at build slash bundled slash public. Save that. Go over to our terminal. So we'll run Firebase serve. And it'll tell us that it's serving build bundled public. Click on localhost 5000. And here you go. Here is our bundled project. And we can verify that it's the bundled version by opening our network panel, clearing everything out, hard refreshing the page. And you'll see that instead of requesting tons and tons of HTML imports for all of our dependencies, we only really see a couple HTML imports. I think we actually only see myapp.html. I'm going to look at the source for that and verify that it is the vulcanized version. So at that point, the only thing you have left to do is to run Firebase deploy.  So that about covers it for today. If you have any questions on using Polymer and Firebase, as always, you can leave them for me down in the comments or hit me up on a social network of your choosing. As always, thank you so much for watching. And I'll see you next time. Hey, folks. What's up? It is Rob. Thank you so much for watching this episode. If you'd like to see some more, we got some over there in the playlist. And as always, you can click the little Subscribe button to keep getting content every time we push it up to the channel. Again, thank you so much for watching.


Hello, thank you for joining me. I'm in the beautiful Sussex town of Lewis today and we've come here to have a look at the ruins of Lewis Priory. Now this is Trinity Church It's not actually part of the priory, but the reason I'm starting the video here Is because there is a small section of the priory on this side of the railway. I must say this side of the railway You'll see what I mean. Most of it is the other side of the railway because the railway has literally dissected the priory. But we start here This fragmentary wall is just here. Around here is the gatehouse.  So there we are. That is the remains of the Cluniac Priory. You can just see where the arch would have been. It's sort of been filled in a bit. And then that's possibly built out of some of the buildings. What we'll do, we'll quickly go round behind it. As if we were going into the priory. Now I've got to turn around, head back to the railway station to cross the road. Now this was the first Cluniac Priory in Great Britain. And it was founded by a monk from Cluna. It's not quite Cluniac, but it's coming on screen now. A town in France. That's where the monks came from. They founded the first Cluniac Priory in Britain. You can't really see much of the back of the gatehouse. But we would have effectively... This would have probably been a building. It had probably gone through there. And these buildings here is priory terrace. So imagine heading that way into the priory. Now like I said, the railway dissects the priory, which we'll get on to later. So the first Cluniac Priory in Great Britain. It was once a very, very large church. Supposedly larger Chichester Cathedral. Let me just let you have a look at the Trinity Church in Southover. Unfortunately that wasn't open. Otherwise it would have probably gone and had a look. Lovely whole mockery there. Now it's interesting this path takes us round here to a bit more graveyard. But I don't think we can actually get anywhere beyond here. It just sort of goes into other houses. So that's the back of Priory Terrace. See the South Downs over there? The rest of the ruins are over there. That line of trees would mark where the railway is. So what I'm going to do, I'm going to go back to the station and we'll find our way into the priory to see the rest of the ruins. So although this isn't planned to be a train video, that is quite obviously the railway station. You can see a class 313. That's some of the oldest EMUs in Great Britain. I'm not going to have a ride on one there later because the actual reason I've come down to Lewis today is to see some class 50s which are on a rail talk. I've got just over an hour so I thought I'd have a look at the priories while I wait. If you want to see class 50s, have a look at the link on screen now. That video will be completely separate to this. Just point up over there is Lewis Castle. Again a video for another day. I'm just making a quick stop in Lewis. So that's the railway station. You can also see a fielding platform again. We'll have to do a video on the cuckoo line which ran north from here. Or was it the cuckoo line? No, it wasn't. It was the line that went up towards Uckfield. The cuckoo line was the line from Eridge down to Polgate. I'm just across the road and when I've got the cuckoo line I'm going to go down there. That'll take us into the priory park and that's where we should be able to go and explore the ruins of the priory. Now this is one of those videos where I have not seen the priory before. So I am going to literally do this video as I explore it. But I've had a little look, read up about it. You can see there is some more ruins than what we've already found. I haven't even been on the railway that dissected it. I came on the line down from London to Lewis. I've been to Eastbourne before, been through to Hastings. I've not ever done the line from Lewis across the Brighton. And I'm not going to do it today either. It's a video for another day. So we come into the priory park. It should open out somewhere. The question is where? There should be a priory. It must be around here. I think we can go up that mound. I might leave that to the end of the video. That's the bowling green through there. So I think I need to go this way around. But yeah, we should be able to go up that mound and have a look at the views. So let's have a look. Oh yeah, it now says I can see a sign from the same priory park. So I can just see some fragments.  You can probably see the south-downs all around us. So it was the priory of some pancreas. Of course, I feel like it's the pancreas think of the railway station, which was the terminus of the middle of Mainline. Now it's more known where it still is in terms of the middle of Mainline, but it's also more known for being the Eurostar terminal. But before all of that, some pancreas, obviously a saint. And it was his priory here in Lewis. If you walk along here, it's a very sunny day today. Nice spring, sunny day. As I came on the train, everyone was heading for Brighton. So it's quite refreshing to... I'm quite glad I'm not going to Brighton, because I think it's very busy there. So when we get round this corner, we have to have a proper look around the ruins. And it's just round here. These flowers look nice growing on the walls. Whether these are little shops and kiosks, coffee kiosks, they'll be all closed at the moment. I'm not sure.  See some of the ruins. So let's go in and see what we find. So as we walk in here now, here are the ruins of Lewis Priory. Now there's a picture here. It shows us what used to look like. So, you can quite obviously see. It's a fairly large building. I think we probably started up over there. So the railway must cut through here somehow. I know it cuts right through the church. Well, we're going to have a look around, see what we find. I've always enjoyed looking at ruined abyss. So I can just see the church. That's where we started. So I'm going to, I'll finish off at the church because I think the church is one of the most impressive features. But on this occasion I'm going to start of it because there's not a lot of it left. There's this little chapel here. So the bulk of the church would have been just over there. So let's have a look inside here. So I'm not sure I'm actually in the main church. I think the main church would have been sort of there. And I'm in one of what I suppose would be the South Isle possibly. But you can see a lot of it sadly has disappeared. I mean, it would have already been a ruin when the railway came. So you can't entirely, you can't really blame the railway for making the priori a ruin. You see how it marks out on the ground where the priori would have been, where the walls would have been. Come to here. So here, that piece of grass there is the site of the infirmary. And there's a picture here showing you what it looked like. I'm going to literally walk through a wall here. Some in the infirmary now. And then more interesting surviving fragments of the ruin down there. So let's go and have a look around there. There's these little interpretation boards so they can tell us as we go. Ro and I must be just up there. So what we'll do before we do this bulk of ruins, we'll head over to there. We'll get a view. I'm not going to hang around waiting to see a train. But we'll at least get a view of a train. I mean, I'll get the idea where the rail is. There's a little model of it here. I think the building we were actually, I think looking at it, the building in front of us there, I think it's possibly this chapel here. So I think most of the church has disappeared as to where the rail is. Have a look over here. There you get some nice views out of the ruins. So I'm quite excited about exploring that. It's sort of a great church. So yeah, we're effectively standing in this court. It's a really great church. I think we'll just walk through the light side. We'll go through the steps. Train now. I didn't buy someone a different train. There is one. Pass three on three. What's rather ironic, one of the oldest EMUs in Britain is making its way through a much, much older prairie. You see the monks in on the toilet. Now there's a drain. Down there. I remember when I was a child in Tintin Abbey, I used to like playing in the drains.  So yeah, so the toilets would have been down there. You can just see, I think the remains of a drain. Let's go down here, though, into the actual bulk of the ruin. Which way should we go? I could go down there. I'm going to go through here. See what this part's like. And then we'll make our way round back to the other side. So we get to here. What's in here? You always get these fascinating little rooms in prairies where you're never quite sure. Yeah, okay. So it gets to nothing. So people leave rubbish in. That is the only problem. When you get a prairie or an abbey or any kind of ruin, it's freely open. Sometimes the wrong sort of people come and they leave rubbish. We're going to get onto that tower there. That's effectively a folly. So we're still talking about it. So this would have also been part of the monks toilet. So I think actually that drain may have been along there because if you have a look, it's what the building looked like. And then of course there's been an upstairs above us. If we go outside again, I'll get the elevation of it all. Have a look over here. You can see the side wall. See the buttresses sticking out. The wall is leaning this way a bit. And I'm sure it's not going to fall over. It's probably been out that long time. Do you see the top of Lewis Castle over there? Oh yeah. You know what's about the toilet. You can just see the top arch where the drain would have been. So the drain would have run along here and probably out into the river towards the sea. So this is usually the most complete part of the prairie. So just walk through here. And then what we'll do we'll go up towards where the railway is. We'll have a look at some other sections of the prairie. So the railway is behind that fence over there. The last time we went to a prairie we didn't have a railway running through it. We went to Crocs and Abbey and Staffordshire. That had a road running through it. And I drove my car right through the middle of the prairie. If you want to see that, do have a look on the screen now. Coming into another little room here. I'm just going to sort of explore each room. See what there is. See obviously the prairie is free. So if you ever do want to come visit you can just pretty much come anytime. Oh that's some impressive things there. And what's good here. That's where we came down the steps. So we came to this section of the prairie. It feels like it's quite low. Ever this has been underground again.  I'm not entirely sure what I'm in now because there's nothing to tell me. But obviously when it looks so, that big arch has been propped up at some point. So now we'll go back up here and have a look around the next section. There's a board here. This will tell us what we're looking at. Oh so it's the dormitory. So yeah look, we were down there above the monks of the slept upstairs. So I've been in the monks dormitory a few years ago. A friend of mine advised me to stay at a monastery in southern France. Now I'm not so religious but I find it quite fascinating. The way monasteries work, so I stayed on there. I didn't stay in a bed like that. I stayed in the guest house. But I saw the monks' rooms and they actually are like that. Like beds of straw and everything. There's a spiral staircase here. Now I don't agree with climbing on walls but when there's actually steps to go up them, I'll go up them. So let's have a look. See what this takes us. I think it's going to go far but we'll go up those steps and have a see what's at the top. This always makes a visit to a room a bit more valuable. It ends here.  But yeah, this is the size we've got. See the railway tracks over there. So I'll let you just hold the camera over there so you can see it further out into the ruins. I think there's not a lot more to see. We kind of see most of it but there's a little bit we can see. So back down here. I wonder where this goes. There's another little room through here. Look at this one. Oh, so this is the refectory. This is where the monks were eating. And again, the railway cuts right through it. So if you look, that is the building. Looking that way. The railway effectively cuts right across that corner. So yeah, it's a bit shame the way the trees are. It'd be really nice to get like a picture of a train and the ruins. But I think that's going to be a bit of a challenge. Let's just have a look at the railway. There we go. There's the railway. Again, I'm not going to wait to see a train. So it's quite a fascinating ruin. It's not a huge ruin like some of the other bigger priors you get. I want to go over and finish by that little tower. That's a folly. I believe the gentleman who acquired the priory, he then built that little folly in the grounds. I think his name was John Blaker. It was a name a bit like Baker, but not Baker. It's on screen now. And he built this folly. I also understand he built a tunnel under the railway. But I don't suppose we'll be very surprised if we get into that. So that's the bulk of the priory behind us. And this will be a good moment for trains to come along. But I'd be surprised if it does. I don't know if it's going to be this little folly. Because I do like follies. And it's quite king of building this one. I don't suppose we can go inside. I don't wonder if there's like a staircase inside. We can get up onto the roof, possibly. But I'm not sure. So I think that will pretty much conclude the video. It looks like looking at the door. It's possibly made from fragments from the actual monastery itself. So here we go. And then that way it's looking back across the ruins of the priory. So I've come back to the other side of the priory. You can see where we first entered the priory. Just over there. I thought we'd come back here and that might be windy. I really want to go up this mound. Because I like finishing videos in a high place. So let's just go up there and see what we can see. And to be completely honest with you, I'm not sure the connection with this in the monastery. But I do know there is one. If you know, do comment and tell me. Let's just ascend to the top. Providing quite a nice view of the priory over there. There's the bowling ring in the foreground. You can see the south-downs behind. No one seems to be coming up here. I did wonder if it was closed. But there's nothing to say it is. So we'll climb it. So that's sort of looking across. Towards Eastbourne Hastings. And well, right away down there is Seaford. Where I'm going to go to this afternoon. Because I've not been on that bit of track. And I'm going to see these class 50s instead of that all the way. That must be the football club over there. We might get quite a nice view over the town. Like I said, oh yeah, there's the castle. I'd love to explore the town and the castle. I'm going to make a bath video for another day. We should see if we can see the station. The railway is directly down there. See what that roundabout is? That's the railway. And then you can just see the railway below us. It's already coming up here. It gives us a very good observation. It might be windy. Apologies if you can't hear me. It is up in the shower. The railway line is stretching off towards Bryson. Looking that way towards Hastings. The river in Lewis, you know, there's a lot more to see than just the priory. But if it's a quick change, it's got to be an hour long like I have. You can go and see the priory. You don't have to pay. So from the mound overlooking the priory. Thank you very much for watching. Please do feel free to like, subscribe and comment. Goodbye.


This paper proposes a novel approach to improving the performance of instant segmentation and high-resolution remote sensing, HRS, images. Specifically, it modifies the structure of the one-stage fast detection network to better adapt to the task of ship target segmentation while also improving the efficiency of the network. Additionally, two feature optimization modules are added to the backbone network to further improve its feature learning capabilities. Furthermore, the network's feature fusion structure is modified to increase the prediction ability of multi-scale targets while reducing the amount of model calculation. Lastly, extensive validation experiments were conducted on the HRSID and SSDD datasets, demonstrating improved instance mask accuracy and enhanced segmentation efficiency. As such, this proposed model is a more accurate and efficient solution for instance segmentation in HRS imagery. This article was authored by Muhammad Yasser, Lili Jean, Shan Wei Lu, and others. We are article.tv, links in the description below.


Making your decision as to where to go to college is a daunting task for many high schoolers and for many student-athletes across the country, that journey has reached an end. As today was National Signing Day and Penn was put to paper across the area as local high schoolers announced their choices. Bemidji's Yvette Morgan will continue her academic and athletic career by playing softball at Chipola College down in Florida. Chipola is a prestigious division one junior college and Morgan says that once she arrived she knew that was there she wanted to play. I really wanted to find a school that fit for me. I visited a few schools before and right when I went to Chipola it felt like the right school for me. The Indians have one of the most successful softball programs in the country. They've won two national championships in the past 11 years, most recently in 2015. And Morgan says that the strong history initially drew her to the school. It's a really good coaching staff and stuff that make you feel like you're part of the family. They're a very accomplished program and I think I'll learn a lot. If you enjoyed this segment of Lakeland News, please consider making a tax-deductible contribution to Lakeland PBS.


So, I watched for about another minute and there's an argument going on. I'm doing research and I basically was like, wow, this is the first thing I see after 20 years not paying tribes.  Multiple people in this discord DM me and we're like, yo, don't judge our community by him. We all hate him and I was like, but he's in your community and you haven't banned him yet. He's there.  By the way, I'm giving a talk about your game at PAX in like two weeks. So good luck.


Hello. Good morning to you all. I'm just great. Sunday morning, which is mostly very early for most of you. I'm going to introduce Steph Rogers, and he's going to be talking about training machines to be continuous. So Steph. Good morning, everyone. Thanks for coming out so early on a Sunday morning. Good. God, what made you do this? Is that on? Good morning. I am not a cyborg, but I am part of a cyborg team. And I'll talk to you about this today. Why would anyone do such a thing? What does it mean, and how do you do it on your team? I'm going to show you proof that this exists. All the proof that causes me to believe that this exists.  So let's talk about why. Why would we want a cyborg team? This is really cool, but it's a short book. It's worth reading. It's called The Great Stagnation by man called Tyler Cohen. And it talks about all of the progress, the advances that we've made over the last 250 or 300 years, and where they've come from. And much of the research and data points to the fact that they come from various low-hanging fruit in the world. But things that changed, such as children that died during childbirth, people that are in real health care, or having them in water and in houses, everyone getting education, new lands to expand into, like the Americas, and then things like electricity being implemented and declared across society. These are things that then we saw by consuming these low-hanging fruit, by having these low-hanging fruit in place. We saw more advances. We saw productivity increases. We saw inventions and innovations just streaming out. But the book says, and the theory goes, that we're starting to see the end of that, that low-hanging fruit has been used up. We eat all the low-hanging fruit of modern history. And you can see this a bit if you look around. I have to tell my son, Kai, that the fastest civilian aircraft, the Conqueror passenger aircraft, no longer flies. And it stopped before he was born. I have to tell him that humanity went to the moon when it was in the days of his grandfather. In various cases where we see stagnation come along. And the theory goes that we need to find new low-hanging fruit in order to push ourselves past that and see the next wave of innovations, the next wave of productivity increases, the next set of awesome advances. So we don't say we have that. Let's see what we have here. We could buy the next set of innovations, the next set of advances. The problem is, when you actually look on this case, Robert Stolo, he's one of the, I guess, fathers of modern economics. And he said that you can see the computer age everywhere but in the productivity statistics. We see machines all over the place. There are all sorts of things. But they have yet to have a big effect on the amount of innovation, the amount of progress we see in society. Let's look, and I believe that that's because we use machines as tools, but we don't use them as team members. Let's look at another analog for this. Back in the industrial revolution, we'd like to pull the industrial revolution into everything, but at least what we have to do right here, there is a power source of a steam engine. In a factory like this, I sit on the top, and all the machines were connected to those line shafts. A massive steam engine drove those line shafts and everything revolved around this power source. The steam engine had to keep going all the time as long as any one machine in the factory was needed for some past. People's lives revolved around this. They had to come in not because the product was ready for them to work on or whatever they're building, but because the power was there, and they had to do it. Things that needed more power were closer to the steam engine, things that needed less were further away. So the first time electric motors came out, they were figured, we can make this better. We'll have some significant productivity increases here. Some advances, we'll replace that steam engine with an electric motor, and everything will be better. Well, it turns out that wasn't the case. Less people died of black line and so on, and there was some moderate boosts to what they were doing, but until you actually thought differently, took that electric motor, put it in all the different machines, ran electricity throughout the factory, released everyone's schedules and work around the fact that they no longer were tied to the power source. You could move different pieces, different parts of the production through a different factory and so on. That took like 20 or 30 years until that change really had a massive productivity increase in advance in society. And that's what we're seeing with machines now. So that's why I'm talking to you about this today. A cyber team is a team that is part human and part machine. It's a team that doesn't use machines as tools, but uses them as team members. Why is this a lonely fruit? Why is it easy for us to do this? And that's because we as engineers, as developers, we can speak the language of the machines. In our projects and in our companies and in our open source communities, we can pull this off simply. We can talk to the machines and tell them what to do. We can interact with them as team members. Imagine that you were doing something completely different, that you were mining or something, something fundamentally different. It would be very hard for you to instruct the machines what to do. You couldn't teach them as team members. You didn't have that skill. This is a lonely fruit for us and we must take advantage of it. So, enough waiting with hands. I want to show you how this is actually working. This is the proof. The proof that convinced me that this was the case. The proof that is a Linux session in a web browser and later, well, not much later, but 1130 in the config management dev room, we're gonna be diving into this and talking about what it does. But in case you haven't seen it, this is what it looks like. You log into a Linux system, need to hurry any more, log into a Linux system and you can check out what's going on, see the states, the load, there's so many different things you can do here. I can talk for hours about this. You can see containers, a cluster of containers and inspect what's going on in each one and how this thing is put together. You can discover and learn about the system. Talk to you about how this works because this is part of the proof. This is JavaScript. This is the Chrome JavaScript console. And in the console, we're typing a Linux command here and we're actually seeing results from it. Cockpit interacts directly with the system from JavaScript. So here we're responding to A, a process and here we're accessing a DBS API. The interesting thing about this is that the pieces of cockpit outside of the JavaScript don't know anything about these APIs. So this is the hostname API. That's part of system D. It has various properties here that you can see. You have information. We can call different methods on that API. Cockpit itself doesn't know about these APIs. It just relays them to the system. And there's a component that translates these into actual Linux calls. We can access Unix sockets. We can monitor files. We can call REST APIs on the system. You name it. We can do it. These guys here put cockpit together. Actually, some of these folks have moved on to other things. And you can see that guy in the middle. He's very incognito. He didn't want to show his picture. As you can see, you can understand. Show his picture at Boston is a big no-no. But fold cockpit together made it do so many different, access so many different parts of the system. You can see here that there's eight folks working on this, or we have worked on this, but they managed to talk to this many APIs on the system. This is a quick list that I built of the different things that we talked to on a Linux system. There's things, there's obviously big APIs, like SystemD, Docker, Kubernetes APIs. There's tools like Q and U. There's processes like SSH or SSH add. There's files in here too, like the password file. There's, yeah, you name it. We're interacting with all sorts of different parts of the system. Take that. Take those 90 different APIs, and cockpit works on 15 different Linuxes or different products. Linuxes like Vintu, Debbie, and Fedora, Atalic, Bell, and then, of course, other things like Kubernetes, OpenShift, or Rev, is places where cockpit interacts with a different version of those APIs. Each of those 90 APIs revs at a different pace on different products and different operating systems. There's different branches with different functionality as well. There's different browsers that then we have to interact with. These are the ones we test with. Of course, people use more than that. And then, on top of all that, the product does weekly releases. Actually, now it's bi-weekly, but for the longest time we've done weekly releases, getting that functionality to work stably every single release. So this leads to a real combinatorial explosion. If you did a math there, that number adds up to over a million. And you can fudge this in different ways and throw different factors in there, but really, this is insane for a team like you saw earlier to pull off on its own. It just doesn't work. The effort of a solely human team does not scale past a certain complexity point. You will reach this barrier where you can't take your product any further. Some companies, the Googles, the Facebooks among us can push this point out a little further. They can add, throw more people at the problem and try to get something more complicated, something more integrated, something more advanced working. But you will reach a barrier where you just can't throw enough people at the problem to make it work. To copy this one of those cases, it would be impossible for a team to pull off that massive combinatorial complexity in an effective way. So that's why cyber teams, machines as team members, is look at the proof of various things that we can see this in action. And I encourage you to not look at so many of the details, nitty gritty. When you look closely at a fish, when you take it apart, look inside, you'll see goop, you'll see little bones and stuff. And, but when you take a step back and watch it swim, you see that it's a life form, you watch what it does, how it acts. So I wanna show you how these things act in a project like cockpit, how machines act as team members. Here's the various ways that they do so. Bots owning mundane work, pair programming with bots, humans training the bots, bots learning from the humans, bots shipping cockpit, bots as committers and the team's staffs without bots. We'll look at each of these. So owning mundane work, there's many examples of this. So much of the work that the bots do is trivial stuff that people are kind of scoff at. They're like, oh, of course I can do that. Here's an example. A bot came and opened a issue on GitHub on the cockpit project and said, you need to pull the translations from Zanata. Zanata is a place where translators congregate and translate the project. Now a human can go and do this task, but it just so happens that a bot decided, I know how to do that and started working on it. You can see that the tagline on the top changed to work in progress and the bot starts actually performing this task. The task itself is rather trivial. It's a bunch of commands that you or I could run. We pull down the translations. We build it, we upload the new translation template and so on and we make a commit out of it. The bot turns us into a pull request and a whole little army of bots come to test this thing on all sorts of different operating systems with all sorts of different browsers you can see there. And containers and so on. Make sure that this works. Actually in this particular case, that very pull request had a bug that broke some of this stuff and it wasn't merged. But typically a simple pull request like this, after testing, after all the bots verified in thousands of different ways actually, it gets merged. A human says, okay, this looks good and merges it. So this is the kind of mundane tasks that you might see the bots doing. There's things like pulling security updates from NPM automatically, noticing that a certain NPM dependency has a update and trying it out. Bring them in to cockpit opening a pull request, making sure that it works, then of course all these tests run. Bring in various operating systems so that we can test them here with new dependencies, new combinations of those 90 different APIs. There's work that to do with, I guess, what was the other one? There's all of these trivial tasks. And these trivial tasks that there's things that take up your time every single day. Some people spend 10 or 20% of your time on this. Some people spend 80 or even 100% of your time on mundane repetitive work that you shouldn't have to do. And of course you use machines to do them but you should be able to hand it off to machines. The biggest one is of course testing. Testing that combinatorial explosion. This is in September of last year, the amount of tests and the amount of operating systems that were booted in VMs in order to test a project like cockpit. You can see there's half a million VMs run in the month and this is that repetitiveness. If you go and repetitiveness of mundane work, imagine you tried to hire enough testers to actually do this, it would be crazy. So the, I guess this is really the core of where cockpit gets that advantage from using machines is repetitively testing all those various millions of combinations over time to make sure that the project is still functioning. So, yeah, we can see those numbers again here. This is a more recent thing. One of the interesting things about this is data, I'll show you how we use it later, but it's data about the tests that were run and in the first line, we can see that we've pulled this data and we're counting the number of lines inside of it. We see that there's more than three quarters of a million different tests run. When we look for the failures, failures, and that points to a very strange situation where most of the work that these bots are doing is basically useless. You throw it away. It's like, nope, that worked, nope, that worked. It's green, it's repetitively doing the same thing over and over and over again in a very frustrating way, so in some ways we're abusing these machines, just making them do repetitive work over and over again to derive that small amount of value to find the places where things do fail. These are things that we never do to humans.  So another way that we see bots interacting on the team is pair programming with them. This happened during the run up to a rel release. Now you may know about rel, Red Hat's Enterprise Linux. We support it for 10 years. That means stuff that goes into rel has to be relatively stable. The people who put it there, our team, if we put cockpit there, we need to know that we're not going to be embarrassed about it in six months, much less 10 years, that we're going to be able to continue to support and update that. So typically when you do that in a team, you have this long stabilization period that comes up to it, or a freeze period, where you say, well, during this period of time we won't make any big changes. We'll make tiny little fixes and changes, and we hope we stabilize the thing and don't break it. That was not what happened with us. As we were running up to our first release in rel, we had big changes landing in the week beforehand, thousands of lines of code changing, and I was asking some of the folks on the team, how in the world is this okay? How can we do this? And their answer was, well, we're actually pair programming with the bots. Marius told me this, he's taking code and pushing it into the testing so often with different combinations, sometimes breaking things on purpose so that they'd find problems running through 50,000 or 100,000 different iterations of checking whether a race is there or a bug is there and finding all the possible problems so that you can deliver this. So another way that we see this in action, humans training the bots. This is actually really critical. When you have a team, the team needs to interact well with each other, and the humans need to be able to teach the machines, teach them constantly about what they need to do, whether it's useless tasks, whether it's telling them to go and do this differently today, and just today, and tomorrow I want you to do it, I want teaching them about process that the team doesn't want to have to do. This is usually things that are broken, things that are awkward, build systems that the team hates and where they have to push their software anyway, making sure that the bots can do the work that the humans don't want to have to do. You can see, if you look at the Git log, these are the, everyone on the team is contributing to teaching the bots how to work. And this is constantly going on. In fact, if you estimate this, it takes about 25 to 30% of the time of the team to go and teach the bots about the stuff that they will then repetitively do, that they will take care of. This is normal though. In a given team, spending that much time talking with each other and bringing on new folks, mentoring them, discussing what you want to do next, that's a decent amount of time. We're just taking that and applying it to the machines to make sure that they pull off what the team stands for, that they're working in the same direction. So bots learning from humans, we see this too. This is a very recent thing, something that I've been working on and interested in, it has to do with machine learning. Bots can actually watch what, what the members of the team are doing and try and replicate or try and derive some information from that. We do this with test flakes. I'll talk about this more later, but at the bottom of every test, some tests will fail just for no reason whatsoever. It just seems random. These pop up in any testing system that you've seen and people typically refer to them as flakes. After this happens, a human will go and say, well, that's not related to my pull request, that's not related to my change, this is insane and either retrigger the test or figure out a way to merge it anyway. The bots will watch that behavior and learn from it and start to apply, we tried two different techniques, a neural network, but also unsupervised clustering to figure so that they can then mirror what the humans did. The humans treated a failure like this in this way before, they merged it anyway. So maybe the bots are now learning from that and you can see they're starting to put these numbers at the bottom to indicate that. Hopefully soon we can actually take action based on it. Bots ship cockpit. This is a bunch of routine work that so many of us are involved in in various distros. Every distribution does this completely differently. It is very awkward. In the cockpit team, a human signs a tag and get with some release notes or some, I guess, release lines in the git tag and then bots come along and they do all sorts of stuff like making the tar balls and patches for uploading to different distros, updating the spec files or the Debian control files. They release preview builds like copers or PPAs. They'll push into the Fedora packages and start the whole process of getting something released into Fedora. They'll upload packages into Ubuntu, into Debian. They'll upload tar balls. They'll do container builds on Docker Hub. They'll upload new documentation and so on. There's all these various tasks that they'll go and do. If you take all of these together, if everything goes well, it might take a person about a day and a half to do this stuff and if you do this every week, that's a phenomenal amount of time that you spend here. If things go bump and break or you have to repeat something and you have to wait on some step, it can take even longer than that and it's just a massive waste of time, of people's time. So that for having the bots own this part of the work allows the humans to actually do some interesting work, create some stuff and not have to sit there and repetitively do this like as if they were sitting in a factory or a sweatshop. Bots are committers on the project. If you look, we have, we've called our bots this, we've given them the name Cockpituous. You can see that the fourth highest committer to Cockpit of all time is Cockpituous. It's the bots. Now, if you look at the actual poll request, no, sorry, the actual commits here, you'll see that they're not that interesting. They're the mundane commits of the project, really the lowest tier of the uncreative commits but the reality is those would all have to be done by humans. They would all have to be, humans would have to spend their time on this. So by freeing up the humans to do some interesting work, suddenly we're able to make dramatic progress in the project. And lastly, the team stops without bots. We saw this a while ago where we had a, we didn't have a distributed architecture for these bots and one of the machines that they were running on went down a key machine and we basically stopped. When you lose a significant portion of the team or your community or your contributors, it is a dramatic blow to the project and this is exactly what happens when the bots go away. The process, the work on the team stops and we in this case had to actually spend two weeks figuring out how to, how to make this not happen again, how to implement it so that we don't have, it's distributed enough that we don't have a failure like that ever again. So over the, those were the proof, the way that we've seen this in action. I don't think this is the only way you'll see this happen. What was cool is that we started off in the project not knowing about all these things and having a master plan of where we wanted to get to, but by implementing some basic rules, some basic fundamentals, it grew, it was sustainable. It grew a life of its own and became something that completely defined how we worked as a team and changed, made the project possible. And so I wanted to share those rules that we discovered with you today. And these are the key to making it happen in your project. And hopefully you'll be able to do way cooler stuff. The results will be way cooler than what we have and that's what's exciting to me. But let's look at the principles behind these rules. Let's say we want to give in behavior, a behavior of having a cyborg team, having the humans not do the mundane work and the bots doing the mundane work and the humans teaching the bots and making sure they're a real coherent team. At the beginning of the last century, there was a social psychologist, one of the early social psychologists and he proposed a theory that behavior is driven by two forces. There's the driving forces that push you in a specific direction, that make you want to go there, make you want to have a certain behavior and there's the restraining forces that prevent you from behaving in a certain way. These two reach a sort of equilibrium. If you think about, let's say, driving your car, getting to a destination, you have the driving forces, I want to get there faster, let's go faster, right?  And at some point you make that decision, okay, this is the right balance between the two. So the way to change behavior, it turns out paradoxically, is you first diminish the restraining forces and then you increase the driving forces. Kurt Lehmann was the one who proposed this and it has played out, it has become one of the foundations of behavioral psychology. So another way to look at this is, why don't we have cyborg teams in a given project today? What is preventing that from happening by itself? There are certain restraining forces that are preventing that from happening and we need to reduce those in order to make it happen. Afterwards, of course, we can encourage it, we can inspire people to do this, but first we need to remove the obstacles. So here's what we came up with as the rules that reduce those obstacles, that make this possible and make it sustainable. Number one, teaching a machine must be as easy as teaching a human. If you get an intern or a new guy on your team, sometimes you make them do the mundane menial tasks, go and get the coffee, and you should be able to teach them, you should be able to say, well, no, don't get it from here, go get it from that other building because they have way better coffee, right? And you expect that to happen from that point on. You expect to be able to teach and change and instruct people. It's not a super easy process, but it's something that we're comfortable with and we know how to do it. We have to make it as easy to teach a machine as teaching a human. In our case, we put the code for the bots in the same Git repository as our source code. All the team members know how to contribute to the source code for cockpit, so now they know how to contribute to the bots. That's not the only way to do this, but that's really important, to make sure that you don't set up an artificial barrier there. The second is machines must produce feedback into the team's workflow. Even if that workflow is shit, it doesn't matter. The bots, like you saw before, when they're working on an issue, they put a work in progress tag at the top, just like a human would, that's how the team works. Try to remove the differences between how a bot works and a human works on the team. Use the same things. Over time, you can change your workflow to be better for both the bots and the humans, for both machines and the humans, but make sure you don't set up this ivory tower, the special place where the bots work and the humans are in a different place working. That does not contribute to building an actual cyborg team.  Given the right credentials, this is really important. If there's a bot doing a task on your project, if it's doing CI, if it's doing releases, I should be able to take that bot, run it on my machine. Of course, I need the credentials to make sure it works. And I should be able to hack on it, change it, and perhaps throw it away and build a better one. This is a very open source principle. Imagine if you had to call Linus every time you wanted to run a build on the kernel. And someone else did the builds. You didn't get to reproduce it on your machine. This is the same way if there's a bot doing some action and you're not able to do that same thing yourself, that's a big problem. And vice versa, if there's tasks in your workflow that only a human should be able to do, make sure that you don't set up a technical barrier to prevent the bots from doing it. It may be your policy. Like on our project, we have policy that clicking the merge button or merging an actual pull request is something that a human does, but there's no technical reason why a bot couldn't do that in the future if we needed to change that or remove that repetitive part of our lives. So, and back to the core of all of this. If the bots, to communicate with the bots and tell them what your team stands for, what is good and bad, what is right and wrong, where you want the project to go and where you don't, you have to write tests. The tests are the way of basically preventing a bot from just doing random actions but working in a given direction. There's a good example of this. There was this cool project done with a Pac-Man video game and machine learning and a camera watching the screen, a microphone listening to the sound, output of the machine and a little controller for the joystick. And without teaching the machine to play Pac-Man but just telling it the test of what is a good noise and what is a bad noise coming from the machine. The machine is able then, of course, with machine learning techniques to play Pac-Man, learn how to play Pac-Man. But without that test, you can see it just doesn't even start. Without the tests that tell the machine when they did something good and when they did something bad, all of this falls on its face.  Those trivial pull requests we saw earlier, let's say the one where that I was talking about about bringing in a new Node.js dependency. If the bots were randomly just updating Node.js dependencies, it would be a credible amount of work for a human to then go and say, well, does this work? Does it not? Is this valid or not? But by having all those tests in place, we're actually able to have the machine do most of the work and the human should say, approve it or reject it. So we don't have much time and we have so many things we could dive into. These are all sorts of things that we did or didn't do in the project. There was a principle about don't assume bots can't. This goes to one of the basic machine learning theories that things that we think are easy in our heads to do as a human are actually really hard for a machine to do, such as walking downstairs. The things that we think are hard, such as playing chess are actually plausibly easy for a machine to do effectively. One of the things we did is don't rework process. Just because you have bots doesn't mean you have to change everything.  Containerization is a really awesome way to make bots work well. The bots we have are distributed, they scale out, they kind of work, nothing orchestrates them. They go and look for tasks to do and see what's available. They'll try to do it and then they use collision avoidance and detection to figure out if someone else is already doing this task, just like two humans would. The bots are self-validating, so when someone changes them, you can have the bot check itself and do that task to see if it works. And, oh well, we talked about avoidance and detection for collisions and then machine learning. I had this highlighted actually from the last time I gave this talk because people were interested. But I'm happy to go into any of these topics and talk about them or we could have questions. We could dive into things that were unclear that I may have skipped over or jumped over. Is there anything here that, yeah, let's do questions and if a certain topic comes up, we can then quickly go into it and I can show some slides about it. There's a question here. Yeah, like, okay, we can do that. So, well actually, here's something that I'd like to show first is there's of course the question, sorry. So, imagine you wanted to start on this and I'm gonna look at the core of that question. It's like, where do you start? And of course, one of the places that we say we start is with the tests. If you don't have the tests, it just doesn't work. One way is to add tests to a Fedora or Debian package to start to define the behavior of that package, not just the content. That's a good place to start. Another place to start is of course with Travis CI that many of you, most of you, I hope, are already using unit tests in some way, validating this and Travis CI is a good way to start there. We have an example here of running a full user land for integration tests. You know, Travis uses Ubuntu. You may want to use something else and here you can use containers to do this. You can use VMs to do this. There's an example directory inside of this project, github.com slash cockpit project slash cockpitchewis and how to use a full user land to do your tests. There's, you can run your testing and your CI and your bots in Kubernetes and containers. CentOS CI offers this to open source projects and you don't have to be in, you don't have to, you know, be in CentOS or buy into that to be part of this. This is just something they want to do. This is one thing they want to contribute to open source and I applaud them for it. They'll give you a Kubernetes account to run bots and testing and CI. Lars here did a really good example of how to run VMs inside of containers if you need to do that for your bots for testing. We do that a lot and there's a, this is actually a clickable link. I'll upload the slides and you can see how clean that, it's a good place to start for a clean implementation. And then there's this container, cockpit release container for the bots for delivery, for updating RPM, for updating a control file and you know, doing builds and all of that stuff. So let's look at that one actually and I'll show you the code for it. Then you can, that's an interesting place to start. Oh, that's a good point. I'll show it in the terminal. So I have this, I have this code checked out. It's under the cockpit project in a repo called Carpichus. And in this project here, you can see these various directories. There's a release one. The release one contains all of the tools and scripts and services that the bot for releasing cockpit into all of those different venues. Here we can see, we can see that when we look really closely, these are all really shell scripts. They're pieces that the bot uses to do its job. They're parts of the various tasks of the, sorry, various tasks that it does throughout that process. Let's look at where this is defined in the project though. Here is the cockpit repo and you can see there's a bots directory and inside of here is lots of bots. The top level of a bot will always be the team member and then it uses tools that hopefully and other projects do use these release tools. The top level of the bot, the thing that actually is the actor that goes and does stuff is specific to the cockpit project and so it's in the cockpit repo, not somewhere else. I've forgotten the exact VIM incantation to increase the font size. But if you look at this file, you'll see all the various jobs and these are the various scripts that this bot does. And so it'll go and perform these tasks. You can see there's awkward things in here like, oh, I need a Kerberos key tab or sorry, ticket to upload to Fedora and you'll see that the top level, this is not very pretty. All the various pieces that it uses though are reusable. So this is one way that it mirrors the actual, well, this is one way that it interacts with the project and mirrors the fact that it's a team member. But I'll show you another. So let me look at this here. Oh, I'll look in the collision avoidance and detection. Sorry, here we are. So inside of the top of the project and the top of the cockpit project, you'll see this file called .tasks. This has all the various things that most of the bots, beside the release bots, they go and look at the kind of things that should be done. And when you run this file, it will run all these various scripts and produce output. The output is just lines of different shell commands that are ready to be run. You can see here that many of these are tests run on specific commits. There's the bot called tests invoke. But there's also other bots here called image prune or various other tasks or refreshing of images, pulling all those 90 dependencies together and making a new thing that we can verify that cockpit runs against. So we print out all of these and then the bots will choose one from the list. They'll randomly choose one and run it. Now if two bots choose the same thing, they'll figure out that they're both touching the same GitHub issue or that they're both updating the same task and one of them, the one that didn't win will back down. But this is a good example of, I think, showing where the bots operate, how they work. I hope that answers the question. All right, another question, yep. Okay, that's really simple. I hope that's not disappointing. Self-validating and self-aware bots. One of the things that they do is one of the first things that a bot does when it's started is it will check out, if it's operating on a certain pull request or issue, it will check out its version in that issue. It will check out the code for that bot as relevant to the task that it's doing. So let's say you run a bot on a certain issue or pull request, you tell it, I want you to operate on this. It will check out the code for itself from that one. Then it will run itself with a slightly different behavior than everything else. But more than that, a bot will actually run things like this where it checks, in this case, first one is checking, it's running in a container, but do I have access to DevKVM? Can I start VMs? Can I perform tasks that need to run VMs? Am I running in such an environment? The second one is that, am I in the Red Hat network? Can I access the next version of Red Hat Enterprise Linux because that's not out in public yet? And so if not, then I'll skip those kinds of tasks. I won't do those tasks. They're not valid for me. Another instance of this bot running somewhere else will perform those tasks. And this points back to how the bots are set up. They're organic and distributed. There's nothing orchestrating them. Multiple instances of these things are run all over the place, in many cases in Kubernetes, which you can easily scale up the number of containers that you run in a pod. Oh, but I run them on my laptop from time to time, machine running under my desk, all over the place, these bots can run, as long as they have the credentials to do the tasks that they need. And of course, check that they do. They check, am I on this network? Do I have these credentials? Can I perform this task? Then they'll go, they'll ask, in our project we use GitHub, so they'll ask GitHub. They'll query the API, they'll look for outstanding issues, things that are not done yet, things to test, things that are issues that they know how to solve, like we saw that updating of translations earlier. They'll all do this independently and go and look for this stuff. They'll post their results publicly. That's really important. Otherwise, in an open source world, you can't have them hiding what they're doing. Even if they're running behind a firewall or under someone's desk on a machine that's not accessible from the internet, they'll post their results publicly. And we then update GitHub for whatever changes are necessary, for example, the fact that a task was completed or pull request is updated or created. And of course, lastly, they'll try and find each other to share state. Sometimes one of them will do a machine learning task to actually train itself from some data. And once it does that, we don't want them all to have to spend time on this. The boss will exchange data or exchange the images that were built for the various operating systems that they're testing on and so on. We have a mechanism for doing that, which is very simple. It's HTTP based and uses hashes to represent state and stuff like that. It's, I guess that's one of the messages here. This is all very accessible to you in an open source project. By making these rules by following those basic rules of enabling that dialogue between the machines and the humans in a productive way. So they act as team members.  So any other questions? Yeah? So the question is, as far as training the machines, how much is it humans training the machines and how much is it machine learning techniques? What is the balance between the two? And honestly, most of it is that low hanging fruit. The low hanging fruit of make it really trivial for the humans to talk to the machines and remove the barriers there. Most of it is just humans telling the machines, I don't want to have to ever do this again. So I'm gonna take what's in my head, how I know how to do this task and code it so I don't have to ever do it again and let that run. That is the low hanging fruit. We also do, we started very recently doing machine learning with techniques like this. There's a theory I'd like to, I want to prove and I'm in the process of doing it and has to do with test flakes. Has to do with the fact that I feel that test flakes, failures in the test, random failures are really like fuzzing. They're mutations that we should be taking advantage of. They're the machine's finding flaws for us and then we use these kinds of, in our case I've experimented with these kinds of ML techniques to do that, to actually take advantage of these flaws. Time is up and I'm sorry about that. I love talking with you about this and learning from you about this as well. So please let me know and there's another, I mean please, I'd love to discuss this more. There's another talk about cockpit 1130 in the config management room. So I do have to run there but if you have anything to contribute I really want to discuss this further with you guys. All right.


Welcome everybody. I know this is going to be a lively discussion. I think for those who have joined us for the last two days, the summit has certainly posed some really big challenges and opportunities ahead of us. And today is all about delivering some concrete answers. And I think the issue that we've got to talk about today, valuing online learning, is really at the heart of how we can make some real and impactful difference. So we'll be talking a lot about how to value online learning and to make that shift towards skills-based talent strategies. And we have a great panel today who's actively doing this and will be sharing their wisdom and ideas to put some tangible ideas into practice. Before we get going, I just wanted to share a little bit of context to where our appetites before we move into the Q&A. You know, what we've been hearing from the last two days and what we've read in the future of job reports is that we're facing double disruption on the one hand brought about by the impact of COVID-19 on industries and economies. And on the other hand, many organisations are wanting to speed up their automation plans. Together, this is already changing the size, shape and skill set that we need within our organisations to fuel growth. And as we look forward to the challenges of job displacement and job creation, it forces us to think about how are we going to reskill and how are we going to reskill at the right speed and pace. During the pandemic, we all saw a huge surge of interest in online learning, both from individuals and also from institutions. I think, you know, whether it was platforms like Coursera or Udacity, they saw exponential growth in the amount of users engaging on that platform. And this is wonderful news, wonderful news for a number of reasons. On the one hand, it really democratised learning, given free access for many people. It also supported something we've been talking about for the last few years around supporting lifelong learning. And it also demonstrated how we can have greater flexibility in how we deliver learning to those that are hungry and eager for it. It's no surprise now that 42% of companies expect to use online learning to deliver not only these benefits, but also to reduce the costs of their learning spend. Powerful drivers in the current climate. But here's the rub. Many of our current hiring and promotion practices are grounded in the traditional world. Many of our practices look towards traditional qualification to see whether people are suitable for a job or suitable for a promotion. And if we keep looking at traditional qualifications as proxies for skills, we sell ourselves short on the full power that online learning can be delivering for our people. Because the challenges without online learning really being seen as an acceptable pathway to employment is going to be tough to ensure that those on that platform keep engaging in lifelong learning because they see the benefits. And it's going to be impossible to get the level of work and ability needed to fuel our economies and close that skills gap we've been talking about. Interestingly, one of the things that struck me in the future job report was people also were not moving into traditional skills adjacencies or jobs that are similar to what they've done before. Again, I think questioning the current paradigm about how we evaluate people's suitability for a role. Online learning platforms and new approaches to interviewing hiring are definitely the way forward. They're unlocking new employment opportunities for those that maybe are non-traditional learners. They're promoting, as we mentioned, that culture of lifelong learning. And they're helping to expand the potential of talent, not just for our own organization, but for those in the broader economy. But we need to couple these new innovations with a new way of looking at talent and talent management. And with most organizations saying that they want to redeploy at least half of their talent into the roles of the future, this becomes a huge imperative. We've got phenomenal opportunity in front of us. Let's start hearing from our panelists about how best to unlock it. Today, I'm joined by four companies who are doing just that at the current time. And as we start to do our introductions, I'd love to hear from you your quick response to this one question that we've heard throughout all the sessions today. What do you see as the biggest barrier to delivering the benefits of online learning that we've all been looking for? Ravi Kumar, president, emphasis. Can I start with you? I'd love to hear your response. Thank you, Kate. Thank you for that question. I'm going to make this really quick and elaborate later. I would say three things. First, we suffer from something called degree inflation. What I mean by that is we are going back to what you said. We depend on pedigree and degrees as a proxy to skills. We need to switch to skills. Second, online training and online learning by itself is not enough. You need to apply a value chain starting from assessing pathways, assessing capabilities, online training and then apprenticeship. Third, I would say adaptive collisions between online platforms and employers and those adaptive platforms, those adaptive collisions will drive how well we can absorb them into our jobs. These are the three important things which are constraining online training, online learning to be mainstream. Well, Ravi, I look forward to unpacking some of that later. I know under your skill and restart, you've done a lot of those adaptive coalitions. So I would love to hear some of your learnings from that. Gabrielle Duport, CEO from Audacity. I'd love to hear your response to that question. What do you see as the biggest barrier? Great. Thank you for the question. I think the biggest barrier is an outcomes based standard in certification. And the real question is how do we know that when someone graduates from an online program, they actually have the skills? And how do you know that one machine learning certification is equivalent to another? So really, what is that standard we measure ourselves by so employers can have confidence in what they're getting? That comment around skills qualification and how we can have confidence is going to be really important. So thank you for those opening remarks. Gabrielle, looking forward to coming back to you. Amy Regan Moorhouse, Senior Vice President at Trailhead, would love to hear what you think is the biggest barrier. Yeah, I mean, I think we've talked about it this week. It is very clear that COVID-19 has fundamentally changed our job market with hundreds of millions out of work. And those that do still have jobs, they see them change rapidly overnight. So this makes ongoing online learning even more important. But 70% of people we just did a study are saying that their jobs have changed and 60% of those are saying, I need my skills to be up to date. They're not up to par right now. So that's sort of a foundational challenge. But I think for employers, businesses are struggling and they're really not investing in their employees as much as they could be right now. And that coupled with the fact that they may not have systems in place to really generate engaging online learning, that'll be a barrier to adoption. Thank you for those opening words, Amy. I think that combination of concern around job security and also your own marketability means that we're tackling this topic in a particular climate that we need to acknowledge. Thank you. Theresa Carson, Vice President of Amazon Web Services, would love to hear from you on your insights here. Hi, Kate. I think the answer, one of the answers, what everybody else said was great, but access. I mean, the biggest barrier is access for all, making sure that we have broadband access, especially if you think about the underrepresented, for sure we don't have that, two other kind of areas we see. It's not the biggest barrier probably, but just the right skills being taught properly. And working back from what the employer actually needs for that individual student or employer to do, making sure that we're teaching those fundamental skills that allows them to get that job and create economic development. So that for me, that's kind of the things I'm thinking about, but access for sure is what we see a lot of lack of. And Theresa, you put it so simply. It's very clear the things we need to tackle. Actually, putting that in practice is always a bit harder. So I love your comments there about access, but also identifying the right skills. There was a poll a couple of hours ago, and I think one in four people said identifying the right skills was the biggest barrier. So I think you're with the audience on that one there. Absolutely. Lastly, let me come to Daphne Kiss, CEO of WorldQuant University, would love to bring in a different perspective here on what you see as the barriers. Daphne, you're on mute at the moment. Sorry about that. Thank you for having me. I've been part of the Skills Consortium and I have to say it addresses a lot some of what's been raised here, the idea of having shared standards of what employers are looking for and what are the skills of the future. I'll let go of course what everyone else has said. I mean, our program, we have 6000 students on our platform at any given time in the data sciences, and we are trying, there's an extraordinary amount of talent on the platform. And I think one of the great challenges, as others have said, is the ability to compare these new delivery models to what we're traditionally accustomed to while providing the academic and training rigor and curriculum that's expected by the workplace. And so when you ask what the biggest challenge is, again, I think it's employers acknowledging that these skills are there, and that they're willing to work with us as a pipeline partner. Because again, education is great, training is great. But at the end of the day, what we're all echoing is that we're looking for employment opportunities. And so for an organization like ours, really being that pipeline partner to employment opportunities is, I would say, the greatest challenge. The end game there is about employability. And I think we have to keep that in our sights. But I also like the language you used around the amount of talent we have on the platform, not the amount of learning. Because I think that's an upend, a whole thinking about how you go about hiring and recognizing skills in a new climate. So great opening comments, I think, to warm us up for today. Actually, Gabrielle, actually, Daphne, why don't we stay with you? I think one of the topics that has got a lot of airtime over the last few years has been this one around universal taxonomies. And I'd be curious to hear from you as to why you think having a common skills language is so important. What are the benefits for both learners and workers? Yes, so I would say that it's certainly a challenge. For one, we're talking about a moving target. As we all know, the skills of today are not necessarily the skills of tomorrow, 90% of jobs that exist today will not exist tomorrow. So I'm a big fan of both the quantitative, qualitative skills that are necessary across these various academic and career proficiencies. And I think that that's something that we need to pay considerably more attention to as we look to the future opportunities. So we use the term, of course, often lifelong learners. And that is a big issue. How do you turn people into lifelong learners? How do you make that exciting? And in a way, it's not so different than the kind of culture you try to create in a corporate environment where people, if there's an environment of trust and an environment of collaboration, where rather than focusing on skills gaps, we focus on skills strengths and support peer to peer learning. I think this is something that all online learning environments need to really focus on and do a considerably better job than we have done. I think it's part of, of course, what's contributed to all those falloff rates that we all talk about in the history of all of this. And I think we're sort of on the 2.0 or perhaps 3.0 version of this, but that we need to start thinking about the environments that we're creating for that learning. Yeah, that sounds great. While I have you, we have got some questions that's coming in from the audience around digital badges and credentials and qualifications for that. Of course, a universal language gives us a starting infrastructure. I wonder if you want to say a few words on the connection there between the two. Sure. So one of our programs, very specifically, is a data science 16 week module that really focuses on Python and data analytics and visualization, et cetera. And people, how do you measure that? How to the point that was made, you know, certainly at Udacity and coming standards that we all agree on. So one of the things we've done is partner with Credly, which is one of the recognized credentialing outlets and probably the most widely recognized. And I think there it's, it's, it's really almost again on trust, which is to say that we can verify that what we say we are teaching, we are teaching. How that it's been up to you to compare the quality, you know, the level of rigor, whatever the product that's expected, but to have just as a starting point, the idea that we can in fact validate that the skillset we are promising you after two units, you know, 16 week program is what you expect. And in some way, you know, we know that's what traditional education, right? That's what accreditation has been as a property for delivering what you promise you're delivering. So some of it's unique to being online and some of it maps well to some of the traditions, educational traditions we all come from. Yeah, some of those mental models are there, but I think new players and new acceptance is what is a bit of the barrier today. Gabrielle, on that point, I would love to come across to you next if you don't mind. You know, Gabrielle, the shift towards skills based talent systems, whether it's, you know, hiring, facilitating talent moves, even pay for skills, again, is something that's trending a lot at the moment. How can elevating online learning as an acceptable pathway to employment really help to drive that agenda? Gabrielle, if you wanted to say a few words on that. Yeah, absolutely. And look, the major challenge to elevating online learning to accept the pathway is really proving that online training leads to hard, measurable skills and actual business outcomes. How do you prove to an employer that a program manager can become proficient in data analytics or field engineer with obsolete skills can start coding in Python or Kotlin? And the truth is the early wave of online education had really uneven results. Engagement rates were very low and graduation rates were even lower. That has changed and has changed very materially. And I think a case study is helpful to see this. We work with Shell, which operates in 70 countries worldwide. And they had a challenge, which is they had 160 central data scientists that were not capable of serving 70 countries. And they had conversely thousands of geological, civil, mechanical, electrical engineers globally that really didn't know anything about data science. And so we created a citizen data science mandatory program with them to re-skill their field engineers worldwide on foundational machine learning skills. And you talk about what are the outcomes. Our graduates then went on and pulled a machine learning model that took streaming field data from every rig across the globe and were able to predict rig failures two weeks in advance and each avoided rig failure save them $2 million and immediately save them over $100 million. And the key learning here is that businesses have a huge untapped asset, which is their employees. They're talented, they're capable, they're excited. Online learning is the catalyst. And enterprises have an opportunity to unleash their employees. And it is a, yes. No, I was just going to say, I mean, that being able to actually quantify and give those, those figures, I think, does make it a lot more impactful. And I'm also keen to hear from some of the other panelists as we go through, we'll certainly come back to you on other ways in which we can deliver that. And Amy, I'd love to chat to you about online learning and hiring systems. I think a lot of people have talked about the end game here is getting that skills based labor market. How can you see that working in practice? Any insights you can give us? Like, I think it's interesting is that, you know, we talk about what, what are businesses going to do? But really, businesses need to partner with governments and NGOs on this, because no group can do it alone. And I think, you know, both Daphne and Gabriel are really talking about how do we work together on number one, a system of credentialing of certification for online learning that says, this person can do this job, they have this skill. And whether that's validation between from a certification or a microcredential, a badge, or on the job, hands-on skills validation, I think that's going to be really important well beyond the traditional degree that we look at most employers look at today. I think, again, that common skills taxonomy, so we can look at what are the skills that we need? What are the skills that we're going to need in the future? And how do we know that these skills, you know, are leading to these jobs and build the online learning pathway to provide people the opportunity to learn those skills? I do think that employers need to kind of lift their heads up from what they're working on right now, and focus on the future. What are the jobs that they're going to need in the future? What are the skills they're going to need in the future? Who has those skills? And how do we provide online training to offer opportunities to fill those gaps? I think from an industry perspective, you know, I think we've discussed here at the forum, a billion jobs, which is like a third of all global jobs, are going to be disrupted by technology in the next 10 years. So that just think about it, that's a billion people, we need to reskill to get ready for these jobs. So it is absolutely necessary that we do this work now. From an employer impact, you know, we're all in this all digital work from anywhere, my house here, your house is there, world. And that means that we need to have online learning that is flexible and available to everyone within an organization. So the impact there is certainly massive, but it also gives a unique opportunity that maybe we haven't had before, which is truly to hire for potential. If you have a great online learning initiative within your organization, you can hire for potential, and then help skill up the employees as needed. And that really opens new pathways to everyone on Team Trailhead. It's always been our mission to empower anyone to learn the tech skills, regardless of their background. We actually can do that now. So I think that's really important. I think also that we need to be focused on the systems that we are deploying for online learning to make sure they're easily digestible, you know, micro learning, you know, snackable bite sized chunks that I can do in between my sometimes distracted world. Mobile is is a huge component of learning these days. We actually saw in COVID on Trailhead Live, our our video streaming, we saw an increase of 400% because we need to offer the types of modalities that make sense for our learners now. So I think that's hugely important. Yeah, and a huge difference from pre COVID to now. You know, I think the apathy we've heard before has certainly moved away. And I think there are a lot of organizations looking at how do we build those skills for the future. It's interesting here, you know, Daphne, you talked about, we don't just need the learning systems, we need a separate way of credentialing. Gabrielle, you said it absolutely needs to deliver the ROI. You know, and Amy, you started off by saying we can't do it alone. If we need to hit those targets, it's got to be in collaboration. And so I'd love to now turn to Ravi, because I think you've done some exceptional things through a coalition type model. I know you've been in the press with your reskill and restart program. And that's actually delivered real outcomes for individuals when we start to think about employability. Do you want to share a few words on that and some learners? Sure. Thanks, Kate. I think what Amy mentioned is spot on. This is an opportunity for rescaling a billion people. It is a lot of heavy lifting. We're not talking about upskilling from your current jobs and your current work streams. We're talking about moving people between industries, moving people between work streams. In fact, a lot of industries have got decimated, a lot of work streams have got decimated. Digital technologies were otherwise being disruptive and COVID has kind of created an inflection point. So we built a consortium which kind of covers a value chain, if I may. And this value chain starts from assessing for potential. You know, almost two thirds of the workforce in the United States actually doesn't have an undergrad degree. So they're working on low-skilled jobs, but they potentially have an opportunity to play on new pathways. So low-skill, high-potential workers, how do you move them to high-skill, high-potential jobs? So assess skills at the start of the value chain. Identify the right pathways. Identify the online platforms which can provide you those right pathways. Then back it up with the apprentice program on the job marketplace where employers and job seekers can come together. We've built an entire consortium and Ravi, I don't know if it's just me, but I'm losing you a little on the audio there. Local state governments in the United States. Ravi, I think I missed some of that, but I think he was saying, you know, it's not just about the online learning, it's actually seeing that to the apprentice program and thinking about employers and job seekers coming together, which I think is particularly unique. You put together an ecosystem of really interesting different partners, and I know assessment's being quite a big part of that. Teresa, I wouldn't mind coming in and hearing from your perspective because I know that you do see skills assessment as a big part of signaling that online learning is a pathway through to employment. Can you share a bit more about the part that assessment is playing and it's linked to employability? That's to Teresa. Online, I mean, the assessment part is, I mean, it's critical. You've got to be able to assess do they have the skills they need to get the job done. And in fact, it's really interesting for us, we have thousands of job openings at Amazon in cloud computing, distributed computing, and we do a lot of our own. In fact, one of the things we've created was our own artificial intelligence university because we couldn't find groups that taught AI in the early days and still struggle with that. So one of the things that we've created for online kind of assessments, we talked about this a little bit earlier is badging and credentialing, but we've created under our AWS Educate and Academy program. And we also work with community colleges and universities on actual cloud computing certifications. And we've taken it from again working backwards from what we need. So we've actually helped them create the skills that we've told them we want to be assessed. And I'll give you a really good example with their educate. I'm a big advocate. We are teaching along the way so that they can work students need to work. We got to create economic development here. So we want them to work so badging and credentialing in areas that they could actually then go work and get a job and continue their education. So one we have as an example an Alexa badge. So we teach them how to code and develop skills in Alexa, which is machine learning AI. We worked on something with the American Heart Association around data lakes so that they're actually learning how to use real world things. And so, you know, if you work kind of backwards from what they need to learn, we're working right now on a museum badge. And it's all online. So, you know, virtual reality. So if you work again from real skills, and we're also an advocate of taking a problem and let them solve a problem through the technology. So they're learning and they're solving at the same time. So that goes a long way. But I will share within my org, all of my org have about 7,000 people take an AWS certified cloud practitioner exam. So from our executive assistants, all the way through every skill, because it's so important to us, we believe technology is like everybody's got to learn. Every job is going to have some coding or technology involved. To a lot of the panelists point, we need to make sure that we're continually teaching these skills. They can't be one and done. We have to have kind of the ongoing skills update and assessment cave. And Theresa, you bring a great point there about it's got to be demand driven. If it's going to actually be solving those employment challenges. And I love those examples there of how you're using VR and problem solving. So people are actually learning to learn in new ways through the new opportunities these platforms can offer. It really starts upending everything from not just talent management, but how we learn to learn, which I think is very exciting.