Gunnery is a very important task that I mean all crews must complete at some point simply because it enhances our crew cohesion and not only that but our lethality. If we were to actually have to go out and fight the enemy I'm completely confident that our crews would be able to get it done. So both Axeman and Banshee company, sister company from 137 Armor are out here conducting crew level gunnery to get after our crew level lethality. We're part of the training glide path to go from crew all the way through company CalFX to meet fifth core training guidance and so we've been able to maintain a very high level of operational readiness right here at the range and so I'm confident to say that the majority of the company will be able to fire off their own tank and qualify. So it's a hundred percent important to the crews to be able to come out here and do gunnery mainly because it enhances those individual crew tasks that aren't necessarily honed all the time and can be forgotten such as reloading the main gun, bore sight, even just MRS update for the tankers that know what it is. It's crucial to continue to refresh on those tasks and gunnery is one of those times that they really get to really hone in on it. So it definitely if it still takes the growth mainly by kind of trouble fire kind of you got to get out there you got to learn it at some point and even when things get hard you still got that battle buddy beside you that's saying hey let's get up let's go shoot and let's get it done so I couldn't be more thankful for them. My name is Captain Austin Bayjack I'm from Hackett's Town New Jersey and I'm the commander of Axman Company 137 Armor tank company. So both Axman and Banshee company sister company from 137 Armor are out here conducting crew level gunnery to get after our crew level lethality or part of the training glide path to go from crew all the way through company to meet fifth core training guidance. It's important for our unit level readiness we are training from crew level through company so we are both ensuring that our crews are ready for our own training for NTC came up in 2025. Additionally it helps support our NATO allies here because we're a posture to support and defend our European allies. Wonderful so the tables test different things and so we go through tables three through six at the range here three being dry fire so there's no bullets four machine gun pure five is with scale targets that are smaller and the table six the actual qualification table is where we we qualify our crews. What it looks like there are multiple engagements that test different things and so for instance some engagements degrade the tank so your laser goes out how do you battle site your tank if the stability goes out how do you go how do you engage targets without your weapon system being stabilized and then additionally there are multiple targets so there's a new gunnery table that we're firing currently in accordance with the new doctrine that has just been released and so now there the so-called widow maker in which there are four targets presented so a platoon sized element being engaged and destroyed by one tank so lethality looks like two different things one is how do I fight my tank through different issues that I have with it and then additionally how do I engage multiple targets rapidly and to standard so those are the two major training things that we get after here so first and foremost is qualification so being able to achieve a certain score on the gunnery tables but additionally being able to qualify on your own tank and so we've been able to maintain a very high level of operational readiness right here at the range and so I'm confident to say that the majority of the company will be able to fire off their own tank and qualify additionally there is a platoon competition between each other and so 900 or above out of a thousand is shooting distinguished and so each of the platoons are chasing after that goal within their own crews to shoot both the highest within their platoon the highest in the company and then finally top tank in the battalion so it's fantastic I think that what ends up happening is the crew becomes very close it leads to very close and connected cohesive platoons and then as a company like there is an extreme amount of pride and it's pretty core that is built and all happens or it starts at that crew level but just lends itself at the platoon and company level so very cohesive unit when it comes to operating day to day sure so I think a couple things I'll kind of break it down by echelon so first at the crew and tank level it's been eye-opening to operate because we're used to desert for bliss hot conditions right when we execute gunnery operating in icing conditions the tank responds differently has different issues to fight through even down to like the lubrication of our weapon systems and so like fighting through those challenges we're capturing them we're going to include it in our tax ops so that future units that come through are gonna be able to understand how to operate their tank to a very high level within these cold and icy conditions so that's a huge lesson learned from the crew level at the platoon level so we were storing our ammunition on the tanks for each gunnery table and so it is beholden upon the section sergeants the platoon sergeant to have very accurate logs that reporting for ammo accountability in addition to POL and so what's that what that's placing on our tankers is a high level of responsibility at the nco level at the section and platoon level and it's been really phenomenal to see that kind of come together and that's going to only help us even more when we're in the field and not and are operating under more austere conditions but it's been a very positive lesson that we've been able to carry forward with us and then at the company level I think just conduct of the range has been phenomenal we've been able to pull in some external VCs for gunnery table six but before then it's been run by the by the two companies I think what's ended up happening our two companies have become very close together we've become a very cohesive unit both companies and I think that's going to lend itself very well for a maneuver when we're going to be supporting each other on the battlefield in training to execute maneuver training so no it's this has been a great experience and I'm looking forward to continue to train with this company


Hello, everyone. It's great to be here. Today I will be talking about product-led growth. I know, I know, some of you might be tired of this topic already, or at least all the buzz around it. But I'm going to try my best to make this tactical, relatable, and practical. I'll be sharing some real-life examples of the work we've been doing at Amplitude. And I'll also be sharing some of the hard lessons I've learned during my past two years there. As product people, understanding the nuances and opportunities of PLG is going to be extremely important, especially how it relates to the rest of the business. You might be thinking, what's new here? Product is already at the center of your universe. But for the rest of the business, it can feel a little bit like our world has been turned upside down. So I want to leave you with two things today. I want to leave you with some practical ideas that you can take back to your company. I want to leave you with a better understanding of the business impact of PLG, especially how it relates to the rest of the business and your counterparts in marketing and sales. Before we dive in, who am I? I'm the very imposter here at ProductCon. I head up growth marketing at Amplitude. My team oversees user acquisition. We own our website, our paid and organic channels. We have a retention pod that focuses on user activation and engagement. And then we have a monetization team that focuses on driving upsell and cross-sell all on top of our product. So I work with PMs, designers and engineers every day as all my work evolves around getting users into our product, make sure they use our product, and hopefully eventually pay us for it. I want to start by us getting on the same page about what PLG means. I think chat GBT perfectly encapsulates the confusion out there. Is PLG a business model or a go-to-market strategy? How is a product so compelling that it can acquire its own users all by itself? And it's a freemium pricing model at prerequisite? This definition is a bit all over the place, isn't it? It's because we're confused. Confusion in, confusion out. It's not really chat GBT's fault. I really like John Cutler's free versions of PLG. He observed these inside Amplitude and I did too. And I also see them often on my LinkedIn feed. I would recommend you finding the post and reading through it in its entirety. So you can spot these different versions inside your company. But very briefly right now. PLG is a tool chest. It's about having different skills and tools, like experimentation, pricing psychology, SEO, that sets normal product apart from product-led growth. PLG is a motion and model. Here we're talking about an actual go-to-market strategy. A growth motion or motions. The last one is my favorite. PLG is a mindset. It's about feeling good about product, believing in the power of good products. Now, when I just started at Amplitude, I couldn't distinguish these three. And so I had a really hard time. I thought I was on the same page as my leadership, my manager, my teams. But I was wrong. I was ready to go, disrupt our go-to-market motion with PLG. But most of the company was in the mindset mode. You can imagine my disappointment. That was about a year and a half ago and since then I probably built about 276 slides on what PLG is, how PLG works, how we apply PLG across a customer journey, what PLG is not, the business opportunity of PLG, what teams should be involved. You get the picture. So my first learning for you here is that make sure that you and your business are on the same page that everyone is talking about the same thing. And if you're just getting started, you have to develop all three versions of PLG. You need to have the right skills and tools. You need to have the right mindset. And you need a strategy. I think one in three come naturally to most of you in here. So today I will be focused on two. PLG as a strategy. As product people, you're going to have to take a lot more accountability for key things like revenue and new sign-ups. So let's make sure you're ready for that. So just so we're all clear in this room right now, PLG is a growth motion across acquisition, retention, and monetization. Acquisition. Is your product able to drive new users to sign up? Retention. Are users able to activate and engage in your product without any help or intervention? Monetization. Are users able to self-serve checkout without any help from sales? Now, if you're lucky, you have or are you able to invest in product growth across each stage of the journey. Most companies do not. Let's take product acquisition as an example. Let's look at Zoom. Someone signs up for a Zoom account, create a Zoom link, and invites their friends to the Zoom call. All these friends are going to experience the power of Zoom, see the value from it, before even having signed it up, and then they're prompted to sign up at the end of the call. Calendly is another great example here. You sign up for Calendly, you invite everyone in your network to use this Calendly link, and they get to experience the product, Calendly, before having even signed up for themselves. If we continue with the Calendly example, they also have Prata-led retention. Super easy to get started with Calendly, super easy to use Calendly. They also have Prata-led monetization. Just put in your credit card and you're on your way. No sales needed. Now, most companies do not have it this easy. They have to rely on marketing and sales. For example, Amplitude, we rely on my teams to drive acquisition, marketing led acquisition. We also have professional services and customer success teams that focus on adoption, making sure users are engaged in our product and know how to use it. And we have massive sales teams that are solely responsible for monetization. So to Zoom and Calendly to some extent, and that's because most successful companies have two or three motions running simultaneously. And that's what I'll be running through next with you. How Amplitude is driving growth across acquisition, retention, and monetization. And how we blend the Prata-led motions with our existing sales and marketing motions. So let's start with acquisition.  You have to be Zoom or Calendly. First, you're right. Second, thank you for listening. We don't have true product led acquisition at Amplitude, but we do have the ability to have the product drive new users into existing accounts. We know that having more users inside Amplitude reduces churn, it increases retention, and it expands the use cases of Amplitude within a company. So because of that, we're always working on improving our activation, sorry, our invite and sharing experiences. Last year, we made some significant improvements to our sharing model, making it easier for users to share content outside of Amplitude. By improving the share model, we saw a 41% increase in conversion to share a chart. And what's more, we saw an overall 4% lift in users sharing. That's huge. We also made minor improvements to our invitation flows, improving simple things like our email prompts, and creating an extra email nudge for users who forget to sign in when they're invited. Small things like that can move the needle. We also made improvements to our new user sign up flow. If you've ever signed up for Amplitude, you might remember it's kind of a process. We ask you a lot of questions. So last year, we worked on slimming the screens down, reducing the number of questions we asked people up front to reduce friction. We also added things like enrichment, so it would be easier for users to sign up. By reducing the number of screens, adding enrichment, we saw a 60% increase in conversion from sign up to creating an account. So while you may not have true product-led acquisition, you're still able to invest in expanding your user base for existing accounts using your product. Now, this brings me to my lesson number two. While it's really rare to have true product-led acquisition, like Zoom and Calendly, having a free plan of your product can help unlock acquisition for you. It reduces friction and increases value up front. Having worked with many other SaaS companies in the past, I can see the stark difference. It's way more efficient to have a free plan. I have to do less work, right? We just get users into the product. It's classic show, don't tell. At Amplitude, we know that the most efficient way to monetization is to get users into our product and see the value for themselves. So if you're right now out there building a business case for adding a free version of your product, make sure you include the potential upside of having a free plan from an acquisition unlock perspective. Now, let's talk about retention. Retention is huge for us at Amplitude this year. Retention is split into two parts, activation and engagement. Our big focus this year is activation. We define activation in two parts, set up and aha. Set up for us is when a user connects a data source and creates their first chart. A ha for us is when a user either saves the chart or shares the chart, both indications of value received from Amplitude. Now, all this talk about metrics brings me to my third learning, which is metrics. With PLG, all your key business metrics are going to change. Now, quick pulse check here because this is a product conference and I'm a marketer. How many in here know or think they know what an MQL is? Marketing qualified lead. I need hands. I see a few hands, not as many as I would have hoped. It's not really that important. But it's important if you want to know what keeps your counterparts in marketing up at night. My point here is that in an old business model, pre-PLG, all your metrics would be key sales and marketing KPIs. In a product-led growth world, they will all change to product metrics. As product people, it's up to you to really understand what is an activated user? What's an activated account? What does a retained user and retained account look like? What is driving monetization? What kind of retention drives monetization? What are the key behaviors? This takes lots of deep analysis and work to figure out. At Amplitude, we're constantly working on these metrics and tinkering with them ourselves. And once you find them, you need to share them with the rest of the company. You can't be the only team that cares about these metrics because you will need help. And so you have to go out and talk to everyone about these key metrics. Don't get me wrong. At Amplitude, we still care very much about MQLs and SQLs. We still have to run our business. But we're warming up the business to these new product KPIs. We're surfacing them every week, reporting on them, educating everyone on them, bringing them along on the journey with us. So at Amplitude, as I mentioned, we're tackling activation this year. As you can see from this very pretty final chart, we have quite the drop-off from a user creating an account to them actually connecting a data source. We know it's complicated. We know it's a technical thing to set up Amplitude. And this brings me to my fourth learning, which is that product growth is a team sport. I know we just landed on a definition of PLG five years ago. Five minutes ago. But unless your product is as intuitive and simple to use as Calendly, you need support. Everyone needs to care about this. So at Amplitude, while product and engineering are the ones striving the in-product activation roadmap, we have many other teams who are gold on activation metrics as well. Take my team in growth marketing, for example. We work on one-to-many onboarding programs. We build trigger emails to force or create a habit. We work with our education teams on building education materials and making it easy to get started with Amplitude. We continue to improve our documentation and support forums. So when a user has a question doing setup, there's an answer right there for them. We even have sales involved. It just shows how important activation is for Amplitude. So while we're making investments in the product side of the house and activation this year, we're also going to make investments out of the product. If you're an enterprise customer with Amplitude today, you get a 60-90-day implementation program where we help you with what questions to ask, what events to track, properties, how to use the product, how to get value from it, how to capture insights. Now, we can't do that for every customer, especially not the ones that don't pay us yet. But what we can do is that we can turn that enterprise experience into a one-to-many version to make sure that every single person, every single company that signs up for Amplitude is successful. We know it's a good investment because activated and engaged accounts drive monetization. Perfect segue into monetization. So for us, it's not product-led growth with monetization. It's product-led sales. It's product-led sales because we don't have self-serve monetization yet. It's very hard to add product-led monetization to an existing sales-led monetization motion. I'm not going to lie. When I was doing research for this talk, I was trying to find some success stories out there, and I could not. So hopefully Amplitude will become a success story on this very soon. Until then, we have product-led sales. Product-led sales is a very scientific definition. Product-led sales is if sales led growth and product led growth had a baby. We're mixing the motions together to make money. The key ingredient here is product usage data. Using product signals to identify accounts that are ready for having a discussion about upgrading. I don't think product-led sales as a concept or motion is entirely new. We just didn't call it this. We have had like customer success teams using customer health scores to reduce churn or mitigate churn. Even sales have had access to key product signals to identify accounts that have gone over their current limit. But I think what has further popularized product sales is the rise of free and freemium. Now users are trying before they buy. And while they're trying, there's so much to learn about them. So at Amplitude, we're still getting started on our product-led sales journey. But we've already learned so much. 50% of our new paying customers every year come from our free plan. So we need to know what they're doing in there. So one of the biggest learnings for me so far has been that proactively identifying accounts in our free plan and surfacing those two sales has performed much better than optimizing in the product for handraisers. So what we did is that we built a simple model where we looked at our free accounts and our paying customers and we looked at the behavior they had right before they were upgrading. And then we scored these accounts accordingly. And the accounts with the highest score we gave to the sales team. With these accounts with PQA, we saw a 3x increase in conversion from lead to sales pipeline. That means that sales is three times more likely to convert a PQA lead into a sales opportunity than any other lead, including an inbound demo request or anything like that. Besides that, we also have surfaced a lot more relevant data from the product to our sales team. We embed key account metrics directly in sales force for them. And we build an amplitude dashboard where they can go in and look at their accounts usage. What kind of paid features have they been interested in? Who were interested? What kind of charts do they use? Are they just using basic segmentation? Or do they have more advanced use cases? Who are the most querying users? Who are the most consuming users? So on and so forth. Arming sales with the right data to have the right conversations. This brings me to my last learning. For product sales to work, you have to become extremely close with sales. Like BFF close. Now, I know what you're thinking. Sales people are scary. Can't do that. But I'm here to tell you they're not that bad. And they're really not that bad if you share a sales target with them. When we were just getting started with product sales at Amplitude, we were gold. The product team, we were gold on surfacing as many handraisers as possible. So we were optimizing for that in the product. Just getting as many people in the product to raise their hand and wanting to talk to sales. And then we threw those leads over the fence to sales. The more leads we generated, the better we were doing. The more leads we generated, no more pipeline was generated over on the other side. Wasn't working. Now we share an actual pipeline target with sales. There's joint accountability across the journey. We optimize for quality, not quantity. And we make sure they have the right enablement and product usage data to have the relevant conversations they need to close these accounts. So where do you start? For Amplitude this year, we're going to continue to invest in product sales. Both for our land use cases, but also for our expansion use cases. We're going to continue to invest in self-serve monetization, making that a reality for us. And we'll be continuing the company-wide effort to improve activation. We know that improving activation is what's going to fuel growth for us this year. For you, it depends on where in the journey you're at. If you're just getting started, hopefully some of my ideas and examples were helpful. If you're way ahead of me, I'm jealous. Good for you. But I want to leave you with this. As product people, you have a unique opportunity to take a front-road seat right now. The way people buy software has changed. 20 years ago, it was sales-led growth with steak dinners and box seats. Ten years ago, it was marketing-led growth with the rise of inbound and HubSpot. Now it's product's turn. It's your time to shine. Seize the opportunity, have clear metrics across the product-led journey, and bring your counterparts in marketing and sales and the whole company on the journey with you. Most importantly of all, make sure you evolve. If not, your competitor will. Thank you.


So, hello everybody, thanks to join so early to listen to me. That's, yeah, surprise. First of all, who am I? My name is Luca Pizzamiglia. I'm an Italian guy working at Trivago. I'm a FrubisD enthusiast. I use FrubisD at least for 10 years now and I received my port commit bit in August 2017. What I'm doing usually is trying to look for new use cases for FrubisD. I really believe FrubisD is a very solid platform and can run basically everything. Specifically in Trivago, I built packages, customized stuff and so on. It's early, Sunday. I had the ability to provide gadgets, yeah, socks. So every time you have a question, you receive a sock. So the first time questions, so don't be afraid. Please speak out. Make this presentation more interactive. Help me. It's really for me also. So this is for today. We'll talk a bit about JSON port, what the port is, something I presented two years ago here. What is a service mesh or what I believe the service mesh is? What is the port image, future work and then your question. The port framework is something I started more than two years ago, actually. The ambitious goal was to basically was, yeah, there is no docker, we know, on FrubisD, at least not natively, but actually is a really nice pattern or at least something that is quite easy to use. But in FrubisD there was actually no container model in general. So why don't you try to do something similar using all the technologies that are already available? Like ZFS, JL, PF, Finet, Sercitiel, CPU set and so on. So I came up with this framework as a set of, basically it's a tool that allows you to manage JL's dataset, I would say pretty easily now at that time was quite rough. But yeah, that was the logo at that time. So a port comes from a port, it's not the other meaning of port. This is someone else suggested. And that was basically the state at that time and how that fit in here. Who does know what a service mesh is or use, wow. So you have to, I have to explain the best of my capabilities, what a service mesh is. A service mesh in general is an abstraction where a developer here can submit basically a job or that would see what a job is to this very big black box, cloudy thing. And then the user can just use the service that is exposed by this job. I will go a little bit in detail. This is not like, I mean it's similar to Kubernetes but it's not Kubernetes. So basically the developer has a job description where it specifies which container to use and which port should be exposed, which type of services is there and so on. Here we have a central point in this case, the orchestrator. It received this job description and they say okay, you want these containers up and running. It looks for the, in the worker cluster, so we'll basically transfer the allocate or orchestrate basically the execution of these containers to the workers. The workers will download the container images there. We'll be spawned basically those new containers. The servers will be up and running and that is one side of the story. Then if you use a network service, you have a service discovery, something, a server basically. We use console here. And what the orchestrator does, we register your service here and say okay, now I decided that your server is running on this machine, exposing the port, I mean it's running on ports one, two, three, four, five, whatever. So these information are registered now here. So the service discovery will know okay, there is this service with this name that is running there. And what it's also doing is continuously checking if the service is up and running. So the health status of this service. And this is the so-called control plane. So now theoretically if you want to reach the service, I mean the orchestrator decides where the service is running so you don't know exactly where it is. So you have to basically interrogate the server discovery. And this is called, I guess, the control path. So basically you can know basically where your services are running. And you have another element that's called, is a proxy or a layer seven load balancer. This load balancer or this proxy, you can configure it basically to say okay, when there is a user that wants something, look for where the service is running. So basically the list of the backend is continually updated to know where all the services are running.  And every time there is a new service that is, the same service is replaced. So the old container got destroyed and a new container is running for instance here. The information is updated in the server discovery. And so the proxy can get updated information and route the traffic in a different direction. I hope I explained myself. I'm not really a super master of this all elements. Any question? Cool. It does not seem a freebie upset to me because images, nomads, and there are products names. So to repeat the question, this doesn't seem a freebies these things to me. Yeah, of course, that was the problem. Actually, those are programs that runs everywhere. They're mainly, I guess all of those are go programs. Yeah, that's actually the point. How can you have something like this on freebies? And that is the solution that I'm presenting now. As you see, the real issue is this container images. There is normally no such solution that has used these names. And exactly that was what is really needed. The features that was really missing was the ability to deal with images. So creating and all of the images. So you need to create and export an image. You need our image registry.  Docker Hub and then the ability to download and then import an image. That was the set of features that we worked on. But more importantly, what is really different is the paradigm. How you work with this kind of, I mean, the workflow is really different. We are used to create jays directly on the machine and then you'll configure them and you let them running. And the developer will just give you the PHP code or the Java code, whatever is running there. But you are directly managing jays on your final machine. So the paradigm here is different. Someone is working on the image creation. So in the EJ image. And someone else is deploying it. So the paradigm is very, very different. And also how the set of tool that you need is different. It's an almost unexplored area, I'd say. And I will discuss before the deployment of the jail or the available notes to make that visible. And then we'll discuss about how to create an image and all those things. There is already a lot of complexity all around. I mean, a lot of new names, new components, things that are going there and there. So the first implementation of POT as basically a jail was composed by multiple data sets. There was the base. There was Monterey-Ridoli and shared with other jays and so on. For this kind of scenario, when you have those moving parts, it was too complicated. So I decided, OK, just go to one single big ZFS data set where I put everything there. So it's easier then to create images how I export a POT.  You customize it. You put the software that you want. You take the snapshots. And then with ZFS send and compressed tool, boom, you create an image. So basically what you want inside your jail. And then you have a file. A file is easy to move. To import a POT, well, is that the way around? The question is, only ZFS is used? Yes, ZFS is mandatory. I don't support other thing. If you want to use a different, please support a patch. I'm happy to extend support. The whole POT framework is heavily based on ZFS because I have all the nice feature, snapshots, a lot back and everything's there. To import a POT, when you download a file, then you can just uncompress ZFS receive. Boom, you have the same basically content somewhere else. And then you can clone the snapshots. I use the clone here because if you have one image, then this image is reused. Imagine an Nginx server and you have the same Nginx with different configuration. It's easier than to clone it and just change the configuration file than to untar again and do as ZFS receives. So that basically is a slight short optimization. Question so far? Other question? Question was if I tried to use Behive. No, because it means completely the point. I mean, Jail is very, very light. Behive is not that light. I mean, Behive makes sense when you have different operating system that you want to run on privacy host. If you have the same operating system, use Jail. I mean, I want to focus on native support. I mean, it has to be native. It's really faster. If you have Behive, you don't have the same performance because full virtualization with Jail, you have a light virtualization. So it's just the operating system side. I'm waiting for your patch on that topic. So why we choose Nomad to try to implement this service mesh dinosaur? Nomad and console are, say, privacy friendly in the sense that they are already in the package system. So you can just install Nomad and console and have them up and running. So console is the service discovery and Nomad is the orchestrator, basically. Moreover, Nomad has an internal structure that allows to add additional driver. And a driver is something that is basically supporting a different type of containers. So there is a driver for Docker container, a driver for, they call it this Java thing, means that basically you have a jar, a target, and then it's executing the JVM right away. So different types of containers. And what happened here is why can we not write an additional driver to let Nomad interact with POT. So Steven, let me see here. A colleague of mine just developed the driver. I mean, tried to create this bridge between Nomad and the POT framework to interact with Jail. So the driver is already available. You can install it on your own. So it's a mature product but stable enough. And this is the job description. I will go through that a little bit. This is a very easy example, even if it's very long. You have on top the name of the job, the job, as you see, it's a service. Then you have the concept of group a service can be composed by different containers. So you want to join them to provide a service. In this case, the group is called example group, the fantasy, and it's composed by only one task. It's only an engine expo. And here you specify the driver. Saying the driver is POT, then Nomad will look for a worker that supports POT and will be able to execute a Jail in that way. I will just jump here. The part that was strictly part of the driver, so basically you say the configuration, where to download your image. You say the name of the image, the version. The command is basically optional, but it's the entry point of your Jail. What is the first command that's to be executed when the Jail is spawned. Then we have here, that's the service part. So that is all the information that you give to Nomad to spawn the Jail. And this part, the service, is what to register to console. The name of the server will be WebExample, for instance. And then you see this port HTTP, port HTTP, port HTTP here. This means that the port 80 in the container will be mapped with a different port outside. There is an automatic PF redirection rule that is injected to the node. And this new information will be registered in the service. So we will see in console there will be a different port. Locally, the Jail, we have an NGNX running on port 80, but then you have a PF rule that makes the redirection. And in console you will discover basically where this thing is running. Questions so far? Yeah, the port registry, there is a slide later. I mean, the question was about what is this port registry, is a web server with files. There is one slide, exactly on that topic. It's complicated. It's all the second part of the, was about that. It seems easy, but it's not easy. I mean, if you think why I'm not using IOCage or other Jail frameworks that are out there. First of all, because I can do my own. It's open source, so you can do whatever you want. Now, this service, it doesn't work. There are features, I mean, a container has something different from a normal Jail. For instance, this command in NGNX is not forking in background. What it means, means that if you use that type of command with Jail, with the Jail command line, the Jail command line wants to return because the first command is basically executed by the Jail command and is just keeping your, still your shell. It's not returning. Jail has this standard assumption that the command, the exact start of the command that you gave, basically at a certain point will fork and will return. In this case, it's not the case. I mean, usually containers are working with blocking commands. So basically, they keep the NGNX is pointing and they're serving directly with traffic. Period. It's not forking. And that creates a series of issues. In this case, for instance, the post start hooks are not executed at all because what the Jail command is doing is just working like create the container, spawn the commands, whether it's over and then executing everything that is after the post start. The post start is never reached because the command is keeping, basically it's not returning. And a nice thing that is nice, took me some time to discover is usually those containers are ephemeral, means that if the process inside the container disappears, the container disappears itself. Jays are by default persistent. So if there is no processes there, the Jail will stay. And IOCH, for instance, only supports persistent Jays. The no persistent parameter is also applied by Jail start as a post start hook. And it didn't fork, basically. I have to find some weird way to be able to have this no persistent flag applied even when a command doesn't return. Last but not least, Jays not clean up themselves. In other words, the post stop hooks or the pre-stop hooks and so on, if you have a not persistent Jail, you mount your devices, you create your PF roles, then the Jail disappears.  But you don't have any notification that the Jail disappeared or you don't have any way to register a callback. What you do usually when you call Jail minus R, that is the Jail stop, is the command that is executing those hooks. So I say, okay, the command says, I have to stop this Jail. So I run the hooks, I stop the Jail, and then I run the other hooks. But if it's not persistent, the Jail will just vanish. And there is no centralized demon, basically, that is controlling what's happening with the Jails. They are just, it's an internal operating system structure that just, it's like when a process is going away, you don't get any notification per se. We solve this basically inside Nomad. If Nomad has to control that the containers are up and running, if it's not running, it's just calling a post stop and it clean up basically almost everything that way. But we don't have generically a way to clean up not persistent Jail. Questions so far? It's clear and too fast. Can you repeat the question? You have to avoid, the question was if you can run something that doesn't demonize, you have to change your command line to avoid that it's demonizing. Oh no, so yes, you can. So the best practice in this world is to have one process per container. So for instance, we have a radius instance and the exporter that is just basically another program is running in a different container. Because if one container dies, you just replace this and you don't replace the whole service basically. That is a common best practice in this way. Taking care you can, it's suggested not to do it. But yeah, you can put more. You can create your own script and spawn different stuff. The point is that the last one shouldn't return. Network. Currently we support two network configurations. One is called host. Meaning in Jail jargon is inherit. So basically you're using the machine network stack. The other one is the public bridge. Meaning that you have an internal bridge. Every Jail has a vNet. It's on stack. It receives one IP and is exported outside. You have an app that is showing everything and you have redirection rule for every service that you want to expose. And those are supported both in port and in nomads. In port, we have also other two network type configuration. One is the private bridge. It basically works exactly like the public bridge. So basically you have one bridge and you attach your Jays there. But the public bridge is one for all of them. With private bridge you can have a bridge for a subset of Jays. So I have three Jays. I want them isolated. So you can have dedicated bridges just for this type of, yeah, for your work. We want to add this to Nomad as well. So it makes sense. For example, for a group you have three different Jays. You don't want them to be on the same unique bridge for many reasons. Currently we have a small issue because the driver was at the task level. But the bridge works at the group level. So it's one obstruction level above. And apparently on Nomad 010 there is this ability to have something at that level so we have to figure it out. Currently it's already working with the public bridge but if you put 200 Jays then this bridge will be overloaded and probably the performance degraded. So this is an area where we want to improve. The last type is alias. This is the typical jail way on doing things. So basically you assign a static IP to your jail and that will be an alias on your network card. In this dynamic cloud environment it doesn't really fit well because you can theoretically have this jail that is moving from one node to the other and this IP will follow. But it means that you can only have one instance of that container. You cannot have multiple. I mean this cloud thing is designed to provide you horizontal scalability so you need more horsepower than you have instead of three web servers, five web servers, ten web servers. And in this case we are limiting to one because there is only one IP that you can assign. And also when you want to respond, I don't know, you want to respond the same, there is a new version of the web server so you have to redeploy your job where you have to remove the former job redeploy someone else and then you have a small downtime between those two times. So I don't know if it's really something we want to pursue. I'm still on the side. Can you use it? Can you play with it? Can you try it? Yes. There is a lot of different tools that probably nobody knows or you don't know. The configuration, it can be a little bit tricky. So... Michael? Suggested, why don't you do a mini part that is basically a mini cube? So kind of an instance that is run on one node so you can play with it. So there is... There it is. Mini part is basically a service mesh. What I described before, all these console, normal, traffic components, everything. This is running on a single node. So you can install mini part as a package. There is one script that will change some configuration file on your machine and then you can start it and you have the same environment on your machine. So you can... It's not for production. Not at all because you don't have high availability and for tolerance, of course. It's running only on your machine. But anyone can try. So if you want to give us both, this is the way. There is... a readme page with the instruction to install it. It's not for production. If it's not clear, it's not for production. Questions so far? Don't be shy. I still have five socks. Want to see a demo? You sure? We'll see. So... This is how mini part looks like. I have mini part running basically on my laptop. And what you have, you have a console running on port 8,500. You have nomads running here. And you have traffic. This is the front end. I don't know how to do it, but... it's here. So... how it works, where it is. For example, first time. Yeah, yeah, yeah. Now I'm... So this is a job description file. For a mini part, you specify a data center. Just to know where it can be spawned. Internal abstraction... of abstraction. That's the very hello world example. So basically just define a service called alofaustem. The configuration is... I mean the standard Nginx. We will see the standard page of Nginx. Where to download the image. Which one? Which version? And to port 80 will be mapped off the top side. And that's it. That is the minimum thing that we need. So how it works is we just say nomad run alofaustem.job And the command line tool is now speaking basically with the... with the server. Say, hey, this is the job description. Oh, the resources. So the question was about the resource limitation. Yeah, I missed that point. So basically I'm using CPU set and it's complicated. Theoretically the resources that you specify are used by the orchestrator to understand if your cluster has the capability to run this kind of job. So the CPU is kind of a bogus megahertz. So if your machine is running on two gigahertz and you say that your job needs 100 megahertz it's basically, okay, your machine is two megahertz and it's using like to say five percent something like that but it's bogus somehow and it's used mainly for scheduling and orchestrating. Memory is, I'm using RCTL to enforce memory but it doesn't work like in C groups. RCTL is trying to limit the amount of memory used by your jail but if your jail needs more memory it won't kill your processes. And that is not ideal in general because then you can have people exploiting that fact so you can use basically more memory invading the others. They have these neighbor effects so-called. One thing that you have to work on is an OM killer. So basically trying to monitor the memory consumption at least really is not able to stick to that value. You have to kill it. But those are resources that are with RCTL and CPU set basically they are enforced. Now we have submitted the job, triggered, created, what that means we just look now at the UI here. So we have our first and job one after running and you see there was only one instance. It's running here. That is basically where it's reachable. And then there is the welcome to NGNX. We have the same information here. Basically the service is registered. We will have also some tech because then you can basically subscribe to specific service that has this tech and so on. And what you have here is also basically now console is keeping this information up to date. So you know he knows where it's running. You see there is not 480 but this weird number. And then it's performing continuously checks. So if that disappears that will be marked as red. And basically you can ask the console to give me all the services and give me the services that are in healthy and not running. So it's trying to keep this information always up to date. And then you have also here basically this is traffic. This proxy. It receives the information from console and it had that here a new front end called Hello Foster because that was the name of the service and if you basically curve with this host header it will be already directed to the back end Hello Foster this year and those are totally dynamic. So basically continually asking console oh there is a new service is adding a new front end and is adding then a new back end to serve this service. I see that I have to run I have another demo but yeah I don't have time to show it. It seems nice, seems working but what is the problem? Creating the image is actually an issue. I mean what is the image? How can I create it? How can I provision my jays? We need to provide automation reproducibility that's a heavy topic for me has to be fast. The images should be not gigabytes but reasonably small. The solution should be portable and usable. Portability especially because the current solution is basically flavor. So when you run a pod create to create a pod you can specify flavors that are basically provisioned scripts. So you can say okay if you want engine x boom I have a script that just install engine x. I make engine x an example and then you can have basically multiple flavors that do just a small set of operations. There are two flavors after the box FBZUpdate and Slim. FBZUpdate is running previously updates before anything because how it works when you say okay this is the base 12.1 is the loaded base.txt but that doesn't have any patches so you always have to run a previously update there and this is a work in progress. It tries to reduce the size of the image. The leading, Selang, a lot of files that if you're running web server you don't need. You don't need GDB, you don't need a lot of things so I'm just removing hundreds of megabytes of stuff that in that scenario you don't really need. Two problems. First of all it's not user friendly. It's not really user friendly. It works obviously only on FreeBSD. What is a shell script that use J's? It works only on FreeBSD. What developers use they don't use FreeBSD, period. I have a FreeBSD, I'm the only one in Trivago, we have a thousand people, I'm the only one with FreeBSD laptop. I mean nobody. Few people use FreeBSD on the laptop but it's an issue. And the other thing that is not really nice if you, let's say, oh, new version of NGNX you have to recreate everything from the scratch. So you keep your flavors there basically those booters, but then you have to create everything from the beginning. This is still an issue. The second one is not that bad. It can be better. Pop machine. Establishment again. Why don't you imitate Docker? With Docker machine. So basically if you want to run a Docker container on a Mac, I mean inside Docker there is Linux binaries but they run on a Mac and how it works. Basically what you have is you have to install Docker machine that is installing Linux kernel in a virtual box or in a VM essentially and it's a security Linux code there. So why can't we not do the same? And pot machine is this project to create basically it works on Mac or CX and on Linux. It will use vagrant to spawn a privacy VM where you can execute and run your jails. It extends all the common available on POT so basically you have two, three comments more and every comment that is not special kind of you have pot machine, two or three comments. The other are just forwarded to the FreeBSD VM and you can have your jails running on Linux more or less. So it's an important thing because it's breaking the barrier so everyone can theoretically now play with jails. You don't need a FreeBSD installation to play with jails. The second work in progress is POT file. We are just looking how it goes. So we are trying to imitate Docker file. Instead of having that complicated script that you put there and there you can just have this typical run from whatever. It's experimental. It's not adding really or changing. It's just translating this POT file in a flavor. I mean the flavor is what actually you want to run is the same. It's not user friendly so that's basically a wrapper where people is more used to use to create POT images. I would encourage you to try if you have some issues. He will fix them. But let's say it's a work in progress so we are still evaluating. We don't want to create a complete new format because I don't think it makes a lot of sense. But I mean also Docker file is not the best format ever so you are kind of puzzled what we can do what we shouldn't do. Registry. Easy to be server with those files you put there. So when you do a POT export you create your image and then you have those files. Copy them over. Period. You have to maintain your own registry basically. I don't want to maintain any registry at all. I don't like the idea to download binaries. I don't want to maintain to have the burden in terms of security having binaries floating there and there. But I'll be more and happy to have a Flavors Catalog. You need engineers. This is the way to create engineers so you have those scripts already prepared. There's a very approach similar to what Bastille is doing. They have all these templates where you can just say, okay I want the jail with man-cache. Boom. Done. You don't need really to understand how it's already there. So there you have really a great approach and maybe I can reuse those templates. I don't know how to speak with the Bastille BSD developer. This kind of registry that I showed in the examples is a virtual machine running somewhere. It's a web server with those two files. Period. It's good for examples so you don't have to create your own and so on but it's not for production use. Not at all. But it's just something so people can try. You can try it. You don't need to create a new image and so on. But it's not Docker Hub. You cannot upload things. It's a web server. It's not for production. Wow. Two minutes? Three minutes? We want to do a lot of things. Many were already spoken but I'm running out of time. We want to focus now on more on image creation because that is where people can then use it or at least try to use it because if you have wow, you have your service mesh but you have nothing to run it doesn't make any sense. So there's more or less especially with many ideas but no time to do it. The dual stack support will come. I get no, it's not here. Olivier is talking with him. Good idea to have IPv6 support. Currently everything is on IPv4 but we will have IPv6 support to provide dual stack. So your container can will speak IPv6. Really thank you to be here. Thank you for listening. I will also my contributors. I put nickname because I'm not sure that they want to be, I can write their name but really thanks. Question. Wow, many. And there's another. One. I'm almost over. But I guess I have time for many, how many questions can I, I mean I will. Do you have any ideas about persistent storage? It's relatively easy. It's not managed here. Basically you can go on the machine, you can mount datasets or directories inside the container. So it's up to you what to use but if you have on a specific note you can pin basically a container running on a specific note where you have a dataset and your persistent storage there and then you can mount it inside the jail. It's a feature I didn't show but it's there. Yeah, you specify mount in. You can mount in as I said a folder or ZFS dataset. You read only if it has to be read only only and then you can pin the job directly to a specific machine. I don't see why not. I use PF. I don't know how your cage is using PF. I only think what it's doing is it's using two anchors that I add and that's it. So I don't really pollute too much and then I have my specific dataset under you specify where to put your root where to create all the datasets and if it's different then it should be compatible. I never tried it but keep me posted.


Welcome to the first talk after lunch. Hope you had a nice lunch. We have here Ian Oswald, who is going to talk about the starter data science process for software engineers. So please give him a nice welcome. Hello, everyone. Welcome back after lunch. Hope you had a good lunch. So I'll be talking on a standard data science process for software engineers.  So I'm going to ask you to self-identify in one of two ways. You're either more of a software engineer or more of a data scientist. So put up your hands. If you had to self-identify as a software engineer, first of all, it's kind of predictable given that. And how many are data scientists in the room? OK, so hopefully you will also learn a little bit, but just a little bit. The rest of you, I hope, will learn a bit more. So who am I? My name is Ian Oswald. I style myself as an interim chief data scientist. So I'm a practicing data scientist. I write tools and intellectual property for clients. And increasingly, over the last few years, I've stepped in to help teams and companies that lack senior leadership on the data science side, trying to shape the teams, help them figure out what they're doing, what is working, what is not working, and how to figure out how to make a project work successfully. And I've been doing this for nearly 20 years. I've been around for a long time doing all of this. I'm very driven by all of this. It turns out you cannot learn enough. There's just so much to learn. And it keeps changing all the time, even to the point that in Sylvain's wonderful keynote this morning, I got a bunch of new ideas. And so I carried on typing for the next two hours. And we've got more bits in the live demo that will happen later on, because there were just too many cool new things going on in that keynote. So yeah, I quickly work to help teams build strategic data plans, data science plans, getting data products shipped. And then I'll work with them and make sure things get delivered. So I really enjoy now trying to shape up data science projects to make them work. I've had with my 20 years experience, lots of experience of things not working and going horribly wrong, typically with me as the only data scientist in the room, or the only AI person, back before we had the term data scientist. And it was kind of terrifying and really depressing at times. And I try to change that conversation now. So at the top there, there's the PiData logo. I'm one of the co-founders of PiData London. That's the largest Pythonic data scientist to meet up on the planet. We've got nearly 10,000 members now. I'm super proud with what we've built there over the last six years. We have our sixth annual conference in the middle of July. Who's coming to PiData London's conference? A handful of you. The rest of you want to come as well. You will learn so much. There are still some tickets available. So I work with big companies like the ones you see up there, hotels.com, insurance company, QBE, Channel 4, and also lots of small and mid-sized companies. So I've got a nice range of the stack across many different verticals about how data science works. I'm also one of the authors of High Performance Python. And O'Reilly have kindly sent me a box of my books today, so I'll be doing a signing. So if you want a free copy of the first edition, then I'll be doing a signing for that outside in the coffee break. And I'm working right now on the second edition. And the first edition is still correct. It's not wrong. It's just Python 2.7, and it's lacking a few things like pandas and Dask, and that'll be added for the second edition. And I'm also now on PluralSite doing video training. So when does data science show value? So I like it when I work with a client where the management are numerate and they're asking good data-driven questions. I've worked in organizations where we need magic. We need to make more money. Just go and look at the data and give us the magic, and it'll all work. And that's rubbish. That does not work at all. I need numerate management who can ask sensible questions and understand the quality and the problems in their data and their process so that we can do some kind of change process management to get a product shipped and doing things in the real world. Particularly, we need suitable data.  That's the way this works. We need good data. And we need well-defined achievable outcomes, so sensible outcomes that let us bite off a sensible problem and deliver something that works and then incrementally improve it. And change has to be enabled. We have to have the permission not just to hear that we want change, but actually deliver change, change an organization, change a workflow so that things get accepted and put out into production. We need to check the business need. We always need to know, so as an engineer, approaching this problem, you know, I want to do some data science. What am I going to do? Well, you need to do something that delivers value inside the organization. There has to be a change driver behind this. It's not good enough to do it because it's cool and it's deep learning with the big data. That doesn't matter. It matters that I can deliver change inside the organization. So is there a burning desire in the company to change a thing or is it somebody's aspiration and idea to have a deep learning project because it sounds cool? That won't actually deliver value. That'll be a waste of your time. We don't want to be doing that. When I was speaking, I had the privilege of giving a keynote out in Lithuania a month ago at Picon, Lithuania. One of my colleagues out there, Jonathan, gave a really nice talk talking about the data science process, and he gave a really nice example. He works in ID matching, where you take a photograph of yourself holding up an ID card and a photograph of yourself holding up a passport, and then they have humans checking these things, and they wanted to automate the process. Do we do big data and a deep learning and then try to do the whole computer vision thing? And he said, no, of course not. It'll be a silly way to approach this problem. Rather than having humans checking every single image to see if they can clearly read the name, we do a bit of OCR. And if the OCR recognizes name and some name, and on the other document, name and some name, and those some names match up, then we say, hey, that works. And for all of the other documents, the other 50% or that doesn't work, will we give that process to a human? And so the machine automates half the problem easily with some off-the-shelf tooling and an afternoon's coding, and then we identify the bits of the problem that a human is required to work on. And then we can incrementally improve the harder cases and get the machine working on these harder problems. And so looking at incremental value delivery. So I'd encourage you to look at that talk that Jonathan gave the videos online, I believe. And it's important that this solution is automatable. If it's not automatable, then it doesn't matter how much deep learning, clever data science, whatever you do, you can't put this thing out into production. I really think we all need more project specifications. This is a slightly unpopular opinion because it means we don't get to go and do the core thing with the machine learning, but instead we write down some things and we get agreement from our colleagues. But in 15, 20 years of doing this, I've realized you really need to write these things down, write down the shared agreed problems and solutions and milestones and the unknowns that might torpedo our project and get agreement amongst all of the stakeholders. Otherwise, everyone assumes different things will occur. Everyone assumes that I know what the problems in the data so you know too, you know it's happening next week, you know it's happening in a month's time. And then when things don't go to plan, there's a lot of confusion. And then the poor data scientist or the poor software engineer learning to be a data scientist is left carrying the can, feeling really awful because things went weird for the third time running. Why did that go wrong? Write a project specification. So I gave a talk at PiData Cambridge a few weeks ago. That's that title on the bottom right hand corner. That's based off of two training courses that I run and lots of other talks that I give where I've boiled down both this successfully delivering notion and software engineering notion for data scientist into a half hour talk. So that's up on my blog at the moment if you want to go and have a check on that. So in the blog is enosvold.com. And you need a good definition of done. And then that story from 10 years back, this was a slightly terrifying incident where I had a client, a very sensible client. They were an e-commerce company. They were doing price comparison shopping. And so the problem was we go and scrape websites, we get the same product from different websites, can we find out which website has the cheapest version of that product? Yeah, very sensible. If we get the same product specification down, we can just do a string comparison. They'd already solved that. Can we use machine learning to do a human-like reasoning when the product descriptions are different but clearly the same product? Yeah, same reasonable. So we agreed. I would work on a data set that represented the real data set. We'd come up with a prototype solution. It took three months. And then we would apply it to a larger data set. And the data set I was working on would be a representative of the real data set. We all agreed that verbally. We didn't write it down. That was silly. After three months, we said, look, the prototype. It works, right? This is kind of cool. This thing actually is doing the right job. It's doing a human-like comparison of these strings and coming up with a good agreement most of the time. They said, great, now you can look at the other data set. I said, well, what do you mean the other data set? We agreed the first data set would be representative of the true data set. They said, yeah, but we didn't want to scare you up front. So we gave you the easier one, first of all. And now look at this one. So we don't know how to do it ourselves. Oh, crikey. So I looked at this and I thought, no, as a human, I can't differentiate this. There's no way the machine is going to differentiate this.  And I thought at that point, as a consultant, I had an idea about how all of this worked. But it turns out I didn't because I wasn't writing things down. So get shared agreement. Write down what the problem is that you're solving, why it's valuable when they come up with you. Agreed milestones around your project. That's the first practical bit around getting a project that works, so a process for software engineers to think about delivering some value. So now we're going to do a short live demo, which I've written in a Jupyter notebook. And then, as I mentioned, given Sylvain's lovely keynote, I've added some bits to it. So here is my pretend example. We want to automate miles per gallon estimates from some engine statistics. The reason I'm using this is because there's a convenient standard data science toolset, standard data set called the auto MPG problem. So I'm using that. I've co-opted it to give you this setup. So we have a colleague. They're an engineer. They go through these data sets looking at characteristics of engines and then mile per gallon estimates or recordings. Their task is to find a good engine that matches some constraints that have been provided by a client. And they've got a big pile of books of all of these different engine configurations and mile per gallon statistics. But they don't always have mile per gallon statistics. Sometimes they're missing. So we want to fill them in with some machine learning. And then the goal would be, well, if you can fill in these missing mile per gallon estimates, then rather than the human having to calculate and run physical tests to figure out what the mile per gallon is, which is an expensive human process, maybe we can come up with some kind of prioritized ranking of these engines fit to your constraints and your mile per gallon requirements, and then you could have a prioritized process of looking through them. So it's a simple way to use mile per gallon estimates to kind of prioritize a workload to make it more efficient. It's a fictitious problem, but it fits to lots of real world problems. But the team is suspicious of machine learning. So how do we get them less suspicious? So here we've got a Jupyter notebook. First of all, can you see that at the back? We haven't got to read the text. We've only got to look at the diagrams that come past. So this is a Jupyter notebook. It could easily have been a Jupyter lab rendering the notebook. I've loaded in some of the standard tooling. There's NumPy, Pandas, Psychit Learner, some of the Psychit Learn metrics. Who's used the yellow brick visualizer before? Or Psychit Learner? Who's used Psychit Learn? Some of you have. Who's used the yellow brick visualizer? One of you. Right, good. So some of you will learn some new things in here. And I'm also using Outer. Who's used the Outer data visualizing tool? One. Right, not enough at all, because it's brilliant. So we load in this auto-MPG data set, and that data set looks like this. So the standard data set. In fact, can I make that slightly bigger? Is that slightly bigger? So we have some characteristics. Mile per gallon and cylinders, a number of cylinders in the engine, displacement, its size, horsepower, and a few other characteristics. So can we use those characteristics to learn our miles per gallon? One of the first questions we have to ask ourselves when using a new data set is what correlates with this thing that we want to predict? What data should I be investigating first to understand what I've got in my data set? And a correlation that I've got there at the bottom, that's the first sensible thing to do. You run a correlation, and it just is all right for miles per gallon. It correlates with weight, and then displacement, and then horsepower, and then a few other characteristics. Once we know what correlates, we've got an idea of what might help predict this thing. So we could use weight and displacement to predict our miles per gallon. So this gives us an in into building our predictive system. But first of all, before we build any predictive system, we want to draw some graphs. We want to investigate our data. So we know what correlates. So we might say, right, I want to draw a graph, a scatterplot, of miles per gallon versus weight. And so we do that. Here I've got out here. So out here, it's a bit like maplot lib, but it's not static. It uses JavaScript, and it's interactive. So we can do some cool things. We can zoom out and zoom in and pan around and all that jazz. And you get tooltips on this. So very quickly, in just those four lines above it, I can get a nice interactive data set and visualize it. That's really, really nice. Out here has recently become quite mature. It's been a couple of years old now. One of the nice things when you're looking at your data set and you need to be doing this as software engineers, you look at your points and you think, oh, it's kind of cool. So I've got this relationship between weight and miles per gallon as weight goes down to the car gets lighter, the mile per gallon, the efficiency increases. That kind of feels sensible. It's not a linear relationship. It's kind of a banana-shaped one, so slightly nonlinear. But it's still linear-ish. And so that's kind of cool. And so I look at this and I think, well, why is it that for this particular weight, I've got a band of engines that have a certain miles per gallon. But for the same weight, I've got a couple of outliers. And so I look up at those outliers. And because I've got interactivity and tooltips, I have a look at the car name and I see in brackets I've got diesel. And then I realize, oh, there's a subtlety in my data. I've got petrol engines and diesel engines. And that's not represented in any of the features that I've got in my data. That's just shown in the title. So when I sit down with my colleague, I can ask, what's the deal with diesel versus petrol? Is that an important thing that I should know about? And I say, run my mouse over this data. I can see that there's different years and then there's different territories. And I think, well, what if I color it by year? And I rerun that. And I think, oh, that's kind of interesting. So for early years, I get a lower mile per gallon for the same weight. And then for later years for the same weight, I get a darker year. So vehicles are becoming more efficient over the years for the same weight category. That's something interesting. I didn't know about that, but I can talk to my colleague about this. And as I run my mouse over this, I notice at the top I'm getting territories like Japan and USA and Europe. And when I come down the bottom, I'm just seeing USA most of the time. So what if I change the coloring instead and use territory? And then we see that exposed very quickly. We've got those red dots at the bottom there. So all the US cars are less efficient than the European and the Japanese ones. Well, that's pretty cool. I can talk to my colleague about this. So I talk to my colleague and they go, well, it's great. I've got this heuristic. When I estimate mile per gallon, I just take 1% of the car's weight with a negative relationship. So that as car's weight increases, my mile per gallon decreases, and I call it a 1%. So we can draw a straight line through our data. We can take our weight, multiply it by negative 0.01, so 1%. And this gives us a mile per gallon estimate. And then we draw it and we get that green line going through our data. So we've come up with a human estimate and we've drawn it. It seems pretty reasonable. But it doesn't fit terribly well, but we've got some relationship going on. And then our colleague says, well, that's great. You've got this machine learning stuff with lots of clever visualizers. I don't really care about that. I just want to know how well am I fitting right now? Well, I could make my own scoring mechanism or I could wrap up this estimator that we've come up with into the scikit-learn framework. And I do that with this my estimator here. Most people would never do this, but as engineers you might be interested in the idea of what goes on behind the scenes with scikit-learn. So I make an estimator. I have a fit and a predict method. I ignore the fit method. I'm not going to fit my data because I've come up with my human assumption, but I've come up with a hard-wired predictive method that just takes the negative 1% and applies that to the data coming in, the weight, and gives me a prediction coming out, which is that straight line. And then I use the scikit-learn. One of their standard scores, the R squared score, zero means there's no correlation in my output. One means perfect correlation, very predictive. And so I can use the standard scoring methods on my own estimator that I've written by hand. And I find out I get a score of 0.53. It's okay, it's fine. It's not super special. But my colleagues now say, well, that's kind of cool and I understand some linear algebra. How does ordinary least squares do on the same data set? I go, wow, we're using the same format as the scikit-learn ordinary least squared. So we can take, we'll jump, ignore that. We can take linear regression, train our data, and ask ordinary least square to make a prediction, use the same output, and we finally get a better score, score of 0.66. That's very nice. And we can print out the coefficients. And the coefficient that comes out is not 1%, but it's 0.8%. So a little bit smaller than the coefficient we'd estimated. And the offset is not 55 mile per gallon, but 46 mile per gallon. So pretty close to what the human came up with, but it turns out this one scores better. So we've used ordinary least square to fit better, but we're showing a progression from human estimates through to using a machine-learn solution. And then maybe I say, well, look, Collie, now you're on board with this. We're using some Hexatike-Learn. We've got a model going. We've improved upon the human model. We're using standard scoring. What if we used a more advanced model, a random forest, a non-linear, more sophisticated fitter? And we'd give it some more data. So we give it more columns of data. And then this will understand the non-linearities in our data set. And it should give us a better result. And indeed, we plot it out and we get a score of 0.83. And we see those green dots, they're interspersed with the blue dots. They're not running at a straight line anymore now. So this is looking pretty sensible. We've come up with an estimate that's doing quite a nice job. And our colleague says, well, this is great. I kind of understand. I'm kind of with you. I think this is working. How do I use it? And he goes, well, actually, how about that thing that Sylvain, our colleague told us about with the iPython widgets? What if I gave you a nice little widget interface where we could put up, say, weight as an interactive item? And then every time we change our weight and we keep the other coefficients fixed, so we keep the other inputs fixed, we can then predict our miles per gallon. And then I get a nice interactive slider here. So as weight increases, we see our predicted mile per gallon decreasing. And then as weight is decreasing, we see our predicted mile per gallon increasing. And we can imagine building up this user interface and making it more complicated, having more features, having graphical outputs along there. But I think just having an interactive slider which we could serve up, that feels pretty powerful. But then our colleague says, yeah, but Ian, right, if I take this, I'm going to break it. I'm going to delete files in your machine by accident because this is being served up inside the notebook. And he'd say, no, hang on a minute. What about voila that our colleague showed us? And so you run voila. And he'd say, look, I'm serving up this not client, sorry, server side rendered version of our notebook. Just exactly the same notebook. I just ran voila, my notebook. It pops up all the output. And then I get that interactive slider down at the bottom. And as I increase the weight, our predicted mile per gallon decreases. We've got an interactive user interface being developed here. That's kind of nice. And if it rendered in time, I may, oh, there we go. That's one, this one, did it render? I failed. Earlier, I had a Binder public interface running as well. So I'm going to run over for a couple of minutes, then take questions. So earlier, I had a Binder version of this running off of my public repo. And then this was mounted on the web. So you could have accessed this from the web on your laptop and interacted with the data along with me. So I was really impressed with that, the Binder component and voila, in this morning's talk. As far as I can tell, it just works. I just did it in the last couple of hours. You should try it too. It's very cool. So back to finish off the talk. There's a couple of resources. I mentioned that I do some training. I've got a course coming up in a couple of weeks. That's not interesting to you. It's aimed at data scientists. But you might know some colleagues who will be interested in this in the future. The two books on the right, on the other hand, will be interesting to you. Python for data, sorry, the Python Data Science Handbook by Jake van der Plaas. That's a really good software engineering focused introduction to the world of data science, not just machine learning, but data science and data analysis in general. So that's a really good book to get. And for your managers, data science for business. It's a formula heavy book. But the formulas are really easily explained. So it's really easy to follow it along, understand what's going on. There's lots of really nice diagrams, really intuitive explanations. So I enjoy reading that book. It just makes everything really obvious. So I recommend both of those books for different components of your team. So I'm going to wrap up here in just a moment. I've been speaking at events for a long time. One thing I realize is many attendees don't realize how much this is volunteer run, how much the organizers, the room co-chairs, the speakers, everyone around are volunteers. So please go and thank your volunteers. Go and find an organizer or a speaker or a room co-chair and thank them. It makes the world a better place. And if you would like a free copy of my book, there'll be 20 of these or so somewhere out here at 3.30. And I'll be signing them. They're sent over courtesy of O'Reilly and their outreach program. So in summary, we want to automate parts of a high-value problem. You haven't got to automate all of it upfront, automate the high-value parts, and then iterate and keep on building out your solution. Keep building that value incrementally and communicate your results early and often and come up with ways of communicating that people can interact with the system. So using, say, Voila and those widgets to make an interactive front-end is much nicer than just having a notebook, but the notebook beats having, say, a CSV file as an output. So come up with ways of getting it into the hands of your users to actually get value out of it. I've got a couple of mailing lists on my website, one of them for the training announces and one of them for jobs that I've got that I know about that are coming around and a bunch of talks and books and other things that I recommend. If you're interested, you can get that off of my blog. And there's lots of other talks that I've given over the years linked to my blog as well that we'll fill in around this. I've been talking about this area for about five years now. So there's lots of background stuff there if that'll be interesting to you. And I'll finish there. Thank you very much. We'll have time for one, maybe two questions. So, any volunteers? Thanks a lot. Thank you for that. So good without saying it. It was an extremely compelling and interesting talk. I just wanted to ask a question about automated testing. So I've got my hand up as a software engineer. I'm dabbled with data science and have a junior data scientist who may be in this room working at a certain place. I do, I like automated tests and the nature of data science. Obviously, you're dealing with large data sets. You might not want to do an end-to-end test with your full data set. And if you knew what the outcome was, you probably would have even run it. So I just wonder if you could talk at all about automated testing in the context of data science. So, and this is something I'm covering on my course in two weeks time because it turns out testing is a complicated subject and it's not often covered in any of the, or it's not covered in any of the data science books. And yet we need testing. Most data scientists don't write tests. That means we can't trust the output of the data scientists work and that's awful. That's not the right thing. So my work style and what I teach and what I talk about is to develop your code in a notebook, find the code that you want to trust and then extract that into modules. And then you've just got a module. Then you can put your unit tests around that module. So that lets you test your code. Then you've got your data as well and there you'll be using your data fixed as a fixture, pushing it into your pipeline, which could be a notebook. It could be an exported notebook as a Python file, importing those modules that you've made that already tested, pushing your data through and then checking the quality of the output. So if I know certain rows of input should give a certain type of output, I can then set up a unit test that says with my big black box thing that I've done, my data transforms, the visualization, the model, the predictive output. When I get this certain row from my fixed set of inputs, check that it gives me the right kind of mull per gallon output within some tiny range, some wiggle room perhaps. And check that none of the mull per gallons go negative or go to 1000s or whatever a wrong result would be. Find some limits and test those. And that mixture of sanity checks along with your regular code unit tests, I found to be, it's a pretty strong way to go. And you can do the same also at deployment time, having checks running on your running system, which is ingesting live data, which could be changing in shape. The mean could be changing in that dataset or the variance or whatever. You could put some checks around that at runtime so that in the background it's always checking that things feel like they're in the right bounds and the system is behaving appropriately. And if not, it flags. And those strategies do a reasonable job. That's all that we have. So please have a nice round of applause. Thank you very much.


This research compared the performance of three different frequency ranges when encoding 16 targets using steady-state visual evoked potentials, SPEPs. The results showed that the narrowest frequency range between 31 and 34.5 Hertz had the best information transfer rate, ITR, which was 153.79 and hashtag X000B1, 6.39 bits, slash min. This suggests that the narrowest frequency range is optimal for building an online BCI system. This article was authored by Xiaojing Chen, Bingquan Liu, Yijuan Wang, and others.


What is going on guys welcome back to channel Critical Overlord here some time about Jeepers Creepers reborn mostly just the Jeepers Creepers franchise as a whole in general right now and Talk about the idea of us exploring another prequel movie now We know with reborn it's going to be following the couple of chasing Lane and they'll be going to the whorehound event Lane is gonna be taken chase will go looking for his girlfriend get caught up with some other characters along the way And they'll most likely be having a lot of the mayhem that results in a few characters deaths Occurring in this new house of pain that it looks like we have seen shown in the teaser and seen countless behind the scenes Set images for It just makes it look like the whorehound event won't be the primary location It'll just be part of the early stages of the movie were a big majority of the Creepers New onslaught will occur in this new house of pain so whether or not a Prequel movie happens again after this one that of course would be up to the writer at the time whoever's involved in it If five even gets made I Think a lot of the fandom myself included would agree that if we were to get another prequel You would want to see a movie That is a prequel to everything that we've gotten that would include three and You're not gonna set it in between anything or anything like that You're going to set it all the way before Everything we've gotten that would include part four and it would be the narrative about Kenny and Darla I think that's what the fandom would prefer to see I think that that is what many of us would have wanted to see in Jeepers creepers 3 I remember when the shout factory collectors edition of Jeepers creepers 1 and 2 came out and there is that special feature where salva himself was already talking about how he had a script for 3 and it was taking place in between 1 and 2 and The fact that that movie actually still got made To this day, you know because it could have been great It could have been something great But it's just some of the stuff that was involved with the third movie was one of the many reasons Why it ended up being bad like the new the new features that were associated with the creeper how they were portrayed on screen The way they were used on the truck and came off cartoony a lot of the other things involved with the writing Decisions other things and how they tried to connect it all together by the end of the film A lot of things ended up making that movie a train wreck Whereas many people coming out of it said we already gave us a Seed and planted many moments of how you could explore a prequel film when you had Kenny and Darla talked about multiple times in the original movie For all we know you could have actually explored that again before you even actually gave us the film We all are still waiting for which is that the fourth movie is gonna be giving us that you could have done that In between us waiting for the creeper to get off the target barn Jebus creepers three could have been turned into a Kenny and Darla prequel And then Jebus creepers four could have been the creeper coming off tag of the barn, which it is still going to be as far as we know Uh, but the three that we got just was not what everyone expected it to be It was not something that many people were on board with after coming out of watching it Many people gave it a chance, but after watching it said I can live without that one So if they were to do it again, I think you would need to explore something that is set Millions not millions hundreds centuries prior to The events of the first four movies where you could have the creeper stalking a civilization We could see said civilization navigate their surroundings knowing that there's this entity known as the creeper out there Uh, and we can just see how the creeper interacted with people with earlier civilizations of humanity I think that's something many people will want to see even though there's deleted scenes That aren't canon to the film minks these comments and knowing what we know about this monster already tell us that this thing has been around For many centuries. It's Definitely outlived certain civilizations. Uh, it's been through different periods We know that there was also a opening in the cathedral script where it was going to have a bar fight Where we were going to see how he came to have his duster coats and his cowboy hat that he has I believe Um, mostly I think it was just the jacket might have included that I haven't read that script in a while but maybe we could have gotten a movie a jeepers creepers movie set in the Wild west completely in the wild west um And just explore that type of period with the creeper involved riding in on a horse he won't have a truck he'll just be riding in on a horse and Different ways you could explore this character as it relates to all the civilizations that it's outlived and terrorized over the over the years I know I've done a video talking about the fall of the roman empire and how you could have the creeper Set in that era with gladiators all these other things that we know Or have heard about throughout the years of human civilization and history Movies we know exist with gladiators and then give us a movie like that then throw the creeper into the mix I think a lot of people will be on board with something like that If the franchise in and of itself wasn't tainted so bad There's just a lot of things they could explore if they were to do another prequel Ideally the one that I would want if jeepers creepers 5 were to be announced as a prequel Would be the kenyan darla story just so that way you could have something to watch before you jump into the original movie and then Many people would actually I could see a lot of the fandom if you actually were to get a kenyan darla Prequel they'll watch that one then they'll watch trish and dairy story and then they'll watch the bus one with the taggards involved and see a cohesive trilogy exist right there Whereas you can probably exclude three maybe soon to be four And not worry about all of that trash that exists within those two movies now again four could still be something great Um, well, I take that back. It could be something decent something decent at best Of definitely better than three, but the whole idea of another prequel. I think would be something They could explore if they wanted to it have to be done Right and I think the way it should be handled is just having us Explore these different civilizations that the creeper has been a part of and outlived and terrorized and maybe digging into a narrative That's centered from the creepers pov Where the creeper is the one driving a lot of the narrative where we have a lot of pov shots from its perspective It won't have lines of dialogue, of course So you'll have to have human characters involved, of course Where they're talking amongst themselves and we focus on them a little bit But how would you feel about a movie where the creeper is Where the creeper is mostly the focus and you have the narrative unfold from its perspective You only jump to the characters the human characters that provide dialogue when it's time for the creeper to attack You get to spend a lot more time with this monster see all of the ways it It interacts with people the way it stalks people the way it maybe looks Looks from afar because we know it was doing that in two So I think a movie showing that would be fine to see it stalk its prey before actually attacking Another prequel movie probably won't happen But if it does I know ideally it would want to be the kenyan darling narrative But they could explore different civilizations that this monster has been a part of Let me know what y'all think about that down in comment section below Would you prefer to see another prequel film if you haven't already make sure you subscribe turn on Post occasion and miss video in the description at links my social media accounts on facebook, twitter and instagram You can message me there of course to limit as any movies news or reviews you'd like me to cover in the future And with all that in my eyes, I will see you in the next video


Hello everyone, welcome back to another AI video. In this one we're in just released Adobe Illustrator 2024 version 28. It's just updated and in particular you'll see it says 28.0. It has the brand new text to vector graphic or text to vector functionality built into it and it is fantastic. It's kind of like generative fill on Photoshop. You type in what you want and it will create the vector for you and it will give you three examples. Anyways, let's just here's a couple examples here. Here's like a cyberpunk scene that I created. It's not nothing special, but I asked for it to be purplish and pinkish and then I also have a matching French bulldog on a throne here and this is more of a subject and then this one here is a French bulldog icon. So you can do four different things, subject, scene, icon and patterns, if that's your thing. So let me show you how to do it. It's super simple. So again, we're in Adobe Illustrator 2024, the latest version and now I've just got a couple of artboards open up here. So it just open up anything that you want to work on any artboard and then go to properties. Under properties, you're going to see this here. It says text to vector graphic beta and then under type, you get four types. You get subject, scene, icon and pattern. Also, you can now under that you're going to see here that you can match active artboard style. So if you have an artboard and you create an icon, for example, the icon will try to match the artboard. You can leave that on or off and then here's where we do the magic prompt. So in this case, I'm going to go cyberpunk French bulldog with awesome vibrant colors. And again, you guys can type in anything you want. I just want to show you now because I have this set as subject and not a scene because that wouldn't make a whole lot of sense. It's going to go ahead and cook up some examples of some cool subjects and you can already see them here being built. So here is the first one. And of course, it throws it right in the middle. I'll click out to show you this pretty cool. And if I move it over here, all right, I like it. Let's take a look at the second one here. Let's go ahead and click on the second one. I've added the second one in this one has some personality. I like it. And then let's look at this third one. This one's mean mugging it looks like, but I'm going to move this one over here and boom, those are the three. These are the three variations that you get. If you don't like it, you can just go ahead and keep on clicking on it. Also, while I have you here, I want to show you a few things you can style pick so you can extract the style from a reference image. So if you have an image that you want to use, you can go ahead and use that. I'm not going to do that in this case, but do keep in mind that it is available. Also, you can do a sample prompt if you're not sure what to type in and you need some ideas sample prompt is interesting. And then the biggest one here is this one here, which is detail. So basically, how much latitude do you give the system? So basically, do you want it to have very little details? So if you type something in, it will just give you a, you know, a very, you know, mixed bag, so to speak, or do you want to be very tight to what you typed into the prompt? And again, you can adjust this as you see fit. It defaults to two and two seems to work quite nicely. Now let's go ahead and delete all of these. I'm going to show you the other functionality really quickly here while I got you. If I wanted to, for example, switch from subject to a scene because maybe I want a cool background, let's go with mountains in a scary haunted movie, something like that. It's kind of silly one, but whatever. I mean, you guys can go nuts, but I'm basically kind of like a scary movie with some mountains in it. All right, whatever. So this is going to go ahead and give us three examples, but these are going to be scenes not subject. So again, let's see what we get here. I can already tell I kind of like this third one. So I'm going to click on that and presto. Here we go. We've got that now. Look at this. Keep in mind that these are vector graphics. So if I hold down shift and I just pull on one of the handles on the edges, this because they're vector graphics, they scale infinitely. So I mean, I could do something like this and I could put the mountain right here and then boom, there we go. I'll just click on okay, something like that. And presto. There we go. I've gone ahead and created this cool scene. And that's one of the other things you can do. The last one I'll show you here that I like to use is an icon. And this is a simple one. So French bulldog icon, something like that. Nice and simple. I'll create this. And then it should, in theory, try to match the active artboard. So it may try to match this here. Again, this is all pretty new. So if it doesn't, so be it.  Let's go with this one. Let's see what this looks like. Okay, yeah, sure, we'll put that guy right at the top of the mountain. How good is that? There you go. So of course, because it's vector, you can go in and you can recolor it and you can adjust it as you see fit. But there you go. These are all vector graphics. And of course, if you want to verify that, just go into your layers panel, drop them down here and look at this, all vectors, all the way down. That's all there is to it. It's awesome. Thanks for watching.