Hello, everybody, and welcome to yet another OpenShift Commons briefing. Today we're going to tell you how to automate and scale your data pipelines in the cloud-native way. Guillaume Moutier from Red Hat will be reintroducing himself and telling you a little bit about himself, and then giving us a bit of a deep dive in our data pipelines initiative. So Guillaume, please take it away, and there will be live Q&A at the end of this. I will get the slides from him, and we will post it all on blog.openshift.com and on YouTube, as usual. Take it away, Guillaume. Okay, thank you, Diane. Hi, everyone. I'm Guillaume Moutier, greetings from Canada. I'm a technical evangelist at Red Hat in the storage business unit, and I'm working mostly on data, not storage itself, but data, the way you consume it, the way you move it around, and especially in the AIML field. But today we're going to look at a standard data pipeline, and the way you can automate it and scale it automatically the cloud-native way. So let's get started. First, the cloud-native, to set the stage, I want you to go back a little bit on what the characteristics can be for a cloud-native platform. Here, I'm listing the things that are most important for me, but what you must never forget is why you are doing things. Here is the business outcomes. What are we trying to achieve when we are implementing those kind of architectures? For me, most important things are speed, efficiency, and foremost, adaptability. We know now that technology is moving fast, real fast, and we have to adapt to our businesses or organizations to be able to handle these kind of changes. And so adaptability is, and was my main concern, all throughout my career. And now we have the tools and we have the technology to be able to achieve these business goals. So let's take a look at what I would call a legacy data pipeline architecture. And I call it legacy, but I know for sure that for most organizations, it's still the start of the way to do things. We look at architectures that are very tightly coupled and not easily scalable. For example, if I take a very basic application where a user will save a five to some storage that can be processed by an application, well, the way it works for most applications is that there is some storage mounted on a computer, can be a shared folder or something like that. And then the file is sent to the storage. Again, it has to be mounted against the server, CIFS, ISCSI, but still some kind of hard connection in between the storage and the application server. And then it's consumed by an application, let's say some Java application here. Problem is with this architecture, well, first I have to have things very close from one another because of this mounting problem. I cannot mount the CIFS over thousands of kilometers, doesn't work well. Also the scalability problem. When I am using this type of connection, that means that if I want to put up another application server, for example, because I want to scale my application capabilities, well, it has to have exactly the same configuration as my first server, exactly the same storage connection, exactly the same mount point and behavior. So that's okay if you have one or two server, but if you have tens or hundreds of them, that's a burden that you have to take care of. Now let's look at a more clownative way to do this kind of things. Well, we can think of an application, again, where a user will just send a file, but this time to an object storage, and here it's a fully disconnected mode. You know, using object storage, consuming object storage, is only HTTP connection, so it's only a put or a get, and then that's it, I'm finished. I have no remaining connection in between my user application and the object storage. Same thing for your data processing function. They can consume this storage directly, and as they need it, so that means they can be wherever you want, and they can scale, it will be much easier to do. And now we have intelligent storage, I would say. In the latest releases of SAF, we have now backup notifications. So that means that whenever something is happening in the object storage, you can send a notification, let's say, to a Kafka bus. That will itself trigger some data processing function. Here I like to put Kafka in the middle of those kind of architectures, because it can act in two different ways. First, as a buffer. Let's say my data processing function is not ready or not ready yet. Well, the notifications will keep coming in inside the Kafka bus, and when it's ready, then it will be consumed, the topic, the notifications will be consumed, and then the function can realize its process. But also Kafka can act as some hub for all those notifications. So we can imagine that we have different processing functions, maybe in different places, different data centers, realizing different operations, but everyone feeling on the same topic. So we'll try to do it for real. We'll try to build an application that will work like this. So here for this demo, I took the example of ACHPments for you people who are not in the United States. ACH can be seen as some electronic check, electronic payments. So it can be used by a customer paying a service provider, an employer depositing money on your checking account for payrolls, all those kind of things happening electronically. For my demo here, I will try to implement this very basic pipeline where someone buys something from merchants and there is an electronic payment happening. The way it works is that the transaction has to be sent to the bank of the merchant and this bank will produce what is called an ACH file. It's a standard file, we'll come to it in a minute. A standard file that will be sent to the Federal Reserve. Here it will be processed and make available to the receiving bank. The receiving bank will be the one of the customer. So it will be the one to process the transaction and debit the account of the customer. So that's the basic process of ACH and as a reference here is the ACH file itself. And the way it works, you know very old fashioned way of describing transaction with the first line giving information about the bank, the bank itself and some basic information about the company, second line of more details about the company and then you have all those transaction fields with the different customers here, the amount of money that they have spent and to which bank, to which receiving bank this transaction should be sent. This is how I have implemented it inside OpenShift. So I have here some kind of generator, we'll come to it, that generates fake transactions and send those files inside an object storage bucket. Then this one will trigger a notification that will be sent to KafkaBus. And here I will be using candidate eventing and candidate serving. That's the way in Kubernetes and in OpenShift to create on-demand parts, on-demand functions. So I have a service that will be listening for Kafka events and then spinning up a deployment of the container that will process the file, that will process the transaction. So here what it will do is create an ACH file for the transactions and send them to the bank, so the bank of the merchant. So here I will have a few buckets. So I do my demo with seven different banks or seven different buckets to which the different files will be sent, depending on the merchant sending the file. At the origin bank, those files will be processed. Basically what it will do is look at all the transactions and recreate new ACH files this time sending it to the destination bank, to the receiving bank. All those files will be created and versed into different buckets. So this time buckets belonging to the receiving banks where they will be processed. So here the standard process will be to look at the transaction and debit the account of the customer. What we will do in this demo is that we'll only look on the amount processed and we will just sum them up in some big buckets just to see how many transactions were processed and how much amount of money was processed all throughout this pipeline. So to implement this, a few things that I need, some Kafka topics to be able to send my notifications. So here you can see at the bottom they have the American top load topic. And the all DFI topic where I will send a file. Then I have some buckets that I have created in my storage. So here are all the buckets that I have. And don't worry, you will have access to the code and everything to be able to reproduce the demo. So I won't go into too many details on this right now. And then we will program the bucket notifications themselves. How it's done in SAF, in RHCS, in Red Hat SAF storage. What you can do, what you do is to create a topic that will point to your Kafka endpoint. Okay, so here I will create a topic with the name RDFI and I will point it to my Kafka cluster. Okay, and then for each bucket, I will use this reference to the topic I just created. And I will here, it's a simple put request to the name of your bucket. Here it's from an old demo. So here it should be a different name and the notification verb and the configuration of the topic that I want to use in Kafka.  The way it works is that it will trigger a container that will generate our transactions and it will run 60 times with a parallelism of five. So that means that I will be able to create five files at a time inside my open shift cluster. So let's go, let's do this. So here, what we can see here, I am in my project. I can see that I have three pods which are the pods, the candidate pods, the serverless, open shift serverless pods that are listening to events. I have also in my open shift serverless, I have three different processes, three different services which will split the H5s or process them depending and they're ready. But you see those processes are ready. The services are ready but there are no pods running. So now we are scaled to zero. So let's create these transactions. Here I will use the exact same file I just showed you. And now it's being put into motion. So here we can see that we have five containers creating based on the transaction image, transaction container that I have designed and they will begin to create new transactions. And as new transaction files are created, well, it triggers containers. It triggers the ODFI split. That means looking at the H5s and splitting them and putting them inside the RadBucket. It also triggers RDFI split. That's what's happening when it looks inside the HCH file and splitting them together to send it to the receiving banks. And then the RDFI process. So here I'm processing the other transactions themselves. It will be better with a live view like this. Here is a graph and a dashboard where I have my pipeline. We can see that we have already generated 15 different transaction files fixed in now. So far, 16 have been processed and dispatched to the different bank of origin. And so far we have treated eight. We have processed eight of them. Those files are split for the different receiving banks and they are sent here to the receiving banks buckets where they are processed. And so far we have processed 75 of them. Of course, we have many more files in this process because we take each originating file and split them, split each transaction towards its own receiving bank. We can see as the process is going on that the CPU usage is increasing. Of course, we are spending more parts as we need them. We have also the RAM usage going on. And I have some lags here on the deployments but it should keep up in a few seconds. And we can see here the value of the transactions that have been processed so far. So we can see it's going up. We are now at about $9 million. What I generate here for transactions is a random number of transactions between 300 and 500 of them for each file. And the amount itself is between $1 and $2,000. Okay. So that's the kind of transactions I'm generating. And here we can see the different deployments now that we have. We are now up to 15 parts. We can see that we have five deployments of the create transaction pod. That's the maximum parallelism that I authorized for this. We have, of course, my listeners for the Kafka events but the treatment itself, the processing itself is how the RFI split is what's happening here at this point. So here it doesn't consume much resources because it's only looking at the files and depending on the American banks, sending it to the different buckets here. So not many resources involved. So there's only one deployment of this process. But here if I look at our RFI split here, that's what's happening in this box. So that means retrieving the file, splitting it into the different transactions and recreating new files and then sending them to the receiving bank buckets. So it consumes some more resources. So here that's why the serverless functions has automatically be scaled to two deployments because that's what it needs to be able to handle the traffic coming in. Same for the RFI process. It looks at the files and process it adding to the amount of money that all those transactions represent and then it needs also two of those parts to do the processing. Here, what's happening? We can see that we have reached the maximum number of files that we wanted to generate. So 60. So our create transaction parts have scaled down to zero. Okay. We, of course, that's what we wanted to do. And then here we have reached also the number of 60 for this, the first step of processing. So this RFI split pod should come then to zero in a few seconds. We can see that it's, we already are consuming a little bit less memory for those kind of things. So here that's a neat way to demonstrate with only using bucket notifications and serverless functions. You can fully automate your data pipelines. Doesn't require some kind of application that would orchestrate everything and will take care of everything. Here it's only a few files, a few configuration files that you put into motion. That allows you to create very simply this kind of pipelines. So speaking of files, I will go back here. Speaking of files, you will have all the code and all the different configuration files and containers, images and things like this in this repo. I will also put it in a few days a full walkthrough to be able to reproduce this kind of demo. And of course, feel free to reach out for some more information or if you have questions or problems implementing this kind of things. It will be a pleasure to reply to these questions. And now I think we still have time for a few questions. Absolutely. Let's see if we have... I don't see any questions in the chat, but I think we're all kind of totally loving the demo that you gave Guillaume. So I'll open it up and see if people have any questions. I'm not seeing anything, which means you did a really thorough presentation. So thank you very much. The repo that you point out here on the demo page, that has everything in it to reproduce this demo? Yes, there is everything. There is the container code to be able to create the parts that will process or create the transactions. There is the Kafka topic creations. There is... Well, there is everything to be able to go from scratch that is starting on the brand new OpenShift installation and install everything that you need. Awesome. So we look forward to other people taking this for a test run and trying and demoing it. And I really appreciate you taking the time today, Guillaume. And look forward to having you back for new updates on this topic. So thanks again. And everybody would like to re-watch this. It will be uploaded on the YouTube channel later today. And I'll steal the slides from Guillaume shortly and also link them up there as well and put a blog post with some other resources up on blog.OpenShift.com. So look for that in the next coming days. And we will continue to provide you with entertaining and educational briefings over the coming weeks to take place of some of the conferences that have been canceled. So look for that on the events page at commons.OpenShift.org. So take care, everybody. And thank you very much.


can i easily transition into med school like is there a bridge program where i don't have to go to med school for four years what's up you guys it's adana welcome back to my channel thank you guys so much for joining me for this video today i'm gonna be talking about can i go to med school after pa school um and i'm gonna be giving you my my take on this as adana the pa student okay all right um if you haven't already done so go ahead and do me a favor right now hit that little bell on the top of your screen that notification bell and subscribe to my channel so that you can know exactly when i post a new video and follow me on instagram at adana the pa all right let's get into this question so i saw this on one of my comments i was really excited to answer it um because i thought it was very you know it was very astute of a question i guess you know i'm sure some people are thinking well i want to be a doctor but um you know i don't know if i want to spend all that time in medical school and then some are just like oh yeah i kind of wanted to be a doctor but i think pa school is easier because it's only two years so that's kind of what this question is all encompassing and that's kind of what i'm gonna try to unpackage for you guys um and and answer it so yes you can go to medical school after going to pa school um because you can go to medical school after going to any school really and truly right you can become a physician after being a pharmacist or a nurse or you know a chef um you know depending on the courses that you've taken do you have a bachelor's degree um have you taken your m cat have you done well on your m cat um do you have shadowing and volunteering experience all of those things um prepare you for applying to medical school and and going through that whole m cas system right so short answer yes absolutely now i don't know if that's necessarily a question that you want to answer or if the question is more so can i easily transition into medical school like is there a bridge program where i don't have to go to med school for four years and i can just go to med school for two years because they've done two years of pa school and so for for at the moment there is not a two-year medical school pa to md bridge program i've read that there are some schools that are trying to propose that there is one school that i know of for sure that has a pa to do um which is a doctorate of osteopathic medicine um which you are a doctor you just uh kind of do a different teaching model but that school uh has a bridge program so a pa to do do program where you will go to medical school or do school osteopathic school for three years instead of the four years of medical school so you take off one year of your life um in medical school but you obviously will still have to go through your three years of medical school and then go through residency which is anywhere from three plus years two plus years depending on the specialty that you're going into so you're still going to be a student for at least let's say five to six more years at minimum but you're gonna have that d.o. or that uh md depending on the school after your name uh so that is an option for you that's what you want um for me that was not necessarily something that i wanted to do i didn't want to go into medical school for all the reasons i've listed before family time all of that right but if you for you if you feel that pa is just not enough you want to i guess have more autonomy you want to just do more by opening a practice and all these other different things that pa's can't do solely on their own at the moment then that might be a better option for you if you're willing to kind of like take the time to do so because it's not going to be a quick process and usually bridge programs are supposed to be quick but to me you've already spent two years in pa school and then now you're going to go to med school for four more years well three more years so it's really like you should have just gone to med school for the four years instead of trying to go to school for five years and that's just my take on it you know because i'm all about the time that it's taking me to get through something because i want to be out practicing seeing patients and being like efficient in what i do i do know that nyu has an an accelerated three year program as well so if you want to do that it's not necessarily a bridge program like the lake airy college of osteopathic medicine school that's the school that has the pa to deal bridge program but it's three years which is just the same amount as lake airy's medical school and then i just recently heard not too long ago in the news that nyu's also offering free tuition to anyone that gets into their medical school so that might be a good option for some of you if that's what you really want to do you know me i am all like pro pa let's push out more pa's in the world but the pa profession may not be for everyone so for those of you who are teeter-tottering between pa or md or can i go after get becoming a pa get some experience whatever the case maybe these are just some options available to you and that is what this channel is all about and that's what i want to do i want to provide you with as much information as possible while still promoting the pa profession go pa that answers your question or you know the question that maybe you were trying to ask i just want to make sure that you continue to do your research go out there google let google like you know accelerated med programs pa to md bridge programs pa to md or do um schools to see what is um out there for you because there are options also for online medical schools which is pretty odd to me that a medical school would be online but there are options for that so um you just have to do your research and kind of figure out what works best for you and um in your life so i hope this was helpful if you have any more questions leave that in the comment section below again that school is lake eerie college of osteopathic medicine that is the pa to do bridge program and then nyu's um accelerated three-year program if you are thinking about you know going from a pa to an md if you already have your pa you're certified as a pa and you want to kind of bridge into an md program those are some options available to you i know there are some other online programs so just kind of look look into them and see what works for you thank you guys so much for watching hope you guys like this video go ahead and give me some thumbs up if you did and follow me on instagram at adana the pa i will talk to you guys next time bye


Live from San Diego, California, it's theCUBE. Covering KubeCon and CloudNativeCon, brought to you by Red Hat, the CloudNative Computing Foundation and its ecosystem partners. Welcome back, this is theCUBE's fourth year of coverage at KubeCon, CloudNativeCon. We're here in San Diego, it's 2019. I'm Stu Miniman, my host for this afternoon is Justin Warren, and happy to welcome two guests from the newly minted Platinum member of the CNCF, NetApp, sitting to my right is Matt Baldwin, who is the director of CloudNative and Kubernetes Engineering, and sitting to his right is Rob Esker, who does product and strategy for Kubernetes, and is also NetApp's board member on the CNCF. Thank you both for joining us. Thank you, I'm sorry I'm on this. All right, so, Matt, maybe start with you. NetApp, companies that know, I've got plenty of history with NetApp there. What I've been hearing from NetApp for the last few years is, the core of NetApp has always been software, and it is a multi-cloud world. I've been hearing this message since before, kind of the CloudNative and Kubernetes piece was going. Of course, there's been some acquisitions, and NetApp continuing to go through its transformations, if you will. So, help us understand NetApp's positioning in this ecosystem. In Kubernetes? Yes. So, what we're doing is we're building a product that allows you to manage CloudNative workloads on top of Kubernetes. So, we've solved the infrastructure problem, and that's kind of the old problem. We're bored to death talking about that problem, but what we try to do is we try to provide a single pane of glass to manage on-premise workloads and off-premise workloads, and so that's what we're trying to do. We're trying to say, it's now more about the apt taxonomy in Kubernetes, and then what type of tooling do you build to manage that application in Kubernetes, and so that's what we're building right now, and that's where we're headed with the hybrid multicloud. There's a piece of it, though, that does draw from the historical strengths of NetApp, of course. So, we're building, we are essentially already in market a capability that allows you to deploy Kubernetes in an agnostic way using pure, open, unmodified Kubernetes on all of the major public clouds, but also on-prem. But over time, and some of this is already evident, you'll see it married to the storage and data management capabilities that we draw from the historical NetApp, and that we're starting to deploy into those public clouds. With the idea that you should be able to take a project, so a project being a namespace, namespace having an application in it, so you have multiple deployments, that I should be able to protect that namespace or that project, I should be able to move that, and the data goes with it. So that we're very data-aware, and that's what we're trying to do with our software is make it very data-aware and have that align with apps inside of Kubernetes. So Rob, maybe step back for a second. One of the things we've heard a few times at this show before, and was talking about the keynote this morning, is it is project over company when it comes to the CNCF. Project. Project over company. So it's about the ecosystem. The CNCF tries not to be opinionated, so it's okay for multiple projects to fit in a space. NetApp moving up to a platinum sponsor level, participant here, NetApp's got lots of histories in participating and driving standards, helping move where the industry is going. Where does NetApp see its position in the participating in the foundation and participating in this ecosystem? Yeah, so great question, and actually I love it. It's one of my favorite topics. So I think the way we look at it is, oftentimes projects, to the extent they become ubiquitous, define a standard, a de facto standard. So not necessarily ratified by some standards body, and so we're very interested in making sure that in the scenario where you want to employ the standard, from a technology integration perspective, our capabilities can operate as an implementation behind the standard. So you get the distinguishing qualities of our capabilities, our products and our services, vis-a-vis, or in the context of the standard, but we're not trying to take you down a walled, garden path in a proprietary journey, if you will. We would rather actually compel you to work with us on the basis of the value, not necessarily operating a proprietary set of interfaces. So Kubernetes, you know, broadly perceived as a de facto standard at this point. There's still some work to be done on running out the edges. A lot of underway this week. It's definitely the case that there's an appeal to making this more offerable by pardon the expression, mere mortals. And we think we can offer some help in that respect as well. Yeah, for us it's usability, right? I mean, that's the reason I started Stack One Cloud, was that there was a usability problem with Kubernetes. I had a usability problem with Kubernetes. So that's what we're trying to, that's how I'm looking at the landscape. And I look at kind of all the projects inside of the CNCF, and I look at my role is, how do we tie these together? How do we make these so they're very, very usable to the users? How we're engaging with the community is to try to align this basically pure upstream projects and create a usability layer on top of that. But we're not going to, we don't want to ever say we're going to fork any of these projects, but we're going to contribute back into these projects. So that's one concern that I have heard from some customers, we were speaking with some of them yesterday. One of the concerns I had was that when you add that manageability onto the base Kubernetes layer, that often various vendors become rather opinionated about which way we think this is a good way to do that. And when you're trying to maintain that compatibility across the ecosystem, so some customers say, well, I actually don't want to have to be too closely welded to any one vendor, because part of the benefit of Kubernetes is I can move my workloads around. So how do you navigate what is the right level of opinion to have and which part should actually just be part of a common standard? It should be along the lines of best practices is how we do it. So like let's take a network policy, for example, like applying a sane default network policy to every namespace, defying a sane default pod security policy. You know, building a cluster in a best practices fashion with security turned on, hardened and done, where you would have done this already as a user. So we're not looking at it in any way there. So that's, we're not trying, I'm not trying to curate any type of opinion in the product. What I'm trying to do is harmonize your experience across all of this ecosystem. So that you don't ever have to think about, I'm now, I'm building a cluster on top of Amazon. So I got to worry about, how do I manage this on Amazon? I don't want you to have to think about those providers anymore, right? And then on top of those, on top of that infrastructure, I want to have a way that you're thinking about managing the applications on those environments in the exact same way. So I'm scaling or I'm protecting an application on premise in the identical way that I'm doing it in the cloud. So if it's the same everywhere, what's the value that you're providing that means that I should choose your option than something else? So we do have, this is where we have controllers that live inside of the clusters that manage the stuff for the users. So you could rebuild what we're doing, but you would have to roll it all by hand. But you could, we don't stand in the way of your operations either. So like if we go down, you don't go down type idea. But we do have controllers, we're using CRDs. And so like our app management technology our controllers are just watching for a workload to come into the environment. And then we show that in the interface, but you can just walk away as well if you wanted to. Cool? There's also a constellation of other services that we're building around this experience, that do draw again from some of those storage and data management capabilities. So, you know, stateful sets, your traditional workloads that want to interact with or transact data against a block or a shared file system. We're providing capabilities for sophisticated qualities of persistence that can exist in all of those same public clouds. But moreover, over time, we're going to be in on-premise as well. We're going to be able to actually move, migrate, place, cash per policy, your persistent data with your workloads as you move, migrate, scale, burst, whatever you pay for it. Whatever the model is as you move across and between clouds. Okay. How far down that pathway do you think we are? Because one criticism of Kubernetes is that a lot of the tooling that we're used to from more traditional ways of operating this kind of infrastructure isn't really there yet. Hence the question about, we actually need to make this easier to use. How far down that pathway are we? Well, I would argue that the tooling that I've built has already solved some of those problems. So I think we're pretty far down the path. Now, what we haven't done is open sourced, you know, all my tooling, right? To make it easier on everybody else. Rob, NetApp's got strong partnerships across the cloud platforms. I had a chance to interview George at the Google Cloud event. I know you partner the year, I believe, on some of these stuff. Help us understand how some of the things that the team are building interact with the public cloud. You know, you look at Anthos and Azure Arc, and of course, Amazon has many different ways you can do your container and management piece there. You know, to talk a little bit about that relationship and how both with those partners and then across those partners, you know, work. Yeah, it's, wow, so how much time do we have? So there's certainly a lot of facets to that, but drawing from the Google experience, we just announced the general availability of cloud volumes on tap. So the ability to stand up and manage your own on tap instance in Google's cloud. Likewise, we've announced the general availability of the cloud volume service, which gives you the managed push button as a service experience of shared file system on demand at Google. I believe it's either today or yesterday, in London. I guess maybe I'll blame that on the time zone conversion, not knowing what day it was. But the point is, that's now generally available. Some of those capabilities are going to be able to be connected to our ability from NKS to deploy a on-demand Kubernetes cluster and deploy applications from a marketplace experience in a common way, not just with Google, but with Azure, with Amazon. And so, you know, frankly, the story does differ a little bit from one cloud to the next, but the endeavor is to provide common capabilities across all of them. It's also the case that we do have people that are very opinionated about, I want to live only in the Google or the Microsoft or the Amazon, because we're trying to deliver a rich experience for those folks as well, even if you don't value the agnostic multi-cloud experience. Yeah, and Matt, I'm sure you have a viewpoint on this, but it's that skill set that's really challenging. I was at the Microsoft show, and you've got people, it's not just about .NET, they're embracing and open to all of these environments, but people tend to have the environment that they're used to, and for multi-cloud to be a reality, it needs to be a little bit easier for me to go between them, but it's still, we're still, we're making progress, but there's work to do. Yeah. What's the question? Yeah, so I just, I know you're building tools and everything, but what more do we need to do? What are some of the areas that you're hopeful for, but what are the areas that we need to go further? So for me, it's coming down to the data side, like I need to be able to say that when I turn on data services inside of Kubernetes, I need to be able to have that workload go anywhere, right? And because as a developer, so I have, I'm running in production, I'm running in Amazon, but maybe I'm doing tests locally on my bare metal environments, right? I need to, I want to be able to maybe sink down some of my data that I'm working with in production down to my test environment. That stuff's missing, there's no one doing that right now, and that's where we're headed is, that's the path, that's where we're headed. Yeah, I'm glad you brought that up actually, because one of the things that I feel like I heard it a little bit last year, but it is highlighted more this year, is we're talking a little bit more to the application, to the application developers, because Kubernetes is a piece of the infrastructure, but it's about those- It's the kernel. Yeah, yeah, it's the kernel there. So how do we make sure we're spanning between what the app developer needs and still making sure that infrastructure is taken care of? Because storage and networking are still hard. It is, yeah, yeah, I mean, I'm approaching, I'm thinking more along the lines of, I'm trying to think more about app developers personally than infrastructure at this point. For me, so I can give you a cluster in three minutes, so I don't really have to worry about that problem. We also put Istio on top of the clusters, so it's like we're trying to create this whole narrative that you can manage that environment on day one, day two type of operations. But, and that's for like an IT manager, so inside of our product, how I'm addressing this is you have personas. And so you have this concept, you have an IT manager, they can do these things, they can set limits, but for the developer who's building the applications or the services and pushing those up into the environment, they need to have a sense of freedom, right? And so on that side of the house, I'm trying not to break them out of their tooling, so part of our product ties in to get, so we have CD, so you just do a get push, get commit to a branch, and we can target multiple clusters, right? But at no point did the developer actually draft YAML or anything, we basically create the container for you, create the deployment, bring it online. And I feel like there's these lines and the IT guys need to be able to say, I need to create the guardrails for the devs, but I don't want to make it seem like I'm creating guardrails for the devs because the devs don't like that. So that's how I'm balancing it. Okay, yeah, because that has always been the tension and that there's a lot of talk about DevOps, but you go and talk to application developers and they don't want to have anything to do with infrastructure, they just want to program to an API and get things done. They would like this infrastructure to be seamless. Yeah, and what we do, like also what I'm giving them is, like service dashboards, because as a developer, you know, because now you're in charge of your QA, you're writing your tests, you're pushing it through CI, it's going to CD, you own your service and production, right? And so we're delivering dashboards as well for services that the developers are running so they can dig in and say, oh, here's an issue or here's where the issue is probably going to be at, I'm going to go fix this. And we're trying to create that type of, like scenario for a developer and for an IT manager. So a slightly different angle on it by understanding that question correctly is, you know, part of the complexity of infrastructure is something we're also trying to provide a deterministic sort of easy button capability for to, you know, perhaps you're familiar with NetApp's Nissan HCI product, which we kind of expand that as hybrid cloud infrastructure. If the intention is to make it a simple private cloud capability, and indeed, our NetApp Kubernetes service operates directly off of it, it's a big part of actually how we deliver cloud services from it. So the point is that if you're that application developer, if you want the effective NKS on-prem, the endeavor with our NetApp HCI product is to give you that sort of easy button experience because you didn't really want to be a storage admin or a network admin. You didn't want to get into the me-mired and the details of infra. So, you know, that's obviously work in progress, but we think we're definitely headed down the right direction. Turnkey on-prem. Yeah, it does seem that a lot of enterprises want to have the cloud-like experience, but they want to be able to bring it home. We're seeing that a lot more. Yeah, so this is like this turnkey on-premise, turnkey cloud on-premise. And like with NKS, we can, like the same auto scaling. So take the dynamic nature of Kubernetes, right? So I have a base cluster size of say four worker nodes, right? But my workload's going to maybe need to have more nodes, so my auto scaler's going to increase the size of my cluster and decrease the size, right? Pretty much everybody only can do that in the public cloud. I can do that in public cloud and on-premise now. And so that's what we're trying to deliver and that's very cool stuff, I think. Well, there's a lot of advantages to enterprises operating in that way because I have people out here, I can go and buy them, or hire them, and say, hey, we need you to operate this gear and you've already done it elsewhere, you can do it in cloud, you can do it on-site. I can now run my operations the same across no matter where my applications live, which saves me a lot of money on training costs, on development costs, and generally it makes for a much more smooth and seamless experience. So Rob, if you could, just love your takeaway on NetApp's participation here at the event and what you want people to take away from the show this year. So it's certainly the case that we're doing a lot of great work. We like people to become aware of it. NetApp, of course, is not, I think we talked about this in perhaps other contexts, not strictly a storage and data management company only. We do draw from the strengths of that as we're providing full-stack capabilities in a way that are interconnected with public cloud, things like our NetApp Kubernetes services, really the foundational glue in many ways to how we deliver the application runtime, but over time we'll build a constellation of data-centric capabilities around that as well. Matt, I'd just love to get your viewpoint as someone that built a company in this ecosystem. There's so many startups here. Give us kind of that founder viewpoint of being in this sort of ecosystem. So I came into the ecosystem at the beginning. I would have to say that it does feel different at this point. I'm going to speak as Matt, not as NetApp. And so my thinking has always been it feels a lot like you're a big fan of that rock band and you go to the local club and we all get to know each other at that local club and there's like maybe 500 of us or 1,000 of us and then that band gets signed to Warner Brothers and goes to the top and now there's 20,000 people or 12,000 people. That's how it feels to me right now. I think, but what I like about it is that it just shows that the power of the community is now at a point where it's drawing in like cities now, not just a small collection of a tribe of people. And I think that's a very powerful thing with this community and like all the what are they called the Kubernetes summits that they're doing? I mean, we didn't have any of those back when we first got going. I mean, it was tough to fill the room. And now we can fill the room and it's amazing. And what I like seeing is people moving past the problem of Kubernetes itself and moving into like what other problems can I solve on top of Kubernetes? So you're starting to see all these really exciting startups doing really neat things. And I really like this vendor hall I really like because you get to see all the new guys. But there's a lot of neat stuff going on. And I'm excited to see where the community goes in the next five years. But it's, we've gone from zero to 60. Like it's insanely fast. Cause you guys were at the original KubeCon, I think, as well. It's our fourth year doing the cube at this show, but absolutely, we've watched it since the early days. I'm not supposed to mention OpenStack at this show, but we remember talking to JJ and some of the early people there.  So yeah, we've been fortunate to be on here for really day zero here. And definitely great energy. So much, congrats so much on the progress. Really appreciate the updates on everything going. And as you said, right, we're, you know, we've reached a certain estate and just adding more value on top of this whole environment. Yeah, we're now like, we're in like junior high now. Right, and we're in grade school for a few years. All right, well, Matt and Rob, thank you so much for the update. Thanks for having us. Hopefully not an awkward dance tonight for the junior people. For Justin Warren, I'm Stu Miniman. Back with more coverage here from KubeCon and CloudNativeCon 2019 in San Diego. Thank you for watching theCUBE.


Assigned that I'm that I'm supposed to start. Hi. Welcome to PHP unit for Drupal developers I am Sebastian and I've been doing things with PHP on to PHP for a really long time now And when I say a really long time that it that means it's Little bit over 19 years now that I had my first contact with PHP I Grew up at least when it comes to computing and programming in the Amiga demo scene I learned programming in see an assembly machine language on on the Amiga back in the 80s and 90s and at some point Someone who I knew from from that community at some point contacted me and said hey back then you did the 2d and 3d real-time Graphics routines in assembly on the Amiga. That's basically the same thing as programming websites But that was our graphics artist from the team that I was involved in and he was not doing websites and he said well I need something That is not just html, but something dynamic on the website Was it was a very complicated? guestbook that they wanted basically But for you that's exactly the same as what you did back then I said I have never done anything with web development What are the programming languages and he said well you have the choice between PHP and Pearl? And I looked at Pearl for like an hour or so and then decided no, I don't want to learn that And I went with PHP and within a week and I learned enough PHP to implement what my friend wanted and then a Couple of months later. I started to use PHP for real and at some point while I was still in university I came across the concepts of testing and quality assurance and unit tests and J unit and I Wanted to have something like that for PHP it didn't exist and without really thinking about it I started to work on PHP unit and that was some time around Summer fall of 2000 and in 2001. I published the first version of PHP unit and now I'm here What am I doing when I'm not speaking at conferences and not working on open source projects? I'm an IT consultant with the PHP consulting company. I help PHP teams Like this team over there to build better software using PHP and that involves a lot of Things around PHP units writing Effective tests coming up with efficient testing strategies and teaching them how to write testable code Because without testable code. It's not fun That's my company, but I'm not Going into any marketing Some of the companies that I've had the pleasure of working with some really small some really large prop most of them You probably never have heard of some of them. You have heard of I guess, but no marketing The only thing that I would like to say before I start With the presentation is I would like to thank everyone who made it possible for me to be here There was a crowdfunding to cover my travel expenses to get here and if you want to find out Why I need my travel expenses covered then I invite you to read this article that I wrote on on my blog And that's it. Thank you to everyone who made it possible for me to be here so Websites testing. What is this all about? I have this horrible recollection of the first time that I set foot into an Agency an internet web development agency somewhere around 1999 No, that's not that's not right summer 2000 or summer of 2000. I had vacation from University The world's first PHP conference which happened in October 2000 in Cologne had just had published their schedule and I was on the schedule and the company from Cologne called me I Still have no clue how they got my phone number But they called me on a Friday and said hey We we learned from the agenda of this conference that you know something about PHP We need a PHP developer and we know that you are at university and university is currently off So you are free and can you start Monday?  Yes, and on Monday I went there and worked there for a couple of days and then then I left But what I found found to really interesting was they really started This paper and big plans planning for six months and then after six months they started doing things in Photoshop and at some point maybe Photoshop or maybe it was Something else dream we were or something front page in 90s long ago. I'm getting old software that we don't want to talk about so today. We probably would use something like Photoshop Generate HTML from that and then you look at that for the first time in your browser and then you need to tweak it a bit and What do you do when you work on HTML CSS JavaScript and you look at it in your website and you are tweaking things You hit f5 a lot That's the at least the front-end developers main feedback loop. I changed something. I hit f5 and It's really interesting in in hindsight how old this f5 hitting for reload is I reason I actually checked recently This by the way is my second computer, but the one that I still have it's an Amiga 1200. It still works I no longer use dial-up modem to connect to the internet now. It's on Wi-Fi and That works and back then the really early versions of Graphical browsers even for the Amiga you hit f5 to reload and that's how I started with HTML and CSS I'm not sure if it was CSS at the time. They may be just plain HTML On the Amiga way back then so you hit f5 when you made a change to see whether or not the change had the desired effect That's our feedback loop Okay That maybe that kind of feedback loop may be fine if you're just looking at the visual things When you're looking at okay, I changed this CSS rule I changed this tag from an h1 to an h2. Does it look right? Maybe then this kind of feedback loop is okay and sufficient, but what if we change Something in our peach decode something that actually has Maybe side effects. Maybe we have an e-commerce application and even I click something I don't own I not only want to see a new page the next page in the process But I actually want to make sure that I Actually talked to the payment service correctly and got my money so that I can send a note to the warehouse and Ship something so what does our feedback loop look like for php code and of course at least in the early days since Many php developers came from front end. They use the same feedback loop They just made change in php code hit f5 and looked what happened But in most cases for for for application logic for the main logic you need to look deeper So we need to actually open up our Not the computer. That's just for for visualization, but we need to look deeper. We need to look what happens inside the php code on the back end and not just on the front end and Maybe We go to the command line use the php build and web server run in this case our application, which is Drupal send a request and then Use var dump print R. Whatever we are used to to put into our php code to do On the flight testing or debugging The old school way that Maybe qualifies as a feedback loop, but for me like around 1999-2000 When I was exposed to J unit at university with Java, which I never really used in production, but I Like the concept and and the idea of writing Something that automatically verifies that my code works correctly That feedback loop started to feel Wrong to me and that's not really the feedback loop you should be looking for Right, that's the main board of my Amiga with some Lego. That's not the feedback loop You're looking for you don't want to do print R and var dump based testing in your production code Historically speaking though, that's how in the really really really early days testing started you put Additional debugging statements into your production code ran the code and checked whether or not At a certain point in time during the execution of your program The program was in the state that you expected it to be in There are programming languages where that is still a Best practice a good practice today. So see C++ and other languages have built-in constructs The assert statement that is that checks at runtime whether or not The software is in a given state. I mean, it's not in that state execution is aborted That works and it can be used to put some safeguards into into place that you do not run into really bad situations into Into situations that you don't want to be in illegal states of the system if you will But it's not really testing. It's just a safeguard and yes That can be abused for some testing or some debugging on why you're testing But the tests do not belong into the production code the code that actually runs in production and that's the realization that I Don't know who first came up with it the earliest framework that I know that had this idea was as unit for small talk by Kent Beck Which according to legend almost nobody used and Then they had this famous Flight from from Zurich to Atlanta where Kent Beck was sitting next to Eric Gamma and Eric Gamma on his laptop Had Java with him like one of the first versions of Java and he had heard about S unit and he wanted to have S unit for Java and Kent Beck had heard about Java but had not used Java yet So they ported while flying from Zurich to Atlanta as unit to Java and out came J unit and Their idea was okay. Let's take these debugging statements Like what we in PHP would use print or print R or Vardam. They just used a print or a print F Take these statements Pull them out of the production code Put them into separate source code files that we can run When we want to test something and we don't need to put that those files into production Don't need to compile them into the production binary. Don't deploy them have them separate so that we have a separation of production code and test code and that's what all these Test frameworks that are in the tradition of the X unit family of test frameworks S unit J unit CPP unit was the third one PHP unit was It came shortly after CPP unit So so that Can give us a different feedback loop, right? We want to automate our tests Another downside in addition to having test code and production code mixed Mixing things is never a good idea at least not in software development Another downside is that you cannot really automatically evaluate it You always need a human to look at the Vardam output and figure out whether or not that makes sense whether that's the correct state Yes, of course, you can put additional Logic in there and make a check and print something nice when it works correctly or only print something when it's wrong but that's It still doesn't belong into the production code that so we put it out we put it out write it somewhere else and Automate it and make it really easy to run these tests and get useful Information out of running these tests things like how many tests that I execute very successful which tests failed What is the reason why these tests have failed? Where in the code was the application running or currently executing when something failed? Which parts of my application I actually covered by tests and where Do I have blind spots? Where am I missing tests? So that leads us to automated tests and automated tests is a piece of code that you write down that can be automatically executed by a testing tool and PHP unit is two things. It's on the one hand a framework for writing tests for making it as easy and simple and convenient as possible to write down your test intent to Capture how you want to invoke the unit of code that you're that you're testing and What you expect to happen what you expect for the test to be successful? What is success criteria is and the other thing that PHP unit is is a command line based tool that Executes these tests and reports useful information based on the test execution When we talk about software tests, there are many different types or categories of software tests one category already mentioned automated tests the opposite of automated tests is of course manual testing using a Real human with a real mouse and a real keyboard in front of a real computer with a real browser exercising the application And these kinds of tests are really useful when you want to do Usability tests for instance is the user interface or is the user experience? Does it work? Do your users actually understand it and can use your application? Conveniently that's automated versus manual another categorization that now comes into play is dynamic versus static Tests that you implement using PHP unit are so-called dynamic tests because they test the software While executing the software while the software is running That's what you need to do for Testing using PHP unit for testing things like performance in scalability With tools like Jmeter and Siege and others There's a complete separate category of tests from that that is called static testing or static analysis Testing the software without actually executing it. That's what PHP code sniffer for instance does it looks at your code and checks Whether or not it conforms to your coding guidelines or things like PHP copy paste detector that look at your code and tell you Where you have duplicated pieces of code that you should consolidate into only one occurrence of the code or things like PHP stand or fan that actually Try to comprehend the code and tell you okay. You have something you have an area here that is hard to understand That's in the long run prone to errors and hard to understand for new developers that have to work with it Refactors this make it simpler so that it's easier to read and easier to understand the code or to find Bugs and other problems We are talking about automated dynamic tests and one of the most common and popular Ways of implementing these automatic dynamic tests is in the form of unit tests exercise tests that exercise one unit of code in Isolation from all of their dependencies these tests are really precise when they fail they point exactly to the line of code Where something went wrong since they only execute very little codes. They are really fast to execute That's all nice characteristics So let's start I Mentioned earlier that PHP unit is a framework for writing tests and like any other framework There are a couple of conventions that you should follow to get a lot of convenience functionality for free first convention is That the tests for something that is named email go into a class that is named email test Such an email test class then extends PHP unit framework test case that is the default base class for your test case classes Should you have a completely different idea for how you want to write your tests and Still want to leverage a lot and reuse a lot of functionality and code that PHP unit provides You can do that. You simply need to implement a very narrow interface PHP unit framework test As far as I know at least nobody has ever done that And I hope that means that PHP unit framework test case is sufficient and convenient for most people at least the next convention is that a Single test goes into a public method Where the name of the method begins with the prefix test? some people don't like this method name prefix of tests and they can just omit it and add an Annotation with at test in the dock block for that method if that makes them happy I am happy and to each their own but most people just use the method name with the prefix test please do yourself a favor and Think for a couple of seconds and come up with a sensible name for the test That precisely and concisely Describes what it is that you're testing in one of the earliest discussions that I had about PHP unit and tests at a customer Some 15 16 years ago The developers were complaining that PHP unit does not give them useful error messages when a test fails And I said I cannot understand that because you can just look at the name of the test And then you see what it is that you were trying to test and then you see what went wrong and what is currently broken No, no, no, that doesn't work because look at the output and what they did was they just enumerated the test That test one test to test three test 4096 and so on of Course if you don't provide the information PHP unit cannot provide the information when something fails There is no AI machine learning crystal ball voodoo magic inside PHP unit that can Understand and comprehend the intent of what you're trying to test you need to encode That information in the name of the test Naming is one of the hardest problems in Programming a Bad name for a class for a method for a variable can cost a lot of time and Nerves and effort in the long run so Take your time take a couple of seconds if then the name is not clear talk to someone Ideally to a co-worker if a co-worker is not there talk to your elephant or your rubber duck and there's this thing in in in psychology and Neurology that basically says if you have a problem sometimes the solution comes when you Say it out loud because in different areas of your brain are active And it doesn't really matter if you talk to a real human or to a rubber duck or in the case of the PHP community to your stuffed elephant Think about names names are important naming things is important So we want to write a test that verifies that we can create an email object from a valid email address and then inside your test You can basically do whatever you want PHP unit does not care What the PHP code looks like that you invoke you can invoke a global function You can create an object and call a method In some really really really old PHP To code that I saw a couple of one only a couple of years ago that is still in production It's like a 20 year old version of PHP. Why not? They were not even using functions What was supposed to be a function was in the PHP script and when you wanted to call that function You included that script and before that you put the parameters Into specific global variables and they were read from there and they were deleted and the result came into another global variable It works and you can test that with PHP unit. Please don't write such code in 2017, but PHP unit does not care what you do inside the test method What PHP unit does care about is you need to tell it how you want to wear or what you want to verify What is your condition for the test to be successful and for that? PHP unit offers you a wide range of so-called assertions assertion methods like a search instance of or assert equals or assert true assert false and So on use the most specific one possible that expresses What it is that you want to verify in this case we want to verify that we get an email object back That's why we use a certain instance of In case you were wondering what weird syntax constructs I was using there like the double colon void at the end of the method signature or The declare strict types equals one Pragma at the beginning of the PHP source code file I'm of course using PHP 7-1 syntax because that's the current version of PHP and because it's my personal opinion as At least as in when I'm wearing my open source hat at a conference that life is too short for old versions of PHP I know that's kind of risky or Scary thing for me to say at the Drupal event where you still support PHP 5 5, which is very very red because PHP 5 5 is dead dead dead for one and a half years now and But okay, that's my opinion And new code that I write I write using PHP 7-1 which is really nice It's also a lot faster and useless memory In case you need more arguments If you want to learn about PHP 7 That's the only marketing that I'm going to do We wrote a book about PHP 7 PHP 7 explained there's a discount 25% discount for Drupal con attendees I There's one thing that is special about this ebook It's a one-time purchase and we'll you'll receive free updates for as long as PHP 7 is current Even if ten years from now PHP 7.10 is released you'll get an update That covers all the changes all the new features of PHP 7.10 and we are Currently working putting the finishing touches on the coverage of PHP 7.2, which will be released at the end of November early December Which is shaping up to be yet another really great release lot a lot faster than PHP 7.1 Dimitri and Nikita and Xintian are doing amazing work on The compiler and the runtime and the optimizer really cool stuff so Side note and over so this is our test now we can try to run it As I mentioned earlier PHP unit is a command line tool That you point at your tests and it runs your tests and Of course it tells us hey, I cannot execute this because I cannot find the email class Probably because we have not written it yet For each test that PHP unit executes there's one character of output While the tests are running to indicate progress in this case. That's an E. That's the E that we see up there the E stands for error and an error is Or an error is reported When a PHP error happens while running the test like a class has not been found Or something unexpected happens like an exception that was not handled That's something different Compared to an F for failure an F you will get when an assertion fails you as you expect The return value of that method to be an instance of email and you get back a null That would be an assertion failure that would be reported with an F for failure instead Of course writing that email class It's really simple that's what it could look like Really boring nothing to see there so now we get a dot in the output dot means successful tests and all tests are successful, so we don't get much more output and that is a Philosophy that PHP unit shares with all the other members of the X unit family of test frameworks Yes, there's this assumption that all tests should pass all the time and as long as that is the case Aside from some progress information while the tests are running and some summary at the end This is the amount of tests that I ran. This is the amount of assertions that were performed This is how much memory was consumed and this is how long it took You don't want any more information than that only when something goes wrong When you get an error or failure for instance, then you get detailed output about that specific test that failed or error out Normally you don't want To see output here for I don't know Imagine the Drupal test suite with several thousands of tests when you run them You don't want to see a list of the names of all these tests. You're not interested in that that would be a lot of output and in that Massive amount of output the information that you're really interested in would be lost Maybe 999 tests are successful and you'll see the information for that and somewhere in the middle There's the one test that is failing and you don't see that That's why when everything is fine. No additional output and that's our feedback loop We had we were in the process of writing something that captures an email address We use the test to describe how we want to interact with that email how that email could look like and Then we actually implemented that and that's a really nice feedback loop. It's called test-driven development test-driven development means the tests drive the development not as Not just for the sake of writing tests That's a nice side effect of test-driven development that you get the tests for me at least the most or one of What the most important thing about test-driven development is that it is a tool that helps me Think about the problem. It helps me understand the problem that I'm currently trying to solve and write Another test probably that that one uses probably probably the most common Assertion method that you use a third equals which compares to values Conventions are again are important. Of course for Comparing two values the order in which you compare them is not important if you come with compare a to be That's the same as compare comparing B to a The order, however, is important when the comparison fails and you want to provide a meaningful failure message That's where the convention comes into play The first argument is the value that you expect and the second value is the actual value that you want to verify and Match against the expected value so that PHP unit can tell you hey This test has failed. I expected this value I actually got this that does not match that is why the test has failed conventions oh Yeah, there's the output example. There's the f for failure. There was one failure. I Expected a string and I got an email object that doesn't match That is why the test has failed and then we can then we can implement the missing functionality In this case, we were missing the two string method that allows us to use the object as a string in a string context and then It just works what happens when something goes wrong, how can we test that in an error case of Indicates happens inside the code that we're testing How can we verify that we get an exception when that happens? that's How you test exceptions and Right before this presentation had a discussion with one of you from the audience about exactly this and the question was The documentation shows At least two different ways of testing exceptions, which is the one that I should use or which is That that we should use and As of right now That's the best practice or the good practice that I would recommend of using For a really long time until some point last year or One and a half years ago. I Recommended using the expected exception annotation Which at first glance is a lot more convenient. You put it in an annotation in a dog block in front of the test method It's not in the test code Some people prefer that some people find that easier on the eye easier to read There's one problem with that. There's at least one problem with that though If I put it at the annotate in on the annotation Then the test would be successful if at any point in the execution of the test method that exception is raised This allows me to tell PHP units Only after this point in time While running the test when I have this method call only now is it okay that this exception comes and that marks the test as Successful if that exception is raised before that expect exception call then it is an error and There were a lot of situations Where using the annotation hides problems in the code and That's one of the worst things that you can have when you're testing that you have a test that you went through the effort of writing the test Maybe even went through the effort of making the test the code testable that you're testing and Then have a test that lies to you because You get the exception way earlier and not in the place where you are expecting it And then it's hidden from you and the test tells you it's green. It's fine. Everything is good continue and it's not You don't want lying tests. That's what it looks like when it goes wrong We are missing what? more code we need to actually Have code in our email value object that verifies that this string that we pass is the valid email and Then our third test also works correctly So those were three examples for writing tests compared to variables for for equality using a certain equals Asserting that a variable contains an object of a certain type and ensuring that we get an exception when we want the code to raise an exception and Basically, I have shown you now all that you need to know to write tests for 70 to 80 percent of the cases PHP unit has a lot of functionality Just look at the documentation There are so many features. There are dozens of assertions dozens of configuration settings and Sometimes weird and bizarre modes of running the tests a Lot of those things a lot of those features were added over the course of the project because to support testing I Don't really want to say crappy code But code that was not written with testability in mind in a perfect world Which unfortunately we don't live in in the perfect world I could delete like 80 percent of the code and everything would be fine But It's this 10 20 percent that you need to know to get started and once you have mastered that then it's really easy to look Okay, now I am in this situation now. I need is a very specific assertion Look in the documentation. Oh, I find something. I need to assert that an array is empty and there's a certain empty And that's okay Maybe you are testing a piece of code that has a lot of interaction with global state It's manipulates a lot of global variables There's a feature that you can turn on that makes a backup of all global and super global variables before each test and Place that back after each test Don't turn that on if you don't need it if you write clean modern Object-oriented code you don't need global variables, so you can turn you don't Need to turn that on It's an expensive operation that takes time and consumes memory Don't do that But you can turn it on for the on a test-by-test basis for those tests where you need to Interact this global variables Maybe because you're testing a piece of code that is 10 15 years old and that still use those global variables and that's okay But to get the general idea of how to write and execute tests You don't need to care about that. You only need to know That's that's possible When you need it so all the tests that I've shown so far are unit tests and all of these Automated dynamic tests What they do is verification through execution. I execute a piece of code and I verify that it does What it is supposed to be doing or at least what I have understood what it should be doing Just because a unit test passes Does not necessarily mean that the feature that uses the unit of code that fit that we're using there That we're testing with the unit test Does what the user what the customer actually wanted? It's a different type of test that would that would be an acceptance test for instance You can write code that is really nice and clean and Modern and has good unit test coverage, but it still does not do what it is what is supposed to be doing Verifications through execution means I have a piece of code. I invoke it. I look at the result and make a decision Yes, this is what I want. This is not what I want There's a very special case Is regard to verifications through execution and That's a category of tests called characterization tests Those tests don't make a statement about whether or not a Software system or a part of a software system works correctly. It just captures how it works today Can give you an example from from a recent customer? They had a really large method four or five thousand lines of code By looking at it. It was not really clear What it was doing why it was doing it was doing a lot you looked at it and you could see okay It was doing four or five six different things Most of which were really important for at least one of those blocks the developers were not really sure whether that was really needed or Whether that related to a feature that you moved from the application three years ago And it was just a leftover We were not since we did not know what it was supposed to be doing He had a couple of arguments and we did not know with which arguments to call it and what the return value should be So we couldn't figure out Arguments to call it with and then an expected result And in a situation like that when you're writing tests for legacy code, that's okay That's where characterization tests can help So we looked at a couple of Requests to the actual site and figure out and got the information from from function traces With which parameters it actually gets called Right now in production and for these Sets of input What is the was what is the result and then we wrote some code a test an integration test that used that information and while we were doing that we realized hey it might be even a better idea to Just duplicate that method in the same class make a copy of that method and In our tests call it twice Call the the original one and called a copy and Then we made changes to that method we started refactoring it We sliced it down into five six smaller methods and we used that test Calling the old version and the new version and just compared The results from those two and as long as they were the same we were comfortable enough that we were not breaking anything that's the characterization test a very special case of verifications through execution You can Have a different perspective on verifications through execution when you Look at a test not only from the perspective of I Want to ensure that this thing works correctly. I'm and leave The view of testing and look at it from a different angle And then you can look at the test code that you wrote at the unit test that you wrote From the perspective of it being executable documentation a unit test is an example for how the unit that its tests can be used Imagine you are a new developer coming on the team That you that has somewhere in its code base this email object I had earlier and For the first time you need to work with that email object Sure thanks to naming, you know what email does but you don't know the implementation details of that and The promise of object oriented programming is that you don't even need to know The implementation details you only need to know the public API But still you need to know how to use that. So what do you do? You look at the unit tests for that email class and you see Actually working examples for how this code can be used Moreover, you see when I call it like this, then this is the result that I get That's perfect. That's really good if you shift your view on that test code slightly Then you go from executable documentation to executable specification Especially in the context of test-driven development the tests are a tool to structure your thoughts and understand the problem and then you use the tests to Specify how the thing that you are about to write should behave PHP unit supports this idea Through an alternative output That's called test docs and you can get that for instance using PHP unit dash dash test docs Then it just uses this alternative output and prints that to the command line. You can also get that as An almost nice looking at least it's not ugly looking html some teams Have really great six have really made a great Success and experience was using at html not to print it out Please don't hurt the trees but they use that in continuous integration and in addition to the normal output that is useful for the developers to generate this html and put it Somewhere on the internet on a wiki for instance and the domain experts the business analysts the non-technical Stakeholders look at this document and they understand that they see okay that the developer is talking about a concept that is called email And that can be created from a valid email address and and so on and that's something that I even as a non-technical person can understand And of course for something rather technical like email. That's rather boring It gets really interesting if you have things like contracts or Other things from your business domain represented as objects in your code plus the executable documentation and specification Through the tests and then you can take this show that to someone here This is how I understood that this piece of our business should work And this is how it currently works because this is automatically verified through tests Did I understand it correctly or not and this output? Believe it or not is a really good communication tool between technical and non-technical stakeholders of a project This Summarizes what I mentioned once or twice already at least for me Test-driven development and the writing of tests Unit tests especially is not just a broad verifying that something works correctly It is a tool that helps me think about the problem that I'm solving and as a developer my primary Job is not the production of code My job is to understand and solve a problem and yes sure as part of that I need to write code But the code is not really That what the user wants the user wants a solution Decades ago Bill Gates already said something similar when he said well The idea of paying software developers by the amount of code they produce is As silly as paying an engineer that create that builds an aircraft by the amount By the weight of the airplane.  You want to solve a problem and you want a good solution And I can only give you my subjective personal experience I cannot work without Unit tests and in most situations I cannot work without test-driven development anymore I have still to figure out how to apply test-driven development to the development of a test framework Maybe at some point I figure that out, but when it comes to PHP unit I always write the tests after I implemented the feature test-driven development at least in most cases for the test framework itself doesn't work for me, but that's a Rather special problem that only people have that work on testing tools. So that should not concern you That's that's my headache not not yours But tests for me are a tool to to structure my thought process While I'm trying to solve a problem and that works for me rather well Yeah, and of course that somehow relates To what uncle Bob talks about or what uncle Bob means when he talks about clean code And to me still the best definition of what clean code is what what what we should strive for as programmers is What Dave Thomas wrote in the introduction? To the clean code book clean code can be read and enhanced by develop by a developer other than your original author I would even go further clean code is code that I can that I have written today and can understand a year from now Does not have to be another person It can be future you write code that future you and Your teammates today can understand Happened to me so often over this I don't know 16 17 years that I've been working on PHP unit that I needed to work On one area and I look at the code What was I thinking when I wrote this I don't understand it and look at the history. Oh, I wrote that 12 years ago Yeah, it turns out I learned something in the last five years and I wouldn't write the code like that anymore But now I have to deal with it and maybe rewrite it or improve it at least a little bit to make the pain go away Yeah, can be read and enhanced by a developer other than its original author It has unit and acceptance tests Meaningful names. Yes naming things is hard, but it's so worth it to have names That really reflects The domain in which your problem is Yeah, one way of doing things And not multiple ways minimal dependencies, which are explicit Meaning dependency injection make it dependencies explicit don't hide dependencies and make them exchangeable lose coupled code and minimal Yeah, and put in yet another way yet into another Phrasing what Steve Freeman and net price wrote in growing object-oriented software guided by tests the goose book For a class to be easy to unit test the class must have explicit dependencies that can be Substitutive and clear responsibilities that can easily be invoked and verified In other terms the code must be loosely coupled and cohease and highly cohesive or in yet another phrasing well designed and Test driven development helps you get there. I talked a lot about unit tests but there are other tests other kinds of software tests other kinds of tests that you can implement using a tool such as PHP units for instance and With all these tests there are a couple of questions that you can Use to categorize these tests What do I want to achieve with the test? What is it that I want to test? Do I want to verify that my software handles an Email address or a credit card number or an iBan or an address Correctly at a very low level. Yes. I want to verify that these things work isolated from everything else. I Do not want to test these things in the largest possible scope I do not want to have thousands of different tests that test with different combinations of credit cards and Credit card information and address information Using the real payment service provider whether or not that works or not That's not what you want to do That's what you want to do on the smallest possible scope because you need a lot of variance in there many different types of test make a lot of test data to exercise these really important pieces of code and If you do that with a unit test these tests are Really really fast and you don't need to talk to other systems and the much more robust robust and they give you precise information Sure, you will need a couple of tests that tests that exercise your entire payment process including talking to payment service providers But you only need very few of those you knew it basically just need to check Does it work for one address with one credit card and if it works for that then you can assume that it works for the rest Because all the variants you have tested on a much smaller scope. How much code do we need to execute? To verify the thing that we want to verify That's probably the most important question when it comes to testing. You always want to execute as few lines of code as possible the more code you execute the longer the test takes the more tech code you execute The more places you need to look for the cause when something fails If you only execute five lines of code as part of a unit test and The test fails then the reason for that test failing can only be in one of those five lines How do we write down the test? How do we formulate? The test impulse which basically means how do I write down how I build my environment? How do I invoke the piece of software that I want to test and how do I write down the expectation for this test? To be successful. There are many different ways I've shown you how to write down tests with PHP unit. There are other tools that allow you for instance to use natural language things like cucumber and be had which are well suited for large scope Acceptance kind of tests When do we write the test do I write a test after the fact do I do a test-driven development and write a test before? When do I execute the test a lot of tests make a lot of sense to execute in production? things like a B tests multivariate tests to test a hypothesis about a New feature does it make sense to implement that feature? Do I get better conversion if I make the button purple or blue things like that? You have to do and you have to test in production what you don't want to test in production is whether or not Your credit card validation works correctly. That would be rather stupid and if you Google or use your favorite search engine to Ask There's a thing called the internet for different categories of software tests. You most likely you will find one of many different pyramids Last time I checked there were like 15 different visualizations for software categories that use the pyramid Going to show you one pyramid That's rather representative of the others At the bottom you have Unit tests pure unit tests small tests Whatever you want to call them that only execute code from one unit of code in isolation from all of the dependencies Including not touching the file system not not not talking to the database not talking to the network in Some rare cases the object does not talk to other objects because it has no other dependencies It's rare, but it happens Most likely you have something on that level where you need to do where you need to use techniques such as stubs and mocks To decouple the unit of code that you're testing from its dependencies One level up you have integration tests Just because you test object a in isolation from its dependency b and be in isolation from a does not necessarily mean that a And be worked to correct it together That's what you do with integration tests Then you one level up you do edge-to-edge tests Which is as and to end as possible without using an a real HTTP client and a real HTTP server This is what you would do with a symphony application for instance where you instantiate the application inside the same PHP process that runs the PHP unit tests Say you you have a php unit test an integration test and you instantiate to Drupal and Prepare a request and give that request to Drupal and Drupal does its thing and gives you a response and you look into that response That would be an edge-to-edge tests It's a lot faster than using a real browser Instrumenting that sending a real HTTP request to the server and looking at the response Of course sometimes you need to do that especially if you have JavaScript in your front end and you need to test JavaScript interaction Then yeah, I'm already talking faster. My my clocks has one minute. Okay This is the last slide. I think So Of course if you have JavaScript and you need to do Something with JavaScript something like Selenium that instruments of Firefox the Chrome There's nothing you can do about that, but the pyramid doesn't stop there. You can go Even further up, but then you cannot automate it anymore Higher up would mean maybe you are an e-commerce and you want to test your Logistics tool chain and your warehouse What you can automate is go to the website search for a product go to the product detail page Put it into the shopping cart make a check out including a real payment and then have people in the warehouse Look at the people who work in the warehouse Do they take the shortest route to the shelf? Do they take the right kind of package? How long does that take and how long does it take for the? The delivery service to ring at your doorbell and deliver the package and yes People are doing that but no they're not doing that in continuous integration. They do that once a year So Almost on time sorry Thank you, I hope this was gave you an overview of what this PHP unit thing is about Hopefully in case you were scared of Writing tests and using PHP unit before hopefully you're not scared anymore Hopefully you're interested in trying this out And the last thing I'm going to say is I'm here at Drupal con tomorrow at the code sprint and Apparently I'm helping people Who work on tests for for Drupal core? But if you have a question about PHP unit be it related to Drupal or not feel free to grab me ask your question And I'm looking forward to discuss all things PHP and PHP unit with you tomorrow and help you make Drupal better. Thank you


Hi everyone! We're here to tell you all about the Teen Reading Club at the West Vancouver Memorial Library. Yes, Teen Reading Club is still on. We have something for everyone in all grades and at all reading levels. This video is for anyone in grades 6 to 12 as of June 2022. For those of you in grades 7 and up, the Teen Reading Club is the program for you. Just sit tight and we'll get to that in just a second. For those of you in grade 6, you have a choice to make. You can choose to do one more year of the Kids Summer Reading Club where you read for at least 15 minutes a day for 50 days throughout the summer and earn yourself a medal at the end. Or you can choose to make this your first year you join the Teen Summer Reading Club. As you'll soon learn, Teen Reading Club is more about the number of books you read, and there's a chance that you can win a way bigger prize. But you have to choose. You can't do both. Now we're going to tell you what Teen Reading Club is, why you should join, how to sign up, and some of our very favorite new books that you can get at the library. Teen Reading Club or TeenRC is all about celebrating the books that you already want to read for fun this summer and helping you read just a little bit outside your comfort zone. So if you feel stuck in a reading rut and need help finding some great new titles, TeenRC is a great way to do that. But more than reading, TeenRC also encourages you to be creative and to get out into our community safely. To complete the Teen Reading Club, you will need to complete two different types of challenges. Reading challenges and beyond reading challenges. First, the reading challenges. You get to choose 10 reading challenges from the list for a total of 10 books over the summer. Don't worry, the challenges are broad enough that you'll still have tons of choice about the titles you read. One challenge asks you to read a book by an author who lives in BC. One asks you to read a graphic novel or a manga. And one asks you to choose a book about an immigrant or a refugee. Many of the challenges come with lists of books we recommend so you won't have trouble finding any. The beyond reading challenges are community related challenges like creating art inspired by nature or climate change or learning three new phrases in a language or even interviewing the oldest person you know.  There are lots of reasons to join the club besides getting to choose what you read. Reason number one, it's totally free. It doesn't cost any money and you don't even need a library card to join. You really have nothing to lose. Reason number two, prizes. Everyone who completes the Teen Reading Club gets entered in a draw and I can tell you that we have lots of prizes. Prizes you'd actually want. Plus you could write a book review every week and submit it for a chance to win extra prizes. Reason number three, beat the boredom. Teen Reading Club is a great way to change up your routine, read something new and discover your community. If you've never tried Teen Reading Club, this is the year to give it a try. Reason number four, make your brain happy. Did you know that even when you read for fun you're getting smarter and giving yourself a head start for going back to school in September? It's true. Signing up for Teen RC is easier than ever. This year's sign-up is completely virtual so you don't even have to come into the library to join. All you need to do is go to the Teen RC website at wvml.ca slash trc and click the link that says register now. You'll enter your first name, grade, school and email address and then you'll be registered. Once you're registered for Teen Reading Club, you can complete the challenges at your own pace throughout the summer. You don't have to come into the library at all, but if you do, we're always happy to see you. At the end of the summer, just email us with your completed challenge card and you'll be entered into the prize draw. And remember, you choose the reading challenges you complete and the titles you read for those challenges. You can also choose how you read or listen to books. You can read a graphic novel or nonfiction book, listening to a digital audio book on Libby. Libby by Overdrive is the name of the free app with ebooks and e-audio books that you can access with your library card. If you haven't tried Libby yet, you have to. It's just as good as using the Amazon Kindle Store or Audible, except instead of paying for a membership or titles yourself, the library pays that bill for you. To get started, just search for Libby by Overdrive in any app store and use your library card to set up your account. We can help you or do this over the phone, by email or in the library. Listen to a play away. These are self-contained audiobook players you can borrow from the library. We have lots of popular titles for upper elementary and secondary grades. Read in any language or multiple languages. We have books in all these different languages that you can borrow from the library. Plus, you don't have to just read library books or books labeled teen or young adult. You can read whatever you want. The choice is yours. But now that we have you here, we can't really resist giving you some book recommendations. I mean, it's not our fault. We're librarians, okay? It's just kind of what we do. We recommend books. Falling short by Ernesto Cisneros. Isaac and Marco couldn't be more different. Isaac is a star basketball player and Marco is a straight-A student, but they have been best friends since they were little. Now they're starting middle school and they want more than anything to impress their families, especially their fathers. Isaac needs better grades and Marco wants to join a sport to earn his father's approval. Their strong friendship and unique strengths will help both of them succeed in middle school and at home. Children of the Fox by Kevin Sands. This first new book in the Thieves of Shadow series follows a 14-year-old con artist who has been contracted to be part of a group of young thieves who are then hired to steal this magical stone from the high weaver who is the world's most powerful man. It is a fast-paced heist novel that is set in this magical fantasy world that feels steampunky at times. Very similar vibes to the Rick Riordan books because it can get a little bit dark, but it's so good. The audiobook is fantastic. The narrator does such a wonderful job at giving everybody their unique personality through their voice and I highly recommend you check it out on Libby. Every group of friends needs a place to belong and for Wolf and her friends that place is their tree fort in Birchwood. It's Wolf's favorite place on earth. It allows her to connect with nature, her friends and her ancestors. When new construction puts Birchwood in danger, the friends devise a plan to save their favorite place. Can they do it? Find out in Danielle Daniels Forever Birchwood. The Girl from the Sea by Molly Ostrotek. Morgan and Kelty each have their secrets.  Their love story blooms when they share these secrets with each other and fight to save Kelty's seal friend's home. This is a beautiful graphic novel to read if you loved Heartstopper. If you wanted the recipe for Kaelin Barron's This Poison Heart, I would say take one gothic mansion, add a splash of family secrets, a pinch of Greek mythology and a sprinkling of witches and add it all up to get a fantastic page turning series starter. Watch for book two this wicked fate out this summer. You can find all of these books in our library catalog but please make sure to place a whole diet even if there's a long waiting list because that means we can order more copies. You can also find these books on our Lucky Day shelf. Lucky Day books are special extra copies of some of our most popular books at the library but you can only find them if you visit us in the library. Finding one means that you get to skip the wait list. It's like winning the lottery! So that's everything that you need to know about the teen RC and if you still have questions please feel free to visit wvml.ca slash trc or email us call us or even pop in and say hi we're on the second floor in the youth department. We are pumped for an awesome summer of reading and can't wait for you to join the club. Remember no matter what is going on you always belong at the library and we can't wait to see you. Happy reading!