So now we're going to add our second node. So we're going to add our UR node into our link list now. So what we're going to have now will be our ADC node. When we finish doing our ADC transfer, we will then start our UR transfer, and then that one will link back to the start, which will then do the ADC transfer again and so on. So first thing is... Quick question, where this ADC takes data from? You're checking the channels from channels one, two, three, four. That's what we set as those four single-ended channels. Right, but the analog data. Yes, it's just receiving analog data. From where? Whatever those pins are currently floating at. Okay. The data that we're receiving isn't important. It's just showing you that data is coming in. You had zeros in your register, you're seeing different values now. I thought the data comes from some measuring something on the board or something. No, no, no. It's just taking whatever those pins are currently floating at at the time. Okay. I don't believe they're connected to anything else on the nuclear board. I haven't had a proper look at the schematics. I believe it's just floating pins because they are moving up and down when you look at the data. Okay. Yeah, so what we're reading is not relevant. It's the process that you're looking at here. But in principle, if you connect those to some source, then you will read them. Yes, yeah. If you widen different voltage sources, you should have nice stable values sat on each of those pins from whatever you're reading from. Okay. If you were reading, say, temperature sensors on each one, then you'd have a fairly slow-moving source whereas ours I think are fluctuating quite a lot because it's just floating whatever noise is on the board. Right. So what we are going to do now is our first node loads in our parameters into our DMA channel, which is going to be our ADC read. That will then go through and count down from its number of transactions it's got to do. So it will load channel one in, channel two, so on, so on, so on, until it fills the all 64 elements of our data array. As soon as we've completed that, our link list will then switch over to node two. All the parameters of node two will get loaded into the DMA channel and the process starts again. This time, the UR will be sending the data from the memory out over your UR channel. And again, it will go through each element at a time until it reaches the end and then it will jump back to configuring the ADC into the DMA channel and it will start the process over again. So let's now do this in the UR, which is why I said you could start from your homework because this is exactly what you did in your homework. So if you've started from your homework, then you've got a fairly easy time now because everything here is already done for you. So if you go back to your cube and X, we're now going to enable Usart number one on the connectivity. You're looking for Usart number one and we want to be asynchronous and that should enable pins nine and ten on our pinout diagram. We don't need to change any configurations. We can leave the board rate as its default value there, which is fine. So that's all we need for the peripheral side. Now we go back to our linked list in Utilities. We need to highlight our queue name at the top and we need to add a new list. Oh, sorry, add a new node. Sorry, not the list. We need to add a node to our existing list. So we need to give our node a new name. So I'm going to call it node two to make life easy for my screen captures. It's your node name two. No space. Just put our node name two. And the configuration that this one is going to be linked to is our Usart number one. So in request configuration, it's going to pull in Usart number one TX. We're transmitting data. So it's the TX version of Usart number one. And in the channel configuration, we need to change the direction because we are going from memory to peripheral. Our source data is the memory array. So that's going to increment. So we need to enable source address increment. Our runtime configuration is going to be our parameters from our data array. So where's my copy screens? There we go. So our source address is data because that's our data buffer. Our destination address is going to be our data register, our transmit data register for our UART number one. And our data size is going to be the same as we had for our ADC, which is 64 times two. And that's it. We don't have to change anything on data width because we're only sending out bye bye bye over the UART. Therefore, nothing has to change in our destination data settings either. Once you've configured all that, make sure your name is node name two, your node name two, and then you can generate your code again. Terminate my debug session because I left my debug session running. I want to open my project. There we go. So now hopefully, once you terminate your debug session, your code should have updated. And in main.c, you'll know if it's updated because you'll see you've now got some UART configuration in here. Therefore, you'll know your cube IDE has updated. We now need to enable the DMA for the use. Do we still have the old code or we have to write it again? No, no, all your old code should have been placed between a user code begin and a user code end, so it should all be maintained. So there's two of the lines that we pasted in earlier there, in mine, in the private variables. So everything you should have pasted, as long as you pasted in the correct locations, between a user code begin and a user code end, your code is maintained. So the first item we need to do is link the DMA to use art number one. So this we can do with one line of code. And that can go inside our user code begin to. So it sits between the link list and the ADC starts. 114 for me. Next item we need to do is enable the UART itself. So we've linked the UART to the DMA. Now we need to enable the UART. So that goes after we've linked it in your code. Make sure it goes after we've done the link. So that goes at line 115 for me. And now, sadly, in this current version, there is a bug inside our Cubemax generation. So we have to go into our linklist.c file and change our labels. Now we have to be very careful of which ones we're changing.  Then you'll see all the configuration for the ADC. And if you remember correctly, your ADC is your node name, not your node name number two. So at line 83 on my screen, this should be your node name. You need to delete the twos between line 83 and 87. Then if you scroll a bit further down your link list, you'll see your UART configuration, which is correct. That is your node name number two. It's between lines 83 and 87, just after all your configuration for your ADC. That is your node name, not your node name number two. So once you've done that, you should be able to highlight the project and build it. Again, you'll see the warnings about the casting incorrect there. And now if you run your code, launch a terminal program. So you can have a launch the inbuilt one that you've done in your homework, or you can do an external one like I'm doing here. I change my setup to be 115200. You can see there I've got data streaming out at one hell of a speed. My PC keeps thinking at me because I'm exceeding the Territor and Buffer because data is streaming so fast. I'll close that because I keep getting annoying pings in my ears. So now you can see that data is now being streamed out of your machine. I think we've set four megahertz. However fast the peripheral clock is for the UART number one there. So 115200 board, which is the maximum we set there. So you can see now that your data is reading the ADC into the Ram Buffer, then switching over and instantly sending that data straight back out and backwards and forwards. And everybody see data streaming out in a terminal window. How am I doing for time? I'll try and do the trigger one just to preview that we can slow down before we go for lunch. Everybody okay? Not really. My streaming doesn't work. Your streaming is not working? Yeah, it prints the first UART prints. I left the homework printing. So the homework printing ends in the third character and then nothing happens. So maybe it works and just kills the ham. It depends. I forget because when I did this particular one I think I ended up disabling the Hello World UART line. There should be a line in user code too. User code begin to, there should be something for your that's displaying your Hello World Buffer. Yeah, and I think I commented it out in the end. Probably comment that line out just because it conflicts with the settings that we're trying to basically do it in DMA and we end up getting a conflict. Okay, well that explains. Yeah, because at the moment you've set your Hello World to use non-DMA and we're trying to set it here to use the DMA. So you might have a conflict there. On that one. Yeah, sorry I should have said comment that line out when I was going through the code if you'd done it that way. Yeah, I apologize. Yeah, it's sensible. I just wasn't thinking. Just following Darmily. That's okay, that's alright. You're okay until this section actually because in the previous section it doesn't matter but now when we add the UART configuration obviously it then does conflict. Yeah, I should have mentioned that one, yes. It's because I'm starting from a blank project because obviously I didn't have the homework one open. Yeah, you don't get it. I left my homework one closed there at the bottom there. Yeah, so you'll need to comment that one line out. Right, so I'll try and quickly, we'll hit what I should have stopped at lunchtime. I'll try and quickly go through triggers with you before we actually do stop for lunch because I think this one's quite good for you because it's going to slow down what you're streaming out of your UART, this will. So what we're now going to do is we're going to link a timer as a trigger source or our DMA ADC command. So it'll only do the process. It'll only start the process once the timer event has expired or updated. So everything stays as before. Your link list will load your channel with your DMA config. Nothing will happen at that point until your timer generates its update event. Then it'll trigger our DMA where it'll do the same process, load all the registers or 64 cycles. Then it'll instantly go and change the settings to the UART and transmit everything else. Then it'll reload our ADC again and then sit there and wait till the next trigger event. Rather than continuous data, we're now going to send it out at whatever frequency we set for our timer update event. So if we go back to our CubeMX and we go into our linked list, which is where I currently am, we need to make sure we're editing now node name. So the configuration for the ADC one, which is node name, not node name too. Make sure you highlight the correct one and we're now going to look at the trigger section. So if I reduce all these boxes that I've opened, if I now expand the trigger section inside your node name, I want to set my trigger configuration to be DMA request on rising edge and my trigger selection is going to be timer number 15. So timer 15 trigger output. So that's setting my trigger event in my linked list. Now I've got to go and enable my timer 15. I'll expand timers on the left-hand side. Timer 15, we need to check the box to say that we're using the internal clock and in our timer parameters, we want to set our pre-scaler to 49,999. And as long as we're running at four megahertz, yes, we're at four megahertz, I want to set my counter period to be 79. Trigger output parameters need to be event selection to be update event. So that's a pre-scaler of 49,999, a count period of 79, and trigger event selection to be update event. So this is going to now hopefully when you look in your terminal window in a minute, you should only see a burst of data every second. So it'll read all the ADCs, send everything out over the UART, and then wait one second before it does it again. So it's your way of controlling how frequently your DMAs are doing its read cycles or transmit cycles. And as I say, all this is completely independent of the CPU core. Once the CPU cores set everything up, it goes to sleep, and then this will carry on actioning in the background. So once you've set your timer, we can generate code again. Project. So we know that my code's updated now because I've got timer 15 code here included in my software. Going to here. So we now need to start the timer. So that goes right at the end after we've started the ADC. And again, because we've now regenerated our code, we need to update our link list because we've regenerated, therefore the bug has reappeared again that we corrected because that's not between a user code beginning and a user code end. So again, at line 85 to 89 now on my screen, I need to change my your name to your name. Make sure you only do it on the section that follows the ADC configuration. Down here is correct. 102 to 106 is correct. And now we can rebuild and rerun our project. Debug my TerraTerm again. Bigger for you so you can see. So now when I hit run, you now only see burst of traffic coming through. Once a second, you will see a burst of data coming through now. And remember the data that we're reading from the ADCs, it's not being formatted anyway, which is why it's not appearing as anything sensible on the screen. So there's no data formatting of any sort. And we're just sending whatever that 14-bit reading is and I'm no idea what that converts on the ASCII table to a character. So you're not expecting to see anything sensible in data. It's the fact that you're just seeing burst of data now coming through once a second, which is our timer update event. So hopefully you're all seeing that now. And now we load our ADC parameters into our channel. It sits there and waits until the timer triggers. Once the timer triggers, reads all the ADC, loads in the UART parameters, writes all the UART parameters. The link list will then jump you back to the first node, which then loads in all the ADC parameters again and then it sits and waits again. So you've now hopefully seen the ADC managing by the timer to give you one second burst of data now rather than that mass streaming of data that kept overflowing the TerraTerm buffer.


So welcome to my talk. Can you hear me at the back? Yes, great So I'm talking today about our road to a community space close build environment. My name is SIGI I'm a cloud development architect at SAP customer experience, and I'm sitting Munich Now quickly the agenda I will talk about the technology itself I will also talk about soft stuff like the product aspect Workflows we introduced in our team while we are we are why we have built that and the learnings at the end So just to give you a little bit of context up front I'm a we are in a distributed team Munich and Montreal We are seven team members two of them are working students and we build on that system probably something like a year There are a few details. I will not go over because there are companies specific with networking and security stuff But in general, it's very generic and I think it's probably the same You could experience the same thing if you start you when you just doing doing you when you just tomorrow When I say close build environment We are building a software package which is going out to the customer. So it has to be a tempered proof It's not allowed to Be changed in why we are building it and we have to be it has also has to be audible So we have to know The package which the customer is using on his infrastructure on-premise systems how we build it So there are certain details and you will see this popping up later on We use a managed Q&A this which means for us is it's a small team for some it's probably a big team I have never had a team where I was Fortunate enough to be built on as the ICD system for miss seven people never so I was only the only guy doing that But still we would not be able to do this without a managed Q&A this cluster would be too much work Now to give you an idea of what we do Sandboard is an open source tool which we wrote and it's basically here to orchestrate Jenkins environment we build Jenkins environment with a shift cookbook and their seed shops So you spin up a Jenkins the seed shop is creating all the built Pipelines you want to have it configures security it configures plugins and all this stuff and We provide this to our teams They are roughly 20 to 30 teams and all of those teams have product aspects So they are part of our product which we are building and they get all the Jenkins instances Now someone 2018 there was a little problem Based on some management decisions our old data center was being terminated Basically, it's not my problem But now suddenly we have to move everything over and we did not want to just move stuff over we decided to Think what we actually want to do Which is a great thing to a situation to be in it's basically green field With the experience we had before we knew one thing if you have 20 or 30 teams They do not care too much about a Jenkins They do not care too much about the seed shops updating the Jenkins is and at the end of the day we have to do this So we start we said we want to provide a Jenkins But we also provide instead of a Jenkins we provide the pipeline as a service So my team is only here to provide the Jenkins instance and the second team Which I will not talk today about is actually building the pipeline as a service on our infrastructure So when I say customer, I mean our custom is the other team which is building the pipelines and they have the customers which are actually using the pipelines Our requirement is really simple. We have to support up to 120 build nodes So we can pick up to 480 virtual CPUs and 1.6 terabyte of RAM It will increase this year to something like double It has to be cost-effective big company. You don't have budget unlimited So out of scaling was for us must we have to have auto scaling We actually did not out of scale before if you own your own data center There are 100 machines just running. It was never a problem before It has to be secure Maintainable because we have to have to have the work right it has to be reliable and At the end it has to be have a customer facing your eyes the customer facing your eyes a real problem If you look at tecton and other modern stuff You have to build a lot of stuff around it so customers can actually use it and that's the reason why we still use Jenkins we Have Jenkins and we still use it because we looked around a lot of our current options we have and Jenkins provides with the job DSL and a lot of stuff around like unit tests graphs access management It provides so much stuff that we cannot trust my greater way from it It's planned in the future to have a look on what's going on how the Landscape is changing but it's changing quite a lot So if we need is to the rescue We had already experienced with me so as you need is 1.6 Which is not that old, but it's there a huge difference between 1.6 and 1.12 13 or 15 or 17 right now. We have experience with chef and Ansible It is does support auto scaling on our platform which we chose. It's Google you can probably get auto scaling on all the other cloud providers as well It handles certificates with this which is unfortunate our team also a little problem Sometimes they just expire and the certificates are far of our internal services. We do are not just use let's encrypt Cannot just gets used to let's encrypt For us, it's more secure than VMs You might know the one or the other snowflake VM sitting around somewhere not have been touched for a year or two You know the same problem And people need is you have a note. It's super lightweight. There's nothing on it. It's auto update You don't care about your underlying operating system. You just assume that it's there and it's there It's personal opinion the last one is probably the future so it is also a change in mindset We switch from imperative to declare truth thing instead of hacking and creating your virtual machine installing software configuring it Now we have to describe it before and then someone else takes care of creating that infrastructure for you This is huge because you cannot start in our environment with something you tell you start don't start with doing you start by writing So everything is basically already Infrastructure's code and when you ever have the problem that something is not running and you just delete a whole namespace with your complete Monitoring setup and you press some swing button and five minutes later the whole set is back there again again back You will love it. It also includes all the dashboards and everything we have it's based on code and You don't need to back up anything the only thing you actually back up The artifacts and you put them on some cloud storage and then magic happens and they take care of your backups Quick jump into communities. It's still quite young. So in July 2015. We had a version 1.1 First Cubecon was only no family 2015 Currently we are already at 1.17 and it has quarter releases What does mean what is what it means for you and if you work with it daily And this is one something we have not troubles with but it's something we have we are very aware of This ecosystem is really young Everything changes you have your open box because you just Run into them and you see that they are getting fixed and two months later new releases out and new features are popping up and Tools you're using now have suddenly business enterprise features because a lot of people started using them and they requested those features I was a Java developer for probably 10 years or something like this Java is great and really stable, but I've never had an ecosystem changing so quick You have to keep up you have to put the effort in don't run an old Q and E this cluster If you still do and you don't have time for it You are creating a zombie machine and it will be a horrible migration mine Nightmare later on I can't guarantee you that already now We are we saw the greenfields. We have Q and E this so instead of saying we are just a Info team and we are just throwing something out and Customers come to us and say hey, can you do this? Can you do this? We said no We created we we see is our product is a product and product means It's focus customer focus. We don't just throw our customers something on We tell we think on how they can log in can use our tools, which we are they can use The documentation is a must it has to be clear and understandable If it's not clear and understandable someone created a ticket and we have to have to do something and I personally don't like that very much So I'm trying to give everyone the chance to do it by themselves But in a way that they like it, right? It's really an effort that they like your infrastructure There was a team and a friend of mine told them told me that Work colleague came to him and said hey, I really love what you are doing. I want to have something very similar You also define our boundaries what is respond where the what has the customer to do and what is our responsibility? So if the customer comes to us and says We need this feature We will like look at our roadmap and check if it fits in our roadmap Of course the customer you're building it for our customers But nonetheless, we don't stop our work just to provide something if we have to do our stuff Maintainability quality and security we are enforcing security by default So we are responsible for security. It's not a discussion. I have met with a customer I will come to this later on and we also have a maintenance window defined already So our customers know if you're using that service, there will be a maintenance window It's not that we have to use it, but it's enforced. It's our responsibility That's actually the entry point for our customer. It's a wiki page It's very defined up there weekly maintenance window all the documentation and links over there It's pop internally public all available everything is a publicly internally available everyone is Happily to is able to browse it It's very transparent and we give our customer this link wiki page and they have to try to get To understand our product We are still trying to optimize it. It's a constant work in progress Now it's a really beginning of what we when we started developing It's a little bit awkward perhaps, but stay with me We one of the first things we build was a nameswis inspector This actually a tool which runs in our community this class and we delete everything Which is not white listed so if you create a namespace and it's called I want to test something One hour later. It's gone We do the same thing with in a default namespace if you try to do something in the default namespace It's gone if you want to test something you can use a dev or test dev is deleted after one week test is deleted after two weeks This creates a general idea that is you as a developer you don't start with playing around You start by coding something because it could happen that it's deleted in a week because it's it's a weekend One week is not that long and it's gone and I think that saved us a lot of headaches where stuff is coming from because they can't So we we can't we were actually for enforce it to us So the second thing what which was really important for us was the insight into our Cluster we started with that out probably the second day after the namespace inspector We decide on Prometheus operator. It's a Beta project and it basically provides you with the whole setup for Prometheus alert manager Kavana It has already dashboards in it. It has predefined alerts So they guy the guys actually know what they are doing I think there are a lot of cubanese operators which only operate cubanese clusters so it will pop up if You notice just getting full you have already the alert defined. You don't have to do anything. It just works Last year on the fourth stem I became aware of Grafana Loki We do not provide access to our cluster to our customers So they want to see the log files, right and Kofana Loki is basically this it's a lightweight log aggregation tool It's no longer an alpha. We started using it in alpha with the risk Or the awareness of it but having an elastic search cluster running in your your Kvinilis cluster It's huge lot of effort. I wanted to try to avoid this And it basically does exactly what we want. It provides check in slaves and master log files for our customers And we have log file dashboards for that as well Now that's a dashboard how it looks for the Prometheus operator in this case. It's just a node. It tells you IOPS network IO memory CPUs it We have our own dashboard and this quite probably the interesting thing here It's already you can see already the outer scaling mechanism. So we scale up a hundred nodes Two hours later. It's gone If you look at this You can see on the right left side. It's we have also introduced a cost model system So we actually know roughly how much money we burn It is really important that we have an eye on it because if you do not out of scale our bill would be roughly something like 30 or 50,000 euros a month and right now it's probably more like 5 to 10,000 euros There's There are a few bonuses if you're in a big company you can Leverage something more so it's getting cheaper, but nonetheless. It's a big problem, right? Everyone has a cost problem And you can of course not do this in your own data center if you scale up and down something you own already It does not make much sense so Now we need a tool for our customer to create the Jenkins So we are talking about the customer which are creating the pipeline. So they are very technical But they only want to have a Jenkins and it should work and should be safe and that's it So we are using QBAPs. It provides Ham applications, it's a simple interface. It only does exactly that it provides you a customer to the way of spinning up a machine or application and The values jammel file is basically the contract we have with our customers We don't tell our customers you have to know how QBAP works, but unfortunately for you, you have to know how jammel works So how does it look like? It's simple interface great project very interactive It creates new it gets new features, so it's actively developed on but it's really young So you you had a problem with the namespace on top. It showed all the namespaces. Sometimes it didn't show didn't show any namespaces So you are constantly have new features on it This is now a jammel file you actually have we are trying to have all the important things Custom I really want to change on top and we try to document it But nonetheless a lot of stuff is down there So basically what they can define or what they do define is a seed shop So they spin up their own Jenkins tell them roughly how much memory and CPUs that they want Which of course means if they create a big Jenkins lay Jenkins masters 24 gig of RAM If was easy pews, they will get it, but it will cost them a few hundred dollars per month versus small Jenkinses For testing this really great in a cubanitas It's really simple to just give your Jenkins to VC pews and four gigabyte of RAM if you are testing something Or you can give them a big one Now you have a lot of stuff configured already in your git repose and now you need to have your Jenkins and all the other stuff You all the monitoring all the cubanitas applications hand charts in your cubanitas cluster How do you do this? I personally would highly recommend our city. It's a great open source project It does nothing else than making sure that whatever you have defined us as implication is getting installed From the git repository into your cubanitas cluster. So right now you see an overview of all the applications We have you also configure our ingress controller to git. So the git ingress controller is in git Obviously to make sure that the ingress control how he's gone is configured in git is configured in communities So it looks if if you go into details, you can see the sync status. You see all the parts the service accounts Pots everything and the greatest thing actually on our city, which I really love is Our city provides your own custom research definition from our city and you basically define an our city application As we am it so you what you can do and what you should do and they actually build it in a way that you can Do it like this you configure our city through our city You only have to put our city once on your community this cluster then our city sees. Oh, there's an application called our city I should maintain it and check it out and suddenly our city manages itself, which is really great The names with inspector looks very similar Not a lot of stuff to do the target revision hedge where the radius come are coming from We inject in security secrets from vault Yeah Now how does a set up now look like it's not very big. It's not very complicated, but already the base layer is 10 virtual machines so learning like 10 40 or 50 vcp use it's what we already need We have permit there is Grafana for our customers and for our setup We have we provide shankin's is but we also need to have a Jenkins to build images and everything And we provide the dex is an alteration. I Open ID providers who we basically tell if you have UI like permit us to look creates an UI and it's unprotected That's not something you can do in our company So we put a dex up front or connect the dex with our adult server So you can use your company credentials to log into dex and then you have access from users Or alert manager what we also have is an artifact for caching reasons Now in our organization you saw the environment right it's not that much, but already we have over 40 git repository's It's huge. We call we have our product is one git organization And we have prefixes for all the repository so car QNEDIS means it's an application running in QNEDIS It's a hamchat or combination of hamchats chop is a Jenkins seed shops and the stuff we configure We also use our own dog food We have our own seed shop which creates all the Jenkins shops for building the images doing security scanning and all this stuff in Terraform Use Terraform to create the Jenkins cluster and then we never talk about Terraform again Everywhere repository has to have a weekly and For the hamchats we have to how to maintain I will come to this in a second and we have also ops manuals We have to have on call which basically means a service is not running you get a call You have to do to have to fix something so ops manual only tells you what you can do in case of monitoring There are a few ideas, but there's also written that You can delete this namespace and we think is for our city not a problem And I think right now we could delete the whole cluster and click on our seeding would just work So I think most of it. Yeah, sure We have our documented documentation in git. It's closer to the code It's test text-based formatting, which is much easier than in word Moving things around Works great, and you can use skitter pages if you want to which is really lovely Now we have a lot of ham charts. We have ham charts for our Jenkins. We use upstream ham charts What do we do? We have literally a documentation on where this upstream ham charts are coming from and you have to maintain it So you have to put it down and you have to change something We are not finished with the migration, but we have moved to customize so customize is Patching our ham charts we have from upstream And we need to do this for certain things we have priority classes on every service we use So we have to have Priority class definition now ham charts if it's not available We can ask nicely if they added to the ham chart But we nonetheless we have to edit and all the images Which are defined in the ham charts We are not pulling them from the internet if you do autoscaling you cannot do this because a node doesn't exist for Too long and you don't want to cash it and you have to cash you do wouldn't I like to cash it right so in Google It's gcr you want to have everything in your gcr registry to basically Yeah Customize it was unclear when we started what we want to use because there's a lot of options out there And it's still not very clear him three just released But customizes part of cubes detail since 1.14 so this is a no brainer anymore and our city trust supports it as well Keeping it stable now We have a lot of software running trust to provide a few changes what we introduced this as a e we name it as a e It's probably not that as a you what you know from Google talks and they are plenty Google talks about this topic It's a stable process. We know we have this system in our team So if I go on holiday and know I know there's certain things are happening It's a weekly rotation So everyone of them our team has to do it and everyone in our team has the time and effort to actually have a look at Every part of our infrastructure and because we have so many different components just from the Jenkins You actually need this time And it gives your team or the rest of the team the focus on working on on the stuff They are working on without getting interrupted all the time How does it look it's a wiki page with a timetable and then there's a checklist nothing magic works great though Now we have we are upgrading and maintaining a lot It's like you can assume one person per week something like 10 to 30 percent of the time what we actually also build was a Gavanna dashboard which shows us all the upstream chart versions and what we are running to see how we are progressing and sometimes Or not sometimes actually quite often there are major versions and we are upgrading again and changing again Which is really nice. It feels interesting if you read a new release notes from August you say, oh, this is a really cool feature I was waiting for this. It's super helpful. You love it But you cannot I would not build something and just keep it there. It will not it will right away The operations manuals we have operations manuals in the service page If you woke wake up in the three in the morning, you go to this wiki page Click through your ops manuals the graph on our dashboard and that's it Now we are really close to the findings already Well, now we have a little problem right our customers are running current shenkins is and now they are not updating them I'm not sure who knows that a feeling or not But we have this feeling quite often So we decided this time to do something else and I think it's a good practice and I we saw it in a few other projects How they do that? We build every night a Jenkins image So there's a Jenkins master and this image already contains all the plugins So we create we we pull down the image with the Jenkins run the Jenkins the Jenkins Jenkins install the plugins The image is done. We take it with latest and with the current daytime and You as a customer or and we deleted all the images in three months So basically there are only images available which are it maximum three months old in theory We could just do these certain versions as well if High vulnerability would be popping up, but it's not implemented yet Now we tell our customer you have two choices you choose latest Or you use a certain daytime version and we tell you the real limitation of that your image is only available for three months and Then customers go figure and figure out how they want to maintain this process To make sure that the Jenkins is not magically running for longer than three months. We just kill it after three months We also do have network policies Policy policies everything in place So if you just spin up a Jenkins and a customer would come here and say nothing to us They cannot connect to the internet and there are a lot of stuff just missing But for us it's more like the right list approach They have to come to us say we want to connect to this system and we tell them no Or if they really have a good reason why they want to connect to the system We will write listed for them and then it's our responsibility at this connection works We have to on big corporate system. We actually have to have a request firewall Rules and stuff like this and we have to put connection stuff in our security concept Our learnings Stateful service on Kubernetes is hard Kubernetes is not has not been built for stateful services if someone tells you this works really well. I don't think They explain they are lying. I think For Jenkins, there's no hey higher will be set up available. I think there's some cloud bees magic plug-in Which doesn't work or work and you just can't use it For us, it's a big problem way because our build this build runs for 6,000 hundred machines So if one Jenkins breaks or five Jenkins breaks, it has to be really reliable Otherwise, I think some one because something like 30 or 50 dollars So you don't want to skip it if you have much smaller builds if you build a little bit resilient It's it's not a problem at all Make sure that you have port priority So a Jenkins master should not got thrown out because a Jenkins slave is suddenly more important than a Jenkins master doesn't work out and System ports will and should always win and they will they will do this So if you think all your Jenkins everything is fine tune. It's now stable. So I'm your Jenkins master is going. Why is my Jenkins master gone?  I will get get in each in detail about this in a second Don't think about this cluster is now set up Assume that you can't get rid of this cluster again So what we do is we move our Jenkins artifacts to blob storage blob storage is independent from our cluster Everything which is persistent needs to be in git or in some blob storage Otherwise, it's not persistent and we don't assume that it exists Sometimes you have our money trying for example the Prometheus only saves it for 24 hours But our long-term storage with Thanos at the back end put put pushes it to GCS. There's persistent discs Which we use but it's not our We don't expect that they are getting not they can be deleted and should be allowed to be deleted For the blob storage for example, we actually build a small browser Which is secured to add up and then you as a customer instead of just going to say you can use the Jenkins artifact browser But for any long-term thing there is this GCS tool and you go on this and you can just use it and it will always be there Although a big issue if you're building long runs, you have to be ready for maintenance maintenance is coming up In our case Google is forcing us to do this if there's a critical security issue What do you do with the critical security issue? You fix it, right? Sometimes most the time you fix it right in this case Google just enforced it for us Which is nice. I assume they also have the security by default We are custom they tell us what to do But they also enforce a maintenance window, which is great for me because I go to manager say hey She case enforcing maintenance window. We want to have a maintenance window. This is a maintenance window No discussions need it We tried at the beginning to make our services as resilient as possible tell them QB need is you're not allowed to kill it and That doesn't work in our specific case Google just deletes it in after an hour. It looks at it for an hour and say Okay, let's go instead of now and for trying to make it as hard for QB need is itself to manage itself We are trying it that different the way We are making sure that if it's getting killed and it's getting killed more often than you wish It's more often care getting killed than you wish for we make sure that is quickly back Now the auto scaling is a big problem if you do auto scaling You you run with 10 or 50 nodes. I think everything is fine Then you do a low test with 100 nodes and suddenly your monitoring is Not happy anymore because it's just 100 notes more than it was five minutes ago You have to do low tests look at it. It has to work And there's a big problem not a big problem, but you have to be aware of it Calico for example has a wonderful demon which actually analyzes how many nodes are running and I have F After every 10 nodes It reschedules it nodes and wants to have more resources And if you remember what I said with the system services There's a system service and it will kill your Jenkins and it instead of using 250 millisecond CPU It's only uses one millisecond CPU one CPU and Then suddenly you have three of them instead of two and if you try to keep your class Highly utilized because you think our cost cost cost saving Cost saving is important, but 100 bucks per month more just to be sure that your system service or not killing your Jenkins master is it's fine It's slow To autoscale something it's great if you have Jenkins jobs, which are running for an hour No one cares, but a few minutes for something which just starts is bad So what we do now while you're using auto scaling right for cost resources, but now we over provision We have a pot defined which is you is a sec is requesting the same amount of resources the Jenkins slave With basically minus one priority.  I just wrote out and then you have a smaller latency again Images are not cached Your nodes are not running for that long. They might run for five hours. They run for an hour. They might run For a day, but they're gone If you already autoscale to a maximum number of what you defined I mean, we have to have we have to be sure that we don't autoscale to 1,000 because that's cost a lot of money and Yeah, so we put it something like from zero to 10 for our default setup Which is we are not out can cancels are running and suddenly you're out. We're upgrading Don't know longer work because you already use ten nodes and If you need to upgrade and you are replacing one or by the other you have to have one additional node so you also don't Run your Kubernetes cluster Fully full you have always have to have a little bit of buffer and pot preemption starts and you have weird Behavior if you run full capacity and we are trying to get we're not run on this if you do this It's effort You thought we put everything on one cluster in one node pool Happily enough or good enough for us It's super easy to define multiple node pools.  I control it with priority classes and I run it to the Fullest to save costs and have a nice it doesn't work at least it doesn't work for us The project structure for Kubernetes cluster of one cluster and you have a tons of service accounts images everything Start to annotate them up front so you actually know why this image is in that one folder and who actually uses that Let's get really messy If you think you have one Jenkins slave and it's just a small Jenkins slave and you are getting Away with one virtual CPU and two gigabyte of RAM think again Kubernetes uses and needs certain amount of resources As the bigger nodes are getting bigger the set up itself gets more efficient We are running now between four and eight core machines We do not go over under four cores and you probably say more like on the eight or six cores per node To have a problem which of course if you do out of scaling your budget is a little bit smaller than ours It means the difference between 50 or 200 bucks per month just because you cannot be that fine granular I Expected to be very fine granular, but we throw this over very quickly So, thank you very much. I have a little bit of time for questions and answers if your feedback send me email If you want to talk to about me this infrastructure more detail that on Big company, it's really hard to outsource it. It's easy to inner source it. It's hard to


Let's begin here in Matthew chapter 18 at verse 1. I'll read to verse 5. We'll get into our study. At that time, the disciples came to Jesus saying, who then is greatest in the kingdom of heaven?  Therefore, whoever humbles himself as this little child is the greatest in the kingdom of heaven. And whoever receives one little child like this, in my name, receives me. And so as we begin chapter 18, we need to remember that Jesus had just instructed, at least in Matthew's account, last time we were together, we'll put it that way. Jesus had just instructed Peter to go to the Sea of Galilee in order that he might catch a fish and pull a coin from the mouth of the fish in order that they might be able to pay what is called the temple maintenance tax. And there was enough money in the mouth of that fish that would pay not only for the apostle Peter but also for Jesus. And as I mentioned to you, this particular tax called the temple tax was received to keep the temple in proper condition and also defrayed the cost of temple service supplies as well as incidentals. So as Peter is leaving and Peter has left to go and pay this tax, Jesus is now having a conversation with the other men. So Matthew tells us in verse one at that time the disciples came to Jesus saying, who then is greatest in the kingdom of heaven? So Peter is not there at the moment. He's out there paying the temple maintenance tax. The other disciples have approached Christ and they're asking a question. And as this is taking place, we see that Jesus is about to teach him a lesson concerning greatness in the kingdom of God. Now by combining Luke and Mark, we once again will receive greater insight into what is happening here because in Mark chapter nine, verses 33 and 34, it says that he came to Capernaum and when he was in the house, he asked them, what was it you disputed among yourselves on the road? But they kept silent. For on the road, they had disputed among themselves who would be the greatest? In Luke 9.46 it reads, a dispute arose among them as to which of them would be the greatest? So they were having an argument amongst themselves and the argument related to who is the greatest in Messiah's kingdom. Now it is possible that the enemy, Satan, it is possible that Satan provoked this argument. It's interesting there are other ancient Bible translations that seem to at least infer that when you look at the ancient Syriac and ancient Arabic translations, they translate this verse by saying, a thought entered into them. And so there are commentators who say it may be that the enemy prompted them to argue about who was the most important because that kind of concern would be useful to destroy the unity that was needed to fulfill their calling. You see, one of the things that we need to understand, and let me lay this as a foundation as we look at the key to greatness in the kingdom, one of the things that we need to understand is that we are always most effective when we work together. And therefore unity has to be valued and unity has to be safeguarded. Paul, when he was writing to the Ephesians in chapter four, verse three said that the church was to be endeavoring to keep the unity of the spirit in the bond of peace. And so if you want to accomplish a task, it is always better to be united as you do so. We know that Satan is the author of division. He introduced division by provoking the angels, a portion of the angels to fall with him. We also know that he divided the very first marriage. He is the author of division. And so Jesus in Matthew 12, 25 said, every kingdom divided against itself is brought to desolation and every city or house divided against itself will not stand. So from the beginning Satan has authored division. So these ancient translations of the Bible are inferring at least that it could have been a provocation from the enemy to divide the unity of the apostles and thus undermine the work that Jesus intended to do through them. Remember how that the apostle Peter had rebuked Jesus when Jesus began to speak concerning his impending death and that Jesus in response spoke directly to the influence that had provoked Peter to say such a thing. Remember how Jesus had said in Matthew 16, 23, get thee behind me Satan, thou art an offense to me. And so we know that Satan intends to keep Jesus from doing the work that Jesus would do. Not only would he come against Christ himself through provoking an apostle in influencing an apostle to try and undermine that work but he would actually continue to do that because he knows that the Lord Jesus Christ in order to put an end to his reign if you will to disarm him that in order to make that function perhaps not possible at least in his way of thinking. Well the best thing to do is just to undermine the unity of the apostles and provoke them to be influenced to if not keep Jesus from going to the cross then to divide the unity that they have. We know that Satan provokes people to evil. In Luke 13 verse two it says supper being ended, the devil having already put it into the heart of Judas the Scarlet Simonson to betray him. We know according to John 13 too that Satan provoked Judas and actually influenced him in the direction of betraying Christ so we know that he has that ability to do that and so what is happening here is Jesus's men are involved in an argument and I want you to see what this argument is. They're involved in an argument concerning who is the greatest in the kingdom of heaven. Now when it says in verse one who then is greatest, that word greatest speaks of the most important. Who is the preeminent one? Who is the stronger one? So what is your argument about? What are they disputing? Who is the greatest in the kingdom of Messiah? So as you look at this, and I want to develop this a little bit further, as you look at this they're picturing an earthly kingdom and they're arguing about who will be in the position of greatness. They're not speaking about spiritual greatness by the way. They're not arguing about who's going to be the most successful preacher of this gospel. They're not arguing concerning who is the most loving and they're not arguing concerning who is the most humble. Can you imagine an argument between people as to which one is the most humble? No, they're arguing about who is going to be the greatest. Now what does this reveal to us? Well it reveals to us insensitivity. When Jesus had made the announcement of his impending death, Mark had told us in chapter nine verse 32 that they didn't understand what he meant and they were afraid to ask him about it. So they should have been discussing this amongst themselves and not the question of who is the greatest. Again, this reveals their fleshly insensitivity, their selfish ambition, their lack of love for Jesus Christ. They want places of honor. They want places of authority in the kingdom of God. And it seems that what is provoking them at this time would be what we call in scripture selfish ambition. The Bible speaks concerning selfish ambition. They want to establish, if you will, a pecking order of greatness. They want to model this system after the failed system of the world, our fallen world. Jesus speaks about that in Matthew 20 verses 25 through 28 where Matthew said that Jesus called them to himself and said, you know that the rulers of the Gentiles lorded over them, those who are great exercise authority over them, yet it shall not be so among you, but whoever desires to become great among you, let them be your servant. Whoever desires to be first among you, let them be your slave. Just as a son of man did not come to be served, but to serve to give his life a ransom for many. When Paul was writing concerning this attitude in Philippians two verses three and four, he said, do nothing out of selfish ambition or vain conceit, but in humility, consider others better than yourselves. Each of you should look not only to your own interest, but also to the interests of others. And so the disciples need to learn a basic lesson that modern disciples need to learn also, the lesson of humble service to God and other people. In Mark 935, it says he, speaking of Jesus, sat down, called the 12, said to them, if any man desired to be first, the same shall be last of all and servant of all. In Matthew 10, 24 and 25, the disciples not above his master, nor the servant above his Lord, it's enough for the disciple that he be as his master and the servant as his Lord. And so if you took notes, you might want to note that the key to greatness in God's kingdom is possessing the heart of a servant. D'Almoudi, great evangelist of another time, once said the measure of a man is not how many servants he has, but how many men he serves. And so Jesus embodied that and his men needed to learn that. Ambition for greatness is to be replaced by humility. Having people serve you because you're so important needs to be replaced by your service to somebody else. Like he said in Luke 22, 27, who is greater, the one who is at the table or the one who serves? Is it not the one who's at the table? But Jesus went on to say, but I am among you as one who serves. So the men needed to learn to die to selfish ambition and the need for attention, the need for attention will destroy your walk with the Lord. It'll undermine your effectiveness. If you become the person, if I become that person who constantly needs to be known, to be recognized, I'll undermine the effectiveness of ministry. If you have a heart and a calling to serve the Lord, which we all do in one capacity and another, obviously. Let's say you want to plant a church or do a work, some ministry that's gonna place you in a position of leadership where people will look to you as a spiritual mentor or look to you as an example. Keep that in mind because it's very important. The desire to be the center of other people's attention is not a good desire to have. The desire to be known is really not a great desire to have. There's only one person that you ought to ever worry about knowing you and that would be Jesus himself. I want to hear him say, well done, my good and my faithful servant. The Bible tells me that the shepherd goes before us and he calls his sheep by name. So for me, when I hear my name, I'd like to hear it on his lips. And rather than doing ministry to be seen by men, to be known by men, and by the way, in this era of the church, at this time in these last days, there seems to be a tremendous, tremendous movement towards giving attention to men and forgetting who Jesus is. We need to put our eyes back on the Lord. Well, on the one hand, it's great to have. Amen, you can clap. I think that's a good thing. It is a good thing because the need for attention will destroy humble service. You'll get to the point where you're saying, you know, they don't know me. You know, it's kind of funny. You know, when I first went on the radio in different places, you know, I didn't know what it would feel like to go somewhere and to do ministry in Holland because they didn't know who I was and I was in other states. And so they didn't know who I was and I can still remember, and this is something that it was, we're talking on 20-some years ago now or more, but I can still remember going to places like Chicago or do ministry in New York and we were on the radio. And nobody, we weren't posting my picture on or anything like that. You didn't have that. You didn't have the internet doing those kinds of things at that time. And so people were not, you know, going onto web pages and all of that. So we were just on the radio and they'd say David Rosales is gonna be in Chicago this Friday inviting his radio listeners to come so we can have a radio rally. He used to call them radio rallies. And I would walk in and nobody would know who I was because nobody had seen me before. And so I'd stand next to people and they'd be talking and I wouldn't speak. I just would kind of just stand there and it was kind of fun to be invisible like that. And I can still remember, I would go up behind the platform. They said, let's welcome our guest today, David Rosales. And I'd walk up and the people would just be staring at me and I'd say, all right, you're surprised, right?  And they'd nod their heads. I said, because you know my last name, you expected something different, didn't you? And I said, you thought, I really said this. I probably shouldn't tell you, but I'm in a good mood, I will.  I'd say, you got me confused with Rawl. And they'd all laugh. You know, Rawl's a very dear friend of mine and I would play with him like, I saw him just yesterday, he's a very dear friend of mine. But I would tease like that.  Not enough pictures of me. Not enough pamphlets about me. Because from the very beginning, the Lord was teaching us, it's not about us, is it? It's about him. And if people can see him, that's what real ministry is. Many of you have been here for a while, maybe you're here for the first time, but that's what that verse behind me is intended to communicate. We would see Jesus. That's a scripture out of John chapter 12. When the Greeks approached and asked, they were saying, we would see Jesus. And so for me, that's real ministry. That's what the Lord is telling us. And so what we need to do is we need to be careful that we don't get enamored with the need for attention so that people know who you are.  Well, I'm serving children's ministry or I'm out there in the parking lot, I'm a nusher, I'm working in the cafe and I'm serving and nobody's saying thank you. Well, you know what? You can do it yourself. Well, when you have that attitude, that's not service now, is it? And so the Lord would say, listen, you guys have it backwards. You don't understand it.  The server or the person who's ordering the food? Well, if you ask the owner of the restaurant, he'll tell you. He'll say customers come first because they're providing the income that pays that server.  As a server, I'm the one, and therefore use me, Jesus would say, as your example, you want to be greatest in the kingdom? Greatest in the kingdom is a servant of all. When I was in high school, yes, they had high schools back then. When I was in high school, one of the things we had to do in order to pass our PE class or a gym class was we had to tread water for 30 minutes.  I can, you drop me in the ocean, I can survive five minutes, I can, but that's about it. I'm not a swimmer, never have been. And so I was next to the edge, you know, so if I needed to reach up and grab the lip of the pool, save myself from drowning, that was possible. So I was there by the edge and we were supposed to tread water for, I think 30 minutes. And as I was there next to the edge, some of my friends and I were gathered around, we're talking, treading water, trying to get that credit for that class and all. And so as that was happening, there was a freshman. I was either a junior or perhaps a senior at this time. But there was a freshman who got on the high dive and was standing on the lip of the board. And as he was standing on the lip of the board with his toes kind of curled over it, he began to yell. He yelled out, coach, look at me, I'm going to jump.  The coach wouldn't pay any attention to him. The coach was doing something over on the other side and he kept yelling and he did it several times.  We all started yelling, coach, look at me. And we started waving our hands, coach, look at me. You know, and we're laughing because boys are mean, we're evil. You women know that. So we're yelling, coach, look at me. I'm going to jump, I'm going to jump. You know, how mean. But anyway, we did that. And then finally the coach turns and looks in his direction and this young boy jumps off and jumps into the water and we all clap for him. But do you want to know something? The Lord taught me a lesson. He teaches me lessons in the weirdest ways. Forgive me, but he does. He said, don't you dare be that boy on the lip of a diving board yelling, Jesus, look at me. Or church, look at me, look at me. Because you're not supposed to be looking in that way at me. You're supposed to be looking to him. And that's what Jesus is teaching his men. And by the way, these are the men who are going to take that message to a world. They had better, better understand that. Listen, if you do ministry for Christ to be seen by men from men, you get the reward. But when you do things secretly, God rewards openly. Your heart has to be right with God. That is so basic. And perhaps some of you right now, who cares, move on to the next verse. But there are others right now who need to hear this because you may be getting weary in doing good. Wondering, when is someone going to pay attention to me? Let me remind you, someone is paying attention to you, the Lord, and he rewards openly. He does, don't forget that. It's very, very important to know that. And so Jesus embodied servanthood. And so how do you illustrate that?  Therefore, whoever humbles himself as this little child is the greatest in the kingdom of heaven.   And he says this, and it's very important as Jesus speaks, because here's this child in his arms and all, as he's there calling this child to himself. And he says to them, unless you are converted and become as little children, he's teaching a lesson. I want you to notice what he says, unless you're converted, you will by no means enter. The word converted means to change. It speaks of really turning away or turning back. A person must turn away from their worldly ambition and they need to humble themselves to enter into God's kingdom. Now, the fact that a person must enter assumes that he is first born outside. So before he is saved, he is not a citizen of the kingdom. Now the Bible makes it clear that before we're saved, we're actually, instead of friends of God, the Bible makes it clear that we're actually God's enemies. In Romans 8, 7 and 8, Paul said the sinful mind is hostile to God. It does not submit to God's law, nor can it do so. Those controlled by the sinful nature cannot please God. The Bible makes it very clear that I actually am in hostile opposition to the Lord. I have a nature, a sin nature that opposes God. And as such, I am only fit for judgment. So Jesus is saying, listen, you need to be converted. You need to turn around. Converted speaks of the other half of repentance. Repentance speaks of changing your mind. Conversion reveals the change. So when I repent, metanoia, when I repent, it's a change of mind towards the things that I thought were proper to begin to accept the things that God has declared. So when I repent, I change my mind, I'm converted, I change my behavior. My life changes. There is a different way that I live. And so Jesus here is speaking concerning being born again. The aspects of being regenerated or born again to become a Christian. Because we are not automatically Christians. We are not born, born again. We're born fleshly. And that's why Jesus said, unless a man is born again, he cannot see nor can he enter into the kingdom of God. There needs to be what is called a conversion experience. There's a transformed life. There are many who acclaim to be Christians whose lives don't speak of any Christianity whatsoever. And so the Lord would say, listen, if you really have changed your mind, you're also gonna change the way that you live. That comes, by the way, through the Spirit, by the power of the Spirit and by God's word. Now, in Colossians chapter three verses 12 through 15, Paul said it like this. He said, since God chose you to be the holy people whom he loves, you must clothe yourselves with tenderhearted mercy, kindness, humility, gentleness, and patience. You must make allowance for each other's faults and forgive the person who offends you. Remember, the Lord forgave you, so you must forgive others. And the most important piece of clothing you must wear is love. Love is what binds us all together in perfect harmony and let the peace that comes from Christ rule in your hearts for as members of one body, you are called to live in peace and always be thankful. Entrance into the kingdom demands the qualities of a child. There's no other way to enter the kingdom of God but as a child. That's what he says, notice again in verse three, unless you are converted and become as little children, the key to entering in, humbling yourself. To be converted requires that I be willing to be like a child by humbling myself. Like it says in James chapter four, verse six, God resists the proud, but he gives grace to the humble. Small child, children have interesting qualities that we can learn from, just look at them. I have been blessed by the Lord to have eight grandchildren, six of them are little girls and two of them are little boys.  And so the door opens and here comes my daughter Anna and she's with her daughter Zoe, Zoe's two years old. And Zoe's a little ball of energy. And so Marie loves her babies and here's one of them and so Marie is in the kitchen and she comes around a corner and stands waiting for Zoe to come to her and she's standing there going, come to Grammy, baby, come to Grammy and you hear Zoe say, no, I want Papa. And I felt so bad for Marie, I really, I want Papa. No, come and give Grammy a kiss. No, I want Papa. Long story made short, she comes this high and all of that and it's time to eat and then she's in the kitchen and Marie is very persistent, very persistent, kiss, Grammy, Grammy wants you to, and I say, honey, don't try so hard. That's why it took so long for me to ask you to marry me. Stop trying, stop trying. Just let it be, let it flow. But so what happens? Well, Zoe decides to let grandma know she didn't want to kiss her, so she hits her, hits her Grammy and then Anna sees that. You don't hit my mama, Anna says to Zoe and all of a sudden I hear Zoe yelling out, Papa, Papa, save me from the wicked witch of the West also known as Anna and she comes running. She comes running from the kitchen to where I am, climbs on top of the couch and grabs hold of me, puts her arms around my neck. Well, Anna is saying he can't protect you from me, he can't protect you from me. And I'm holding on to her and Anna's trying to take her out of my arms and I'm not letting go. Oh really, I had to let her go. But one of the things the Lord reminded me of is this, he has a little child, what do they do? They run to the one that they think will protect and keep them, right? That's what I do with the Lord. Who do I run to? If I run to man, can man protect me? No, I thank God for those who are equipped to do so in a variety of ways, civilly and all, but ultimately can a man protect me spiritually? No, only God can, only God can. And so one of the things that I see about it, little child is the little child runs to the one who protects and holds them, cares for them. And there are other things obviously, when you look at kids, you know, I learned this as I raised along with Marie, when we raised our babies, we learned for example that children are very trusting, they're very trusting. If you make a promise to your child, they trust that you're gonna keep that promise. Isn't that true, mom and dad? I mean, if you said to them, listen, you clean your room, do this and that, we got a surprise for you, we're gonna take you to Disneyland. Did they ever forget a promise? They never forget a promise. They do forget to clean the room, but they don't forget that you said you take them to Disneyland, they don't forget that. Another thing is they're very forgiving.  If you've had a bad day and you're short with them and you may not speak to them in the way that you would prefer and it makes them sad. If you walk up and say to them, I am sorry, which incidentally parents ought to do, I'm sorry, I should not have been upset. You know the first thing that they will say to you is it's okay? First thing, it's okay, daddy, it's okay. They're very, very forgiving. They're very loving, they're very simple. They don't require long answers, do they? The kid walks up to the daddy and daddy, I wanna know where I come from and daddy says, what? I wanna know where I come from. Where's your mom? She's not here right now, daddy. Can you tell me where I come from?  Starts giving the birds and the bees and afterwards, after they're all through, the father's been sweating bullets. The kid says, listen, I already learned that at school. What I meant was, where was I born? Was I born in Ontario? Was I born in Chino? You know, they need just simple answers. I've discovered that also. They're very innocent in many ways. They're very dependent. They will rely on their caretaker, they're dependent on that caretaker. These are all things that we need to remember and also, a child does not resist receiving a gift. How many of you have kids who on Christmas will say, oh, you've given me too many gifts? It's their birthday. And I'm sorry, you gave me four gifts, I really only needed one. Let's give the other three to our, no, they don't do that. They do not resist a gift when it's offered to them. And what is the gift that we have from the Lord if it isn't eternal life?  You need to receive what I'm offering you.  Even so, these are the things that are qualities of a follower of Christ. And so in verse four, whoever humbles himself as this little child, well that's the greatest in the kingdom of heaven, humility draws you to a deeper dependence on God. Now he says in verse five, whoever receives one little child like this and my name receives me. Now that's interesting, until you think of it in this way, it's impossible to separate Jesus from his people.  So whoever receives a child of God in the name of Christ, he's saying in reality is receiving me. In other words, they're embracing not only you as a person, but the one who sent you. It's like what it says in Matthew 10, verse 40, he who receives you receives me, he who receives me receives him who sent me. Now I want you to notice he's not speaking about receiving just any child. He said one little child like this. So this toddler represents the children of God. It represents Christians. The physical in other words represents the spiritual. Verse six, continuance makes it clear, whoever causes one of these little ones to believe in me to sin, it would be better for him if a millstone were hung around his neck and he were drowned in the depth of the sea. What do the world because of offenses? For offenses must come, but what do that man by whom the offense comes?  It's better for you to enter into life lame or maimed rather than having two hands or two feet to be cast into everlasting fire. If your eye causes you to sin, plug it out, cast it from you. It is better for you to enter into life with one eye rather than having two eyes to be cast into hellfire. Now as we look at that, I want you to notice a couple of things with me as we look at it. Notice how in verse six says, whoever causes one of these little ones to believe in me. So believing in Christ obviously is a very, very important thing to talk about because there are those who think they're Christian who really don't believe in Christ. When he speaks of believing in him, he's speaking of spiritually being united by faith with him. Believing in him is not a conceptual belief. I mean, if you ask many Americans, if you believe in Jesus, they will say, yes, there was a Jesus. Most Americans to this day continue to say, well, yes, I believe there was a Jesus. How do you know there was a Jesus? Well, we celebrate his birthday and then they have Easter still and that's pretty much as far as they go. So believing in Jesus is not simply saying in your mind that there was one called Jesus. The Bible makes it very clear. It speaks about embracing him by faith and when you believe in him, you're actually transformed. It's like what it says in John 647 where Jesus said, most assuredly I say to you, he who believes in me has everlasting life. In 1 John 5, 11 through 13, it says, this is the testimony that God has given us eternal life and this life is in his son. He who has the son has life. He who does not have the son of God does not have life. These things I have written to you who believe in the name of the son of God that you may know that you have eternal life and that you may continue to believe in the name of the son of God. So when you speak about the ones who believe in me, he's saying those who are spiritually united with me, those who have been born again. And so in that reference, he goes on in verse six to say whoever causes one of these little ones who believe in me to sin, it would be better for them to have a millstone tied around their neck and dropped in the deepest part of the Sea of Galilee. Now, when he says causes one of these little ones to sin, who offends one of these little ones, the Greek word is skandalizo, and the word skandalizo speaks of to cause, to stumble or to fall. It is used in reference to entice someone to sin, to cause someone to begin to distrust and desert one whom he ought to trust and obey. It speaks about causing someone to fall away.  At Cal State Fullerton, I had a professor of cultural anthropology who was vehemently against Christianity, vehemently. I had a history professor that vehemently was opposed to Christians. When I went to Cal Poly Pomona, I had a social psychology professor on the very first day of class asked this question. He said, how many of you are born again Christians? There was a class of about 30 of us. A handful of us raised our hands. Very first day of class raised our hands. I feel sorry for you.  He says, I trust in science and I trust in studies. That's where I find my hope and peace in. You find it in your superstition, I find it in science. That was my introduction to the class. That was our first class. Later during the class, several weeks into the class, he stood up and said this, I'll never forget it. He stood up and said, there are various studies that I cling to as a smoker that demonstrate that cancer and smoking, lung cancer and smoking are not connected.  And I have studies to prove that. That was part of one of his lectures. He died of lung cancer. What's the fact? He died of lung cancer. He felt sorry for me because I trusted in God. He trusted in science. And so I expect, I expect the world to scandalize.   I expect the world to say, this is what we believe. You guys believe in no one is art. You guys are really backwards types. You guys are intellectual hillbillies. I expect that. And so Jesus is making it clear. You know, offenses will come. A woe into that person that brings those offenses. One of the things that we as Christians though, we need to really take to heart in this warning is we need as believers to understand that we can also stumble a brother or sister with sometimes our opinions or our perceived liberties. We need to be aware of that because sometimes people who are genuine believers take the liberties that they believe they have in Christ and use them in such a way as to cause others to stumble. We need to remember the tremendous influence that we have on one another. We need to understand that though I may have liberties in Christ for a variety of things, that the love of Christ within my heart ought to motivate my behavior towards other people. And thus I should not put any stumbling in front of any of you or my family or my friends or my neighbors because I don't want to scandalize you. I don't want to offend you. I don't want to stumble you. Now there are a lot of people today that I encounter and I've encountered them quite often who say, well, you know what, if they're bothered by it, that's their problem. Who cares? That's their problem. They have to grow up. That's their problem if they don't like it. Well, that's not love. What that is is that selfish. You're not thinking about other people. So Jesus is teaching us that in the world, there will always be offenses. But he says it's especially something I'm concerned about when people stumble one of these little ones who believe in me as a matter of fact he says it would be better if a 500 pound stone were tied to their neck and they were dropped in the middle of the sea. Now when he says that, he's speaking of the Sea of Galilee, he would be referencing something that had happened when the Romans had actually taken some of the Jews who had rebelled against Roman authority and had actually dropped them in the sea and killed them. So Jesus would be referencing something that these people were familiar with and he would say it would be better for them to be to perish in such a way. And so offenses he says in verse seven will come but you have to be aware that that's what happens in the world's system. It's inevitable that enticements to sin will come but woe to those who encourage believers to sin.  Now when he speaks in verse eight if your hand or foot causes you to sin cut it off cast it from you. It's better for you to enter into life lame or maim than having two hands or two feet to be cast into everlasting fire. And if your eye causes you to sin, pluck it out cast it from you. It's better for you to enter into life with one eye rather than having two eyes to be cast into hellfire. Obviously what he's doing is giving us a picture that is not to be taken literally or ours. I'd be standing up here with one foot, one hand and one eye. None of you would of course all of you would be in perfect health but me I'd be the one eye preacher, one hand preacher. So he's obviously giving us a picture and what would he be speaking of here? He's not saying self-mutilation because self-mutilation doesn't lead us closer to the Lord because it doesn't cleanse the heart. What he's speaking of is what you do, where you go and what you watch. He's saying you are personally responsible for all of these actions and all of these activities. What am I supposed to do? He's saying, well, dedicate yourself entirely to serving the Lord. What you do, where you go, what you watch, those things are to be dedicated to him. Sometimes you might think if you've been in the search for a while that Pastor David has a legalistic bent to him. You can think that. But let me give you one quick illustration and then move on. Some of you have heard this and maybe you've heard it more than once. Forgive me for repeating myself to you but others haven't heard it and I was just speaking with the brother about this just yesterday. So let me share this with you. Again, I mentioned to you a moment ago that there are liberties that you may perceive yourself to have that no one's gonna argue with you about. I grew up as a non-believer obviously as we all did and I began to abuse alcohol at an early age. Became a confirmed drinker and probably would have been classified alcoholic by the time I was 18 and was not one of these guys could drink a beer. I drank a six pack. If I wasn't a guy who drank a glass of wine, I drank a half gallon. I didn't drink a shot of whiskey. I drank a pint and that's how I would start my Fridays and my Saturdays. That's how I started. That's not including the dope that I smoked or dropped. So I was pretty into that lifestyle and it was killing me. It was killing me. Rested three times for drinking related offenses. It was killing me. Sent to a psychologist after my third arrest. My dad said there's something wrong with you. We've gotta find help for you. Alcohol related offense and burglarizing a jewelry store smashed my car into a light in the center island. I drank to excess waking up in places I didn't even know how I got there in the back seat of my car with vomit all over me. I was bad. I was a drinker that drank to excess and then I get saved. Then I get saved. And I didn't drink like I did it before. Once in a blue moon I'd have a beer and I'd say, ah, but I didn't do that anymore and went in the military, got out. I had a friend of mine that I started hanging around with and he had been raised in the church and I was going to his church at that time. I was 23 or 24 years old. And no longer was I addicted to drinking and my sin had been overcome by the blood of Christ and I was free, but he'd grown up in the church and I was only two and a half years old, three years old in the Lord. And he started telling me things like, you know Dave, he said, there's nothing wrong with having a beer. There's nothing wrong with drinking once in a while. Now I looked at this guy and I thought, well, he's been a Christian a lot longer than me. Maybe I have freedom in Christ. So I drank a beer. Not every day, once, twice. He and I go some place together and as we're in Huntington Beach, sitting in a pizza parlor. He says to me, you know what goes good with pizza? And I say, what? He says, beer. And I said, yeah, probably. He orders a pitcher, pours the glass, pours his own glass.  I'm not gonna get drunk. I'm not gonna get drunk. I sip the beer. When I sip the beer directly across from me on another table, but looking straight at me as an older gentleman, frail. I still remember very white hair, frail gentleman, directly in front of me. He couldn't have been 12 feet, 10 feet away from me. He was the table next to us. And I'm looking right at him as I drink my beer and he's looking in my direction. I put it down. The Spirit of the Lord speaks to my heart and says, go tell him I love him. And I say to the Lord, I can't. And he says, why? This is true. You can say this old man's making up a story. It's true. The Spirit is speaking to my heart in a very direct way. Go and tell him I love him. Tell him about me. I can't. Why? He saw me drink this beer.  As God is my witness, two people, two young men come walking off the street, walking to this pizza parlor. One man sits on his right side. The other one sits on his left. And I see this guy pull out a Bible, open it up and start sharing the gospel with this old man. And the Spirit of the Lord spoke to my heart and said, if I cannot use you, I will use somebody else. I have never forgotten that. I, forgive me, it's real. I wanna be used by the Lord. And alcohol will not enable that to take place. I don't care if you claim you have freedom. It's deeper. It is deeper than that. It is a soul. Your freedoms cannot be used in such a way that you cannot be used by God. I will put aside those things. Like Paul said, I'll put those things aside so I can be used by Jesus Christ. That's the key. That's not legalism. That's love. That's love for God. And that's love for people. Don't use your freedoms as an excuse to continue the flesh. And Jesus said, do not stumble, this little one who believes in me. In Colossians 3.17, whatever you do in word or deed, do all in the name of the Lord Jesus, giving thanks to God the Father through him. So when he's speaking, he's saying, deal with sin ruthlessly because it has eternal consequences. The eternal consequences are spoken of in verse eight. It's referred to as everlasting fire. And verse nine, hellfire. Hellfire is Vienna. It speaks of hell. And the Bible makes it very clear that hell was not created for man. It was actually created for the angels who rebelled. Matthew 25.41 says, he will also say to those on the left hand, depart from me.  It's a place of torment for those who reject God. According to Psalm 9.17, the wicked shall be turned into hell and all the nations that forget God. He says in verse eight that it is everlasting fire. It's a fire that will never be extinguished as the picture he's given to us. It's a punishment that lasts forever. And therefore he says, this is what takes place. You don't want that in your future. He said, it's more important for you to follow me. Moving into verse 10, take heed that you do not despise one of these little ones.  So when he says, do not despise that word despise.  Don't consider them to be inferior. As a matter of fact, you're to love one another, not to think yourself to be better than you actually really are. You should treat one another with love because God intensely protects you. God loves you. When it speaks of the angels always seeing the face of my Father who is in heaven, we need to remember, according to Hebrews 1.14, that the angels are ministering spirits sent to serve those who inherit salvation. He's saying angels are constantly on the alert, ready to minister to God's children when they're in need. And then finally, verses 11 through 14.4, the son of man has come to save that which was lost. What do you think? If a man has a hundred sheep and one of them goes astray, does he not leave the 99 and go to the mountains to seek the one that is trained? If we should find it assuredly, I say to you, he rejoices more over that sheep than over the 99 that didn't go astray. Even so, it is not the will of your Father who is in heaven that one of these little ones should perish.  And therefore they should not be treated poorly. Jesus said, I have come to save them. In Matthew 9.13, learn what this means. I desire mercy and sacrifice. I didn't come to call the righteous, but sinners to repentance. John 3.17, God sent not a son into the world to condemn the world, but that the world through him might be saved. He says, I sent my son, I came, Jesus saying to seek and to save those who are lost. It's not the will, he says in verse 14, of your Father who is in heaven that one of these little ones who are trusting in him should perish, to be brought to loss.  It's not God's will for them to be not cared for, but to be cared for. God wants them to grow, and therefore he says, don't stumble them. You wanna be great in the kingdom of heaven. Use Jesus as your model. You wanna be great in the kingdom of heaven? Don't stumble a brother or sister. You wanna be great in the kingdom of heaven? Step away from the things that are distracting you and taking you away from God. You wanna be great in the kingdom of heaven? Become the servant, the serpent of all.