I'm Brian May with Cal OES at the Creek Fire in Fresno County. This fire now over 176,000 acres, still 0% contained. And you can see behind me, one structure is burned, the structure next to it still standing. This fire moved through here quickly. People here had minutes to evacuate.  Russ Davis is Chief of the Bald Mountain Fire Department. This is home for him. He knows this Sierra National Forest well, knows Shaver Lake and the fuels that are still feeding this massive fire. The ongoing firefight includes hand crews, dozers, air support and mutual aid being coordinated by Cal OES from across the state. This fire dynamic and changing by the minute. More evacuation orders could come at any moment, meaning those in the area need to be ready and pay close attention to local authorities. Like Russ. When it's pushing that fire and if it's kind of creeping on you, don't let that fool you. That foolish type of situation or denial that it's going to get to you, that's foolish very much. It's going to come and get you. For more information on wildfire evacuations, go to ready.gov. And for the latest on the wildfires currently burning in California, go to fire.ca.gov.


And so let's begin reading together here in 1 John chapter 3 and let's see how much will I read. I'll read verses 1 through 3. 1 John chapter 3 beginning at verse 1 to verse 3. John writes, Behold what manner of love the Father has bestowed on us, that we should be called children of God. Therefore the world does not know us because it did not know him. Beloved, now we are children of God and it has not yet been revealed what we shall be. But we know that when he is revealed we shall be like him, for we shall see him as he is. And everyone who has this hope in him purifies himself just as he is pure. Now as we've been going through 1 John in chapter 2 and now that we're entering into chapter 3 but in chapter 2 John had pointed out that that God is righteous and that his children are also righteous. Notice in verse 29 of chapter 2 how it said, If you know that he is righteous you know that everyone who practices righteousness is born of him. And so as his children our works of righteousness are pointing to something. Our works of righteousness are a visible indicator that we are his children. Now what is it that has inspired and energized our works of righteousness? Why do we perform works that God would state are righteous? What is it that has inspired this? What has energized it? Well it's his love. It's a love that has moved him to adopt us as his children. He desired us to be his children and his love accomplished that desire. We need to remember even as we introduced this passage that righteous works don't save us. Righteous works are the evidence of salvation. When Paul was writing in Titus chapter 3 verse 5 Paul said he saved us not because of righteous things we had done but because of his mercy he saved us through the washing of rebirth and renewal by the Holy Spirit. So he saved us not because of the righteous things that we have done. And so we're saved by his grace. His love has motivated him to accomplish his desire for us to have fellowship with him. Now notice how he begins. This is something that you might want to take a moment to look at in chapter 3 verse 1 how he begins with the word behold. Behold. He says behold so we can more deeply appreciate how marvelous this is. He says behold look at this. Look closely at the incredible kind of love that God has lavished upon us. Behold this. Look at it closely. Behold. And even as we sang a moment ago I was thinking behold is used so many times in scripture like behold the Lamb of God who takes away the sin of the world. If any man be in Christ his new creation all things are passed away. Behold all things are become new. And even before behind me behold I make all things new. That means this is to grab your attention. Be aware of this. This is incredible. God has loved us and God has demonstrated his love for us. And it's because of his love for us that he has allowed us and made it possible for us to become his children that we might be called as it is saying in scripture children of God. And that again is something amazing how that God actually identifies us and acknowledges us as his child. It's the incredible love of God and it's incredible when you really think about it. It's incredible that God loves us at all. We don't deserve his love. And even in our best days we're not that lovable but God has called us his children. Behold that's incredible that God would call us his children. Now what did he do to make it possible for us to be his children? How great is his love for man? What can we use as a measurement that demonstrates the depth of his love for us? Well the depth of his love is revealed by the death of the Lord Jesus Christ. When Paul was writing to the Romans in chapter five verses seven and eight he said it like this.  But he went on to say God demonstrates his own love for us in this. Well we were yet sinners. Christ died for us. We were sinners. We were far away from him and yet Christ died for us. And so how did we become his children? We became his children by receiving his gift. Around 20 years ago my father-in-law passed away. My father-in-law very, very, very dear man, very dear man to me. Here's a good man. And he would come here when we were building the new sanctuary because he owned a construction business with his brothers. And he would come here on Fridays and he would just watch the construction he had retired. And so he had some time and he would come here and he would watch the construction. And he did it regularly, so regularly that it was expected that that he'd be here on the grounds and one Friday he he wasn't here. And so my sister-in-law and my son Joseph got concerned and so they went to his house and when they went to his house they walked in. Now my son Joseph at that time was working in an ambulance service. He ultimately finished college and became a registered nurse. But at that time he was going through some basic things and all and he was working for a particular company and he'd already been trained in a variety of things pertaining to to you know immediate emergencies and all. And so he went to the house and when they walked in to the house in the hallway was my my my father-in-law and he had had a stroke. And so Joseph took his vitals told my sister-in-law Patty called 911. He was brought to the hospital. But before that happened Joseph knelt next to him my son and he said to him grandpa are you are you ready because he could see that this was indeed very serious. He said are you ready? And my father-in-law knew what he was saying. So my father-in-law said to Joseph I'm a good man Joseph. And in that he was right. My father-in-law was a good man.  You're not good enough. He said all have sinned full show to the glory of God grandpa. You need Jesus and he began to share with him the gospel. And there on the floor in his house my my father-in-law gave his heart to Jesus Christ. And when I went to see him in the hospital the first thing he did when I walked in and he saw me he was he had that he had a stroke. He he tried to speak to me and he said to me David I got angry because he had been disappointed by a friend and it had gotten to him. And I spoke to him a little bit I shared with him some things later on I told my wife I said you know I didn't realize what your dad was doing. He was making his last confession. That's what he was doing. He was looking for absolution is what he was doing because of the little time he had ever gone to church he was one who would drive by the church and that's pretty much when he went to church he drove by it. My mother-in-law was a more frequent visitor but not my father-in-law. And so Joseph had prayed with him he gave his heart to the Lord he wanted to confess to me because he saw me as as like a priest to him in that way. Well he went home to be with the Lord a couple of days later and then they had a funeral service in Chinoate St. St. Margrits St. Margrits. And several of us had opportunity to speak of the wonderfulness of my father-in-law but the priest got up and spoke of him and in his his speech and speaking of him he said Andres which his name he really we called him Rubin so he called him Andres he said is in heaven because he was water baptized and went on with how Catholic doctrine is. My son Joseph came up after the priest and my son Joseph said my my grandfather is in heaven not because he was water baptized but because he gave his heart to Jesus Christ as his Lord and Savior and shared the gospel with the whole family that was sitting there who had otherwise not had the opportunity to hear the gospel clearly presented. When you have the opportunity you share the gospel because that's the amazing story of God's love for man. It is not works of righteousness which we have done. It's according to his mercy it's by his grace that we receive salvation. God so loved the world he gave his son and while we were yet sinners the scripture says Christ died for us. And so we receive that that gift it's a gift of eternal life like it says in John 1 verse 12 as many as received him to them he gave the right to become children of God to those who believe in his name. And so we have by faith received this free gift and in doing so we became his children. Behold what matter of love that is. Well as a result notice verse one second portion the world does not know us any longer. This world system that we've already looked at does not recognize our value who we are and what we do. The world does not recognize us as its own that's what that verse means the world does not know us any longer the world does not recognize us as its own. The world in other words rejects us the world hates us and the world persecutes us. You know we look at the founding of this nation and sometimes we look at the founding fathers and we like to to regard some of the things that they did and said and all of that. And I was looking at something Thomas Jefferson who was one of the founding fathers he was the third president. I was looking at something Thomas Jefferson said he said or wrote he said and wrote the Christian God can easily be pictured as virtually the same God as the many ancient gods of past civilizations.  If one wish to know more of this raging three headed beast like God one only needs to look at the caliber of people who say they serve him. There are always of two classes fools and hypocrites. So before we begin worshiping the founding fathers maybe we ought to remember some of the things that they actually said. Now there's a man by the name of Alistair Crowley. He lived 1875 to 1947 and he was what we would today refer to as an occult leader. And this is what Alistair Crowley said he said one would go mad if one took the Bible seriously but to take it seriously one must be already mad. And so the fact is is the world does not own us as its own. I could give you so many citations of that. Those are just two examples. You see Jesus in John 15 verses 19 through 21 said if you belong to the world it would love you as its own. As it is you do not belong to the world but I have chosen you out of the world.  Remember the words I spoke to you. No servant is greater than his master.  If they obeyed my teaching they will obey yours also. They will treat you this way because of my name for they do not know the one who sent me. There's an apologist by the name of Frank Torek and he said if Christians continue to rely on emotion and ignore evidence they will continue to lose their children to secularism. Half-hearted Christianity cannot withstand rabid secularism and make no mistake secularism is rabid. The world isn't neutral out there. Today's culture is becoming increasingly anti-Christian. And so that's just what the scripture is saying here. The scripture is making it very clear that that is exactly what is being said. The world does not know us because it because it did not know him. Now he goes on to say this so in verse 2, beloved now we are children of God and it has not yet been revealed what we shall be but we we know that when he's revealed we shall be like him for we shall see him as he is. So we have not yet been fully illuminated concerning everything that's before us. It has not been revealed what we shall be. So even at this time it has not been fully revealed but one day it will be. Somebody said how pure and holy, intelligent and wise in our souls, how spiritual and glorious in our bodies, how exalted in dignity, how great in power, how rich in inheritance, how happy in enjoyment, how happy are we who have this awaiting us. You see one thing we do know when he appears he says we shall be like him. We shall be like him. We will have glorified bodies. We'll be fully conformed to him. Paul and Philippians 3 20 and 21 says it like this. Our citizenship is in heaven from which we also eagerly wait for the Savior, the Lord Jesus Christ, who will transform our lowly body that it may be conformed to his glorious body according to the working by which he's able even to subdue all things to himself. No more diets in heaven. Isn't that great? One size fits all. It's just going to be wonderful. In second Corinthians 3 18 it says we all with open face beholding as in a glass the glory of the Lord are changed into the same image from glory to glory even as by the spirit of the Lord. You see in verse two we shall see him as he is. We shall see him in his glorified state in his glorified body. No longer at that time will be we be walking by faith. At that time we will see him face to face. In Revelation 5 verse 6 the first portion of that verse John said I saw a lamb looking as if it had been slain standing at the center of the throne encircled by the four living creatures and the elders. We will see him and we shall see him as he is. And if you have that hope in you notice verse 3 everyone who has this hope in him purifies himself just as he is pure. That is our motivation for living a life that is set apart for the Lord. We live lives that are holy. We live lives that are set apart for his sake because we long to see him. Now somebody said John does not hear speak of any man purifying his own heart because this is impossible. But of his abiding in the state of purity into which the Lord has brought him. So when we come to faith in Christ we are brought into that state of purity and we remain there. We remain we abiding him and our lives are demonstrating that. In 2nd Corinthians 7 1 it says having therefore these promises dearly beloved let us cleanse ourselves from all filthiness of the flesh and spirit perfecting holiness in the fear of God. So we anticipate being with him and because we anticipate being with him we live lives that are set apart for him. The way that that a young bride to be sets herself apart. She sets herself apart as a matter of fact in the Jewish culture she would be referred to as the one who is set apart. And so the Jewish bride would be set apart. She was waiting for her husband and then ultimately she would see him face to face and she would make herself ready for that. In in our day that's sometimes not looked at as seriously as it was in previous times. Sometimes people actually get married in ways that really kind of like don't reflect the sanctity of the marriage let alone its importance you know because sometimes people like to do things be silly or whatever in their wedding and they think it'll be memorable if they do such and so things. There was a time when and and there still is amongst many when getting married was a big deal. You know young girls would be raised in in homes where moms knew that they were girls and would raise them in that way. And as they did so they would be prepared. Everything in their life in any young women's lives that this isn't a stereotype many of you can say this is true this is true. The way a lot of us were raised you know what do you give to a two or three year old little girl for Christmas. A lot of different things but a lot of things I've seen this with my own grandchildren I saw we did it with my own daughters. You give them a baby doll.  You know and in little girls they used to give them you know stoves and kitchen things and they would play and they would make food and they we gave them little little you know pretend vacuum cleaners and and so now they're living their dream when they got married just kidding. But they were prepared weren't they. They really were really believed in and true love they really believed in marriage that last they really believed in in a man that one day would come and sweep them off their feet and and love them forever. There's nothing wrong with believing that because there are plenty of men who will do just that there's nothing wrong with believing that and they would prepare themselves. They would prepare themselves for the wedding day. The wife will go out and wife to be the bride to be will go out and she'll find a dress and she'll you know sometimes she'll feel the need to lose a few pounds and she'll put that dress on and ten years later she just looks at it. But the women women go out and they they purchase things that they keep and men we're different men go out and rent a suit that somebody wore last week that's what we do. But women are different in that and there's nothing wrong with I think it's beautiful but they prepare themselves and and the wedding day comes and it's really about that brightness in it when the music plays and everybody stands and they face the door and you hear that that bridal song dum dum dum you know and she comes walking and you put a chain on that groom so he can't get away it's just a beautiful time for the victim and it's bright and so as as that goes forth there is a lot of preparation and it's a beautiful moment the one picture that I have in my room the one I have of our wedding day is when I first saw my wife when she came walking out the photographer took a picture of my girl and I have it on my desk and I see that every day every day I look at that picture and I remember the beautiful moment when I first saw my girl and one of these days there'll be a beautiful moment as the Lord first sees us so we prepare ourself it's not that he just isn't aware of us of course but it's that moment that we're with him and on we prepare ourselves and so he's saying everyone who has this hope in him purifies himself just as he is pure we we live a life set apart we we stay away from the things that that will will cause impurity in us we we don't play with sin we don't mess with his grace we we just prepare ourselves that's what believers do and the closer the time comes for us to see him the more prepared we want to be and that's our hope and because we have that hope we we do purify ourselves from all filthiness of the flesh and spirit we do perfect holiness in the fear of God well as he continues in verse 4 he says whoever commits sin also commits lawlessness and sin is lawlessness and so the world is divided into two sections if you will there are those who are the children of God and there are those who are the offspring of the devil and the characteristics of such lives are either righteousness or rebellion you see he says whoever commits sin also commits lawlessness so he tells us what sin is sin he said is lawlessness now the Greek language is the word lawlessness in the Greek language which the New Testament was written in is a nomia a nomia a nomia without law lawlessness it speaks of a contempt for and a violation of law why would he write that here after saying these things well it would seem that the Christians are being influenced by false teachers and are being tempted to regard sin with indifference you see one clear earmark of a believer we are no longer slaves to sin the one who commits sin is a slave to sin Jesus said in John chapter 8 you commit sin is a slave to it well we are no longer slaves to sin Roman 6 verse 6 says it like this we know that our old self was crucified with him so that the body ruled by sin might be done away with that we should no longer be slaves to sin the one we yield ourselves to becomes our master and when we yield our self to sin it rules over us but because we have yielded ourselves to Christ because we have given ourselves to him he now rules over us and we have the power of the Holy Spirit who lives within us and has given us freedom and one of the earmarks of a believer is freedom in Christ now there is much confusion about what it is to be free in Christ someone said it like this they asked the question what then is the nature of true liberty not being free to do anything you want to do but in coming to the place where you delight in the performance of what you ought to do you see there are mistaken understandings of the liberty we have in Christ today where people you know I've been in the ministry for a while I've pasted this church for a while I can tell you this as a fact I've been in the ministry for 49 years I guess I have a little experience in it and I can tell you that over the years on more than one occasion I have encountered people who think that liberty is the freedom to do whatever I want and still go to heaven that came out of the Gnostic religious faith that had come upon Christianity and had imported its beliefs that you could do whatever you want with your body that's called dosadism you could do anything you want with your body and that doesn't infect the spiritual person but that's not true in the sense that we are free that is true but not to continue in sin we are free so we're no longer slaves to sin and that's what the Bible teaches very clearly we aren't indifferent to sin and we don't indifferently practice it as a way of life you see what he is saying here is whoever commits in that word commits there that word in the original language commit speaks of a continual habit it speaks of abiding in as a regular way of life continuous rebellion it's living in sin as the habit of your life it's not that you can't on occasion sin all of us do in word and thought indeed it's a regular thing none of us will be free from that that that ability until we see Jesus face-to-face but at the same time there are those who think oh no it's no big deal and so he's saying this whoever commits sin is really a slave again speaking of a way of life he's saying that a false Christian's life is not free from the entanglement of sin again we we sin as Christians but we don't practice it we don't indifferently practice it as a way of life someone said it is one thing for sin to live in us it's another for us to live in sin so believers no longer continue to practice in as a way of life but false Christians do even when deceived into thinking that they aren't sinning there are cult members whose outer lives can sometimes look better than the life of believers I remember when I first got saved because I came out of the rebellious period of that day as a hippie and long hair and barefoot in the whole nine yards you know if you put me next to a Mormon young man and you asked who's the Christian the average person would have pointed to the Mormon kid because they wore their hair short and all of that and me I looked like a grubby guy didn't wear shoes and things like that so there's always that that outer appearance and sometimes you can be part of a religious group that has outward appearance of righteousness and you can be looking at this guy over here this one over there you can be thinking that person couldn't possibly be a Christian it was amazing to me and I won't speak too long about this but it was amazing to me when you know people started getting tattoos and all and and in this fellowship you know I had somebody an employee who was working for us who who have had tattoos and you know and people were upset people got upset guys got tattoos let's see there was a time when you know motorcycle gangsters and people in prison and sailors people like that they got tattoos and people say oh that's that's a rough person but that time changed but what's interesting to me was how people was would treat I even had to this is years ago now I felt the the necessity to on more than one occasion to to remind the church that you don't judge people by how to appearance you know I used to tease about it some of you heard me when I'd say this you know because there was this phrase that was used I don't even know if it's still used tramp stamp is that still used is that I don't know if it's used yeah remember that's trapped out right I really didn't know what that meant I did hear it but it was on the lower back so I tease the church and some of you girls are putting hummingbirds on your back I said they call it tramp stamps and the church kind of giggled over that one I said you know and it might be cute right now you got this hummingbird but your skin stretches and one of these days you're gonna have a vulture so be aware of that and I used to tease them about that and so when you walk is it's gonna be flying away if we get caught up in the outer appearance I guess that's what I'm trying to say and I shared the with the church you know that and I still do obviously I'm doing it right now is to judge a person by outer appearance is always wrong you may see the outside but you don't see the heart so look to the heart and how do you know what the heart is well you really don't know but you do know this that out of the abundance of the heart the mouth speaks so that can be one of the ways that you're aware and another thing would be the way they live how they are and so when you combine the things that they're saying with the things that they do you have a pretty good opportunity to to understand whether they know the Lord or not so somebody may have an outer appearance because the cult members had an outer appearance some of them were rigid aesthetics they had an outer appearance of holiness but their life still is lived in a lawless way they they they are really not abiding by the rules of the Lord in their life and so he says whoever commits sin whoever practices it as a continual way of life is lawless is without law and sin is lawlessness but he goes on in verse 5 you know that he was manifested to take away our sins and in him there is no sin so God provided a way for us Jesus who takes away he carries away our sins Jesus was manifested he says that speaks of the fact that he took upon himself human flesh that relates to the incarnation and he took upon himself flesh that he might take away our sins remember in John 1 29 behold behold the Lamb of God who takes away the sin of the world or in 1st Timothy 1 15 this is a faithful saying worthy of all acceptance that Christ Jesus came into the world to save sinners of whom I am chief so Jesus was manifested to take away our sins he came to bear our sin so a life of habitual sin is incompatible to a relationship with Jesus Christ the result of understanding this is freedom when we understand that Jesus bore away our sin as the Lamb of God and that we in Christ have a new life empowered by his spirit and directed by his word and spirit when we understand that our our sins have been completely forgiven and and that God is given to us yes commands but he also gave us a power to obey them through the power of the Holy Spirit and that this really in many ways rests on who we are understanding of who we are and our desire to do that which pleases him our life changes and that brings to us freedom from bondage to sin in 1st Peter 1 24 it says speaking of Jesus who himself bore our sins in his own body on the tree that we listen that we having died to sins might live for righteousness by whose stripes you were healed having died to sins when we continue to practice sin with no sense of remorse it's been said it's like kissing the tip of the spear that was plunged into the side of Jesus when you love the Lord you don't do those things that displeases him you see sin is incompatible with Jesus because in him is no sin Jesus is that perfect offering he goes on saying verse 6 whoever abides in him does not sin does not practice or continue in it as a lifestyle whoever sins continues in it has neither seen him nor known him so sin is no longer compatible with our deepest desires we love the Lord with all our heart we desire to please him we're actually what the scripture would refer to as a disciple one who was thoroughly in love with Jesus Christ and we can no longer enjoy sin we can no longer just do something forget about it find more sin to do that doesn't happen it's not that that you become sinlessly perfect it simply means that the conviction of the Holy Spirit is so great in your heart that when you do something wrong you're grieving over it you grieve over it before you were saved it didn't matter you did it you enjoyed it and you looked for more but when you got saved you may have stumbled into it or chosen to do it but it wasn't enjoyable anymore it wasn't what you used to get out of it now you have nothing but grief and sorrow of heart but you say to yourself how did I do this I don't want to do this anymore and at a certain point you actually cross over mentally you understand why is it that I didn't like that I didn't like that because it's wrong and because I love Jesus and I want to please him and that's what happens so he's saying whoever abides in him when he uses the word abide it speaks of a permanent abiding so whoever abides permanently it could be translated in him does not practice sin sin is no longer compatible with our desires he says in verse 7 little children let no one deceive you he who practices righteousness is righteous just as he is righteous don't allow people to seduce you into error you have responsibility not to be deceived let no false teacher persuade you that a child of God still practices sin and God accepts that again even to this day some false teachers say sin isn't really sin as a matter of fact they're in the business of redefining it they're making it acceptable sometimes they legalize it and in legalizing it it makes it acceptable not only to to individuals between entire nation we know that we've seen that in Isaiah 5 20 what went to those who call evil good and good evil who put darkness for light and light for darkness who put bitter for sweet and sweet for bitter that's what we're living in right now we know that I'm not going to preach a lot about that you're aware of that we're living in that time where evil is good where darkness is called light where bitter is called sweet you see these false teachers were teaching that a believer could continue in sin and enjoy it that was compatible with the way the world had operated but we're not to be that way in Colossians 3 5 and 6 Paul said put to death therefore whatever belongs to your earthly nature sexual immorality impurity lust evil desires greed which is idolatry because of these the wrath of God is coming now he speaks in verse 8 interestingly notice how he says he whose sins is of the devil for the devil has sinned from the beginning for this purpose the son of God was manifested that he might destroy the works of the devil he whose sins is of the devil in other words sin originated with Satan who was the first rebel against God when he said he whose sins is of the devil Satan is a fountainhead of sin and the originator of rebellion against God in John 8 Jesus said it like this verse 4 4 when he was speaking to some religious opponents he said you are of your father the devil and the desires of your father you want to do he was a murderer from the beginning and does not stand in the truth because there's no truth in him when he speaks a lie he speaks from his own resources for he's a liar and the father of it that's heavy is a murderer and a liar that's what Jesus said so a person who is following the Lord isn't going to practice sin because the ones who practice it as a way of life actually belong to the enemy and he said for this purpose the son of man was manifested that he might destroy the works of the devil the word destroy for this reason the word destroy means to loosen or untie it speaks of releasing or breaking it speaks of dissolving Jesus was manifested to release captives from Satan's control and the way he did this was by giving himself as a ransom in Matthew 2028 he said the son of man did not come to be served but to serve and to give his life a ransom for many in 1st Timothy 2 5 and 6 there's one God one mediator also between God and men the man Christ Jesus who gave himself as a ransom for all the testimony given at the proper time in Hebrews 2 14 and 15 since the children have flesh and blood he too shared in their humanity so that by his death he might destroy him who holds the power of death that is the devil and free those who all their lives were held in slavery by their fear of death I don't want to seem to be callous and uncaring and it certainly can appear that way by what I'm going to say but I do believe that one of the things the enemy used especially to his to his own benefit was through the COVID situation the United States has gone through and continues to is he actually opened up my eyes to to the reality of many who have a fear of death because after all that when I don't want to make a long story out of this just just an observation because after all this time all the all the research is finally coming in and various things if you watch the news and try to stay up on all of this you'll see that there are a lot of things that are coming to light that were at one time hidden for whatever reason but one of the things that I did see is how easily it was for people to wear their masks even when they're in their car when they're walking outside by themselves and and I certainly don't want to to judge them with a broad brush because undoubtedly there are many who who may have had contamination didn't want to infect others and I give them grace and charity for that on the other hand there are some who wore it simply because they were afraid and the fact is that that Jesus came to free me from that fear the fear of death what's the worst thing that would have happened to me I had it twice my wife and I had it twice what is the worst thing that would have happened there's no worst thing that could have happened the best thing could have been I'd have been to be with Jesus that's that's a good thing it's not a bad thing you know that's a good thing it's not a bad thing again don't presume on the grace of God don't do something silly or don't do that but don't live in fear either don't live in fear either do you know Jesus if you know Jesus if you know Jesus you're gonna go to heaven you're gonna go to heaven well I want to go but not right now yeah and I've heard these young people say I don't want to go right now I hope the rapture doesn't occur I want to get married so you'd rather go through Helfers anyway I'll keep going we don't have to be afraid so whoever has been born of God verse 9 does not sin does not practice or continue in sin for his seed God seed remains in him he cannot sin you cannot sin without a sense of conviction you cannot sin because he's been born of God so we are God's offspring God's spirit has created an entirely new life a life that glorifies him we cannot continue in sin not because it isn't possible for us to sin but because the spirit within us is grieved when we do the conviction of the Holy Spirit comes upon us we turn from that it is no longer our way of life and then finally verse 10 in this the children of God and the children of the devil are manifest are openly revealed whoever does not practice righteousness is not of God nor is he who does not love his brother okay we'll close with this John is actually giving us a test to reveal the believer or the unbeliever he says the unbeliever does not practice righteousness in other words he doesn't she doesn't desire to live a holy life a pure life because sin is their way of life and they enjoy it and Psalm 10 verse 4 the wicked through the pride of his countenance will not seek after God God is not in all his thoughts so sin is their way of life you can speak to them they don't want to hear it keep your religion to yourself he doesn't want to practice righteousness but at but another earmark is that he doesn't love he doesn't have love within his heart doesn't love his brother a believer loves habitually and we love those who are members of the body of Christ we're family and that love is manifested in in various ways like Romans 12 10 where he said be kindly affectioned one to another with brotherly love in honor prefer one another or Galatians 5 13 rather than you've been called to liberty only use not liberty for an occasion to the flesh but by love serve one another so you prefer one another you're kindly affection to one another these are things that earmark or demonstrate that you know God I'll say this briefly and then we'll close with one last thought somebody I've heard this so many times and so much seriously even recently so I'll say it to you guys I've had people say this to me they have said this and I'll kind of gel a lot of comments over the years they say one thing that they have seen in our fellowship and I'm so blessed to say this to you they said they said Pastor David your church is a very loving place and it is it is we have our share of stinkers that's that's true but who doesn't but our no our church is our church is a very warm place I love our church you know I tell you that because I mean it I don't say that just to say it I say it because I mean it I love our church I don't know everybody of course but I love our church and when people come in and just last week when I was in Calvary Downey I was part of of service honoring and and blessing Jeff Johnson from Calvary Downey people were telling me that love your church you've got warm people loving people your staff you know what you guys are great I love you guys I really do and it this is from my heart to yours I love you guys you're you it makes it's a joy for me to be able to come on a Wednesday and talk to you it's a joy I love it on Sunday mornings I love it just be able to see the family to gather together to enjoy each other's presence love Jesus together and that's what Christians are right is we love one another that's what we do Jesus said that's how people will know we belong to him he's just saying that that our faith is is revealed in the way we live in the way we love and the believer will be known by their love for others in chapter 4 we'll get there verses 7 and 8 it simply says we love it let us love one another for love is of God and everyone who loves is born of God and knows God he who does not love does not know God for God is love that's one of the songs we learned to sing when I was first saved he loves not north not God for God is love when you are connected with the God of love your life changes your life changes it really does when you have a belief in a God who is never satisfied with your efforts you will never have peace you'll never have joy you certainly won't experience love you'll be too busy trying to be good but when you understand that in spite of what you are God has loved you in spite of what you are he sent a son for you even in your best day you're not good enough when you understand that and yet he loves you anyway changes your life changes the way you think and it changes the way that you live because we worship a God who has loved us we love him because he first loved us and that's how it works and it works well and the unbeliever does not live a righteous life and nor will he live a biblically loving one


Hi Chris, this is Ashley with Augsmore Toyota. I wanted to send you over a quick video greeting that way you can see who you'll be working with. Thank you so much for your recent interest in a new forerunner. I want to hit instant two over our current inventory for you to look over at your convenience. When you get a moment, please give me a call. My number is<PHONE_NUMBER> or just email me back. Thanks. Bye.


Hi, my name's Sam. I'm from Melbourne University down in Australia. I'm part of the Nectar project. This is a kind of a three-part talk. Here's myself. We've got Belmiro here from CERN and Matt here from Rackspace. We're just going to give you a bit of a talk about sales and Scaling OpenStack. We gave a talk in Paris six months ago, so this is kind of a bit of a follow-on from that talk. So for people who are there, we'll hopefully have something different to tell you. So Nectar is the project that I'm part of in Australia. It's a government-funded project that started in 2011. We've started Early Cloud on the Cactus branch and kind of started production in Diablo and we've been slowly upgrading ever since. We're mainly on Juno now. A few Ice House nodes around. We're kind of part of a federation as you can see. We've got eight sites around the country that kind of geographically dispersed. Well, we've got, lucky enough, in the research network we have big networks, so we have kind of a big pipe between the data centers. So we use sales to really join up all these sites around Australia. The reason we kind of went for sales is there's really a couple ways you can scale OpenStack. One of them is regions and the other one is going to scale up the sales. We wanted to have a single end point for our users. We have, most of the users on our cloud are researchers and scientists. They're not all IT professionals, so we wanted to give them a nice, simple interface. We want a single set of security groups, a single set of key pairs, and just one central front door for everyone to kind of go to and use our cloud. One of the other reasons that we see as an advantage of using sales is the, I guess, the OpenStack expertise or the lack of the OpenStack expertise that we have. And so we can kind of have a core team in one location, like handling all the central APIs and things, and really each of our sites around the country, they all need to just deal with the compute nodes and a small subset of OpenStack. So that really is another reason that we go for sales. How big is Nectar? There's eight sites of us around the country. We're growing pretty rapidly still. We're currently at our final deployment in terms of size. There's around 700 hypervisors, so we have around about 100 or so per sale. I guess we're not as big as these guys, but in terms of how we scale our system, I think we're a decent size. In size of people as well, I guess we're a small team. We have three people running everything locally, maybe less than that. We have a high staff turnover, so as everyone does, I think. Interaction with other services. In terms of with ourselves, we have other OpenStack systems involved. We try and have a swift region in each sale, and then we can use local glance APIs to then feed out and over. Especially when we're quite geographically dispersed, having a local object or a local image really is the main thing for booting is quite important to us. We also have Cinder around the cloud. All of our sales will have at least one Cinder volume host. We're using different back ends at each. All of our sales are the same, but quite different. It's up to each site to determine what vendors and stuff they use. Solometer is another thing that we use to varying success at the moment. We have a large Mongo cluster, and we really just kind of fire everything up to it. Some things work, some things don't. We're a Nova network installation, so when we started, there was only a Nova network, and we're still on Nova network. We do have kind of a big network around all our sales, so in the future, we might be able to leverage that when moving to Neutron. In terms of actually what we run on each sale, this is just a list of everything you run, APIs at the central level, and just really just the compute stuff going on at the sale level. One of the things that we really need to deal with is scheduling in terms of sales. Where do we put instances? We have geographical requirements. We have users that want to schedule at a certain sale, or for us, an available zone. We kind of use the sale term as an internal thing for scaling, and we use the available zone as a user thing. They kind of match up one-to-one in terms of a sale on the zone, but not necessarily, and so users can choose where they want to launch their instances. Now we're doing, we have some sales that are, we have a test sale, I guess we call it, that we only let certain tenants on. We use filters there. We have some sales that have higher performing nodes, GPUs, fast.io, things like that. We use the same things that in a non-sale environment we use, aggregates and flavors really to determine where those instances are scheduled. The other thing to do with scheduling is when we're bringing on new sales, it's one of the, it can be quite an issue. We want to test out this new sale before we open up to the public. We don't want to, we don't want to flood a new sale. When we bring on a new sale, part of the scheduling is how much RAM is available, and so if you bring on a new big sale, everything's just going to port it at, and if you've got a faulty new sale, then it's even worse. So we use a few techniques there to kind of slowly bring on a new sale. We will start off with having it restricted to only certain people. We'll kind of treat it as a pre-production. We'll have beta users on there, and then slowly we'll open it up to the global scheduler, so we kind of have a nice smooth on-ramp. We also run an open stacks system to manage all our infrastructure, which has been something that we did just before Paris, I think, and it's been quite important to us, and it's really made it easier for us to deploy the infrastructure. We've got so much infrastructure now that we need a cloud to manage that, and that's been really helpful for us. Everything is mainly virtualized in terms of our control infrastructure. We have physical databases and rabbits, but everything else is pretty flexible, and that really helps when we're upgrading. I think we've kind of got a process now when we do our upgrades. We've just almost finished the upgrade to Juno, so we have about half our sales are on Juno and half of them are on Ices. We've kind of got a process of doing conductors and APIs, and at least the last two upgrades we've had have been not impacting to the users. We can kind of do a sell at a time, and rolling upgrades is just a lot easier. Monitoring and tender sales, because we're kind of a federation of different organizations, we don't have the same people at each sale, and so really we have to have the firm interaction between the rabbit servers and sales, so that's where their kind of contract is. We send a message to your rabbits, and it's really up to them to deal with that, and how the instance gets scheduled, or how the request gets handled, and so having strong monitoring around that interaction is quite key for us, and vice versa, so there's some requirements. We have to make sure that our system is very highly reliable, and one of a good test is really just the console log. Spend up an instance in all of your sales, and do some monitoring on console log, and it's a, because it's a call, you need a response, it's a good way of actually testing all the functionality that sales does. The future is that we're looking at Neutron at the moment.  There is no real differences in terms of, with sales there's a few little things that we had to make work, but really Neutron can work in a sales environment without any changes. We're looking at, because our networking is very simple, we just have a single flat network, public IPs everywhere. It's probably a simpler migration for us to move to Neutron. Once we're at Neutron, we're hoping to get some more of the fancy, you know, tenant networks, and load balancers service, and some of those high level services, but that'll be a slow progress, I think, and, you know, throughout this coming year, we hopefully will get there. You know, we're looking at other services, I mean, all the other services, really, that are high level, don't really interact, or don't need to know about sales, which is quite nice, you know, it's a, I think that's all of what I've got. Yeah, and I'll pass over to Belmiro. So, thanks. So, my name is Belmiro Moreira. I work for CERN. So, what is CERN? CERN is the European Organization for Nuclear Research. It was created in 1954. It has 21 member states, and is located in the border between France and Switzerland near Geneva. CERN mission is to do fundamental research, and is the biggest international scientific collaboration in the world. We have 10,000 scientists from more than 100 countries that work at CERN. For the scientific research, CERN operates a network of particle accelerators and detectors that they are used by several experiments. One example is this one. This is the Large Hadron Collider, and this is the most powerful and largest accelerator in the world. It's a circular accelerator. It has a diameter of 27 kilometers, and it sits 100 meters in the ground Geneva. When in operation, the detectors that are connected to the Large Hadron Collider can produce one petabyte of data that needs to be filtered, stored, and analyzed. To analyze all this data, CERN provides computer resources to scientists all around the world, and to help us on that, we now provide the cloud infrastructure based on OpenStack. So, six months ago in Paris, I presented what was the motivation of CERN using cells, and now I'm going to give you first a perspective what is the state, what change, and now what are some problems that we are facing. So, the cloud infrastructure is in production since July 2013. We are now running Juno. We finished the upgrade three weeks ago, and we started also offering it to our customers. We have two different virtualization technologies in our cloud. We have KVM and also Microsoft IPv, and we're starting upgrading our scientific Linux 6 nodes to Cento7. The reason for this is to continue to use the OpenStack distribution packages. Our cloud runs in two different data centers. One is located in Geneva, the other one in Budapest, and in numbers of numbers, during these six months, we add a few more nodes. So, now we have 120,000 cores in our cloud. In average running, we have 11,000 VMs, and in total, we have 16 cells. We also changed the way in these six months how we deploy cells. Now, our new cells have around 200 nodes, and this means that if they are smaller, we're going to have a lot of them. However, if one fails the impact in the infrastructure is not so big. Also means that with smaller cells, we can have smaller control plane, which is much easier to bring back in case of failure. For this reason, we now are not clustering rabbit at child cell level. In fact, most of our problems in the past were related with network partitions in our rabbit clusters. Different cell types have different requirements. We have what we call three cell types. The compute one for CPU intensive tasks, for example, LHC analysis. The service cells that run web servers, databases, so on. And the critical ones that run very critical applications that we have like the radiation control systems. Now, we map one availability zone to a cell. This means that a cell is only one availability zone. We don't use aggregates anymore in our infrastructure. It turns out if you have thousand of nodes managing aggregates, it's really challenging. We still have big, big cells, the ones that we set up two years ago. We have cells with eight hundred nodes. Our biggest one has one thousand seven hundred nodes. We are looking, how can we split these cells? Possibly a next talk, next summit. So last summit, I told you about the all the motivation, problems, and challenges that we have at certain running cells. So what I want to do today is go through some of those problems and tell you how we are dealing with them. The first one is to prune over databases. So during these two years, in our cloud, more than one million VMs are created. Of course, we don't have resources towards one million VMs in one go. So for new ones to be created, others needs to be deleted. In average, we have eleven thousand VMs running. With the creation deletion rate between 100 to 300 VMs per hour. And over time, of course, the databases will grow. And this is not only a problem if you are running cells, but if you are running cells, you multiply this problem by the number of cells that you have. Because each one has a database. The problem is that when you delete an instance in OVA, it's only self-deleted in database. All the information is preserved there. At certain, we have the policy to preserve this information at least three months after the instance is deleted. After that, we can forget this information. We can delete it completely. So how can we do this? So OVA has the ability to archive deleted rows. However, for our use case and using cells, this is not really ideal for us. So we needed to build a small tool, basically, that goes to the top cell, to the parent cell, checks out all the instances that are deleted before a specified date, and then removes all the records related to those instances, and then goes to all the child databases and delete exactly the same instances that they have. So in this case, we keep consistency between both databases. When we delete, we really remove one VM from the top one. We also remove it from the child. Well, all the code is available on GitHub. You can have a look. And also, I wrote a small blog post explaining more of these works. So next, cell scheduling. So we have different cells. Each cell has different requirements, because we have different hardware, different location, and the network configuration in those cells are different, different type providers type, so on. So how can we make sure that a VM from a project is scheduled to the right cell? So child cells have the capacity to expose to the parent one capabilities. So we use that feature, basically, to inform the novicell, the novicell scheduler, the capabilities that the child have, and also we use metadata that the user can provide when it puts an instance. The other thing is novice scheduler, novicell scheduler has the same function as novice scheduler, so it supports scheduler filters. So what we did was to build a set of scheduler filters, like the selection of this data center, select an availability zone, that I provide our type, and map a project to a cell to help us on this. So the code again is available on GitHub. You can use it to write your own scheduler filters. Probably it will be a good help on that. Flavors management. So because different cells have different requirements, the projects that run at CERN all of them have specific needs. So all of them need special flavors. So how can we manage different flavors per project in a big infrastructure? Unfortunately we cannot use novice API directly, because when you create a flavor using the novice API that is connected to the parent cell, the flavor is only created there. It's not propagated to the child ones. Of course you can think, well, but we can also run of API in the child cells, as we do, and I can use novice API individually in each one. That doesn't work because the record ID that when you create the flavor in the top cell will be different in all the childs, and then you create a big mess because you think that you are starting a VM from the flavor small, and in fact you are creating a completely different flavor. So initially what we did was going through the databases and add all the flavors individually on them, but since we're starting to add more and more cells it starts to be risky and difficult to manage. So again what we did was a small script that goes through all the databases, selects a free record ID and use that and propagates the flavor in all the cells available. The trick here is to add the flavors in the child cells as public. If you do that then all the management could be done in the parent cell. So if you remove it then you don't need to mark this in your child cells or if you want to dedicate the flavor to a project then you only need to do this in the parent cell. Again you can check out the code on github. So increasing the number of cells also increases the challenge of keeping everything working. So how can we make sure that our cells are performing well before and after being deployed into production? So for this we're starting to look into rally so for benchmarking and also test functionality. We have multiple scenarios basically for the basic functionality for example create and delete the VM using nov and eC2api for example and we integrated all of these with our keyman infrastructure. The reason for this is that the UI that the that rally provides doesn't give us a historical view about the running tests. You only see the iteration. So integrated this with Kivana and for now we don't have alarming but will be nice in the future to have some kind of notification if a test fails, a scenario fails. We are looking to do this and probably we will use Spark for this. So this is one example about our integration with rally and our keyman infrastructure. We can see that in this example we have three cells and different scenarios here and we can immediately identify in the it map that we have which is running which is failing and you see that we have this per hour. Something that to do the rally UI at the moment doesn't provide us. Well, certain cloud doesn't look as bad as it looks here. Okay, also we can go deeper on this and see the duration of this scenario and what was the duration of each individual operation. So for example in the first case to boot a VM and delete it it was around 29, 28 seconds and you can see that most of the time here was to boot a VM and then to delete it. Okay, so I hope that this will help you when you are deploying your cloud infrastructure using cells. So next Matt we'll talk about cells like its rack space. So yeah, so just to give you a little background, so I work with our fleet management group at Rackspace. So I have several engineers that are tasked with sort of automating both the physical and the virtual nodes that make up our cloud and I'll talk a little bit more about that as we go through. Just quick background, so managed hosting company, it's been around for a while. At this point we have over 200,000 customers in a 120 countries around the world. From a cloud perspective we're still running six basic regions that span these particular locations that hasn't changed yet. Tens of thousands of hypervisors you can read the rest of the details growing all the time. I have one of the nice luxuries in that I kind of this constant stream of gear headed my way because we use it to make money so I know a lot of people struggle with capacity. Our struggles are keeping up with the flow of capacity coming in in a lot of cases. Most of our cells, the deployments run anywhere from three to just still about mid 30s I think in our largest regions as far as number of cells in a region and that's largely dictated by the size of the data center we have in each place and the growth of customers in those places. We size about 100 to 600, not so much on the 600 side anymore. I think it's when we first started some of our original standard flavors got about that high. We actually found some problems with broadcast domains. If tenants started doing bad things, so that and some and sizing of our IP blocks for our internal network that connects different products we've kind of brought that down a little bit. So most of them now run about 100 to 150 with our newer hardware models. We break up cells really based on hardware more than anything else. We offer several flavors to our customers. Some of the names are on there, the general purpose, the hyo, compute, optimize all those things. We'll have multiple cells of each of those in a particular region but that's the primary way that we break up our hardware. We are working on exposing some additional constructs around that, maintenance zones, so the idea that I can schedule instances near or far from each other within certain flavor types so that when we have to go do work we're not necessarily bringing down everything at one time. We do run a separate DB cluster for each cell. The other thing I'll point out, I think Sam mentioned they do the same thing. We run all of our control plane and it opens that cloud itself. We actually use cells within that one as well. They're for two reasons. One, we have multiple hardware types now available in our internal cloud. Secondly, we offer some of that space out to other groups at Rack Space to do their own testing or development work and so I want to make sure I can isolate the things that the instances that represent my control plane for the public cloud from say an IT group who's testing a new version of the billing system. And so to do that we use some of the same techniques to isolate cells within this private cloud to individual tenants. Cell scheduling, like I said, multiple cells within each flavor class. Also, we have in most cases multiple vendors and some of our different hardware types and so we try to keep a cell homogenous from a vendor perspective just so there's no quirks there. This is especially important for matching the CPUs exactly. We are starting to use lab migration more and more for internal maintenance work and this is block live migration so this is not with attached storage. So as we try to make that more and more transparent for customer standpoint we just don't want to run into problems with CPUs not matching exactly so that's why we we keep hardware as homogenous as possible. And then within each flavor class you're going to have different flavors offered much like Lermont talked about where the child offers up the flavors it supports. So for example in our general purpose offering we offer a one to four eight gig flavor class flavor within that class. Tenants are scheduled by flavor class first and then by available RAM. We don't schedule on any CPU information it's purely on RAM right now everyone in that flavor class gets the same DCPU allocations for the class. I would like to have scheduling based on IP availability I'll talk about how we get around that now but that's probably the one other thing I would love to see from a scheduler perspective because sometimes those two things don't line up well. And then just to point out there is a lot of work going on with Cells V2 around the scheduler itself. This is one of the the blueprints out there. There's about three I think on Wednesday when we have the large deployment team working group one of the sessions we're going to dive into Cells V2 blueprints and so this is one of them that we're going to dive into. It just has to do with where scheduler starts updating some information about the instance earlier in the process than where it does today but you can dive in more if you want to read up. So deploying a cell pretty similar. We run everything like I said in the cloud so we lay down the control plane with an ansible playbook the DB pair, the Cells service scheduler rabbit the key pieces we're still testing out conductor so we don't have those integrated yet. We run a playbook as well that populates the the the same flavor stuff that we talked about earlier and then we run a separate playbook that bootstraps our hypervisors. One thing I'll point out here I'll talk more about it on Thursday in another talk but we actually run our compute nodes as a VM so from a node perspective that gives me twice as much to deal with and extra headaches but it's a nice isolation piece so worst case I can blow away my compute node and not affect the underlying instances except for you know taking action against them. So the bootstrapper in our case pretty much is responsible for getting that VM created getting the updated code pushed on it laying down the right version of OVS on the hypervisor some of those things and then pushing routes and whatnot that we need to talk to other Rackspace products. After that we provision IP blocks we test and then we link it up through another playbook which you mentioned getting the some of those things backwards we've actually had one case where the Ansible assumed things in a way we didn't realize at one time and so it actually linked up one cell as the global and caused some interesting shenanigans as things tried to schedule to the wrong place so that's almost as dangerous as the flavor classes getting advertised wrong. All right so we too had to deal with purging the DB nodes we had we were a little bit further along than CERN was when we started messing with it but just to give you an idea so our largest regions right now are running up around or over 50,000 VMs at the moment and some of those regions though we have thousands of VMs being created and deleted in an hour in fact I would tell you that a lot of that is y'all's code submissions to OpenStack drives a lot that in some of our regions so obviously we start stacking up those deleted instance records pretty heavily. We also have a 90-day retention period for deleted instance information and I looked this morning even in my small regions I'm still sitting on about 132,000 deleted instance records and all the metadata to go with them in that 90-day period so but what does that mean? Well here's an example of the API latency in one of our larger regions you can see that back in November it was running you know we kind of reached a tipping point where it was starting to slow down a little bit and then that sharp drop of the end was us successfully purging all the global deleted instances as well as the cells and I see Jesse sitting out there now so he remembers those days so anyways there are benefits beyond just the data that gets that you get taken up you can see actual API improvement getting all that still data out testing our cells so we also have to figure out how to test cells that are being added into a working environment all the time the primary way we test the new cells with the bypass URL essentially you just add an API node in our case we call it we use an admin API node supports a few extra functions but we add a separate API node temporarily from the time we finish provisioning the cell to the time we link it up and all of our QE tests are run against that API node so we're not actually having to put it into production so to speak until we're comfortable with it the trickier part is when we're adding additional capacity to an existing cell because there's just no concept from a host perspective of I'm sort of kind of in production but I'm not really ready for primetime so there's some things we can do there's targeting filters all those kinds of things we actually have a change I wasn't able to find the code on my schedules off after all the Venom Shanae against last week but a couple of my engineers are working on a filter that will actually modify the targeted host filter to still build even if the host is disabled and so our hope is that we can use that to actually get all of our host provisioning completely hands off our goal is hopefully by this year a cabinet can roll in and be turned on and as long as a network switch is configured everything from there after from installing the operating system which is in server today to all the boot strapping to all the testing to all the linking up happens hopefully hands free that's our goal is this year we'll see how far we get and so we're going to need those host host based hints to work even on disabled hosts for that to happen the other big thing that I thought I'd bring up from like sharing pain and agony is the concept of disabling a cell it does not exist you're kind of linked up or you're not and when you're unlinked there's a lot of problems for the existing instances that live there the reason this is important us is actually IP space I mentioned this earlier so we've actually had cases where the amount of available IPs depletes faster than the amount available RAM which isn't a big deal except that unfortunately right now at rack space to get additional IP blocks added does require manual steps by other teams to go update routers and whatnot I haven't I don't have the ability to completely do that programmatically and so used to what we would do is we would just we have a resolver service that picks up alerts and handles them for us and it would just go and it would add it would add an offset of a big number right and that works most of the time but there are still cases where a brand new cell had so much available capacity that even with an offset or even with the weight high it would win the calculations and it would just keep hammering that cell with builds and so what we did was we cooked we cooked up and this is kind of a dirty hack right now we're trying to come with a better way but we cooked up a special filter that instead of specifying a big value we specified a specific value so anytime we now put minus 42 on a cell our scheduler knows don't send anything stop what I'm doing move on go somewhere else so I pasted the bulk of that code in here the slides will be up later I think we'll post them yes yeah again it's not it's not the cleanest way to do it but it allows us to then allow to have that same resolver service pick up an IP alert drop the minus 42 or for any reason if we detect any massive anomaly in a cell our ops guys can go in and add the filter the weight right there and then we know that at least no new builds are going to try to go there till we sort it out and so here's a shot from that same management application where you can see a cell weighted the other day minus 42 based on IPs more than anything so neutron and cells this will be the very highest level overview of that but essentially we have neutron running and we do run it with cells that's largely the based on the fact that we wrote the quark plug-in the real drivers for the quark plug-in had a lot more to do with the fact that we provide we have multiple provider networks per instance by default we give them a public and a private internal network also we bridge those things straight to the to the outside so when you get a public IP under instance it's not going through a firewall or something it's bridge the outside but to pull it off with the with the quark plug-in and I've got the link there for the code I'll talk a little bit more about on Thursday I think but we give we borrowed some ideas from back when we had quantum and melange so we have a tenant for each cell each cell becomes a segment from this perspective of neutron our subnets which for us represent an IP block are assigned to those segments and then NOVA will request ports from both the public IP blocks the private IP blocks and actually we allocate the max dynamically as well so it'll pull all those things from from neutron based on this setup so I think I don't know how much time we have but in a couple one minute I think one minute yeah so one question now yes sir what's some of the major issues you've ever faced in one village or anything like that from which in general in general we'll be good so yeah also messaging was probably the most recent one it would knock over rabbit quickly I mean from my perspective we try to pull code quite often and so usually it's finding the things that weren't tested at any kind of large scale and figuring out how to deal with those quickly once we've deployed it but I don't know what you guys have added to that I haven't yeah so we can talk more offline if you want to see cool thanks thank you thanks guys