Well, anyway, here we are in chapter 3 of the book of Micah. We're going to be looking at these verses. I'll be honest with you, man, as we're looking at this, I was thinking, well, there's only 12 verses. I can go through that and maybe even do, you know, chapter 4 or 2. That's not going to happen. So, you'll see why in just a moment. Beginning at verse 1, Micah, chapter 3. Looking at verse 1 and reading to verse 3, Micah writes, and I said, here now, O heads of Jacob, and you rulers of the house of Israel, is it not for you to know justice? You who hate good and love evil, who strip the skin from my people and the flesh from their bones, who also eat the flesh of my people, flay their skin from them, break their bones, chop them in pieces like meat for the pot, like flesh in the cauldron. And so, once again, we have a very cheery introduction to a chapter. As we look at this passage, this would be the third message Micah brings. And notice with me here that he is speaking against the leaders of Israel, and he's speaking concerning their sins. So he's speaking against the leadership of Israel, and the leadership of Israel actually consists of the princes and the prophets and the priests. And there's a religious theocracy is what it's called. The nation of Israel was a theocratic nation. And so we need to remember that a theocracy, for those of you who perhaps aren't familiar with the term, a theocracy is a system of government where God is recognized as the ruler. And he has priests who govern underneath them. It's a system that derives all of its authority from the God that it serves. And so Israel in its Old Testament form is a theocracy, a nation under God. And so the king and the priests and the prophets were all rulers. They were all intended by God to wield authority in that nation. And so the kings of Israel, the kings of Judah, were to serve under God as his agent as he ruled the nation. And they were expected to observe his covenant and his laws. You see, princes, prophets, and priests were all subject to the word of God. There was no such thing in a theocracy. There was no such thing as a doctrine of separation of church and state. There was just a theocratic rulership, God ruling through the prophets and the priests and the king. So when you look at the way that the kings would rule, for example, the king was to have a copy of the law and he was to keep the law himself and actually govern according to the law of Moses. Deuteronomy 17 verse 18 says, when he takes the throne of his kingdom, he is to write for himself on a scroll a copy of this law taken from that of the Levitical priests. In 1 Kings chapter 2 verse 3 it says, keep the charge of the Lord your God, walking in his ways, keeping his statutes, his commandments, his ordinances, and his testimonies, as it is written in the law of Moses, so that you may prosper in all you do and wherever you turn. And he was saying that to the king. So the king was to keep the law. He was to have a scroll and he was to rule by that, a theocracy. The prophets who were genuine were subject to God's word and they were to declare it to the people. So when you look in the role of the prophets, the prophets of the Old Testament were called by God to speak for him and often would lead on his behalf, especially during a chaotic and often violent period in Israel's history. So the prophet was to speak God's word to man, even as we have seen here in Micah and we recently saw in Amos. The priests were also subject to the word of God and they were to serve under its declarations. They had various duties in the tabernacle, temple, as well as in their duties to worship the Lord. But they had the responsibility of teaching the people the word of God. In the Old Testament book Leviticus chapter 10 verse 11 it says that you may teach the children of Israel all the statutes which the Lord has spoken to them by the hand of Moses. He was speaking to the priests. In Deuteronomy 3310 he said they shall teach Jacob your judgments and Israel your law. And so the law, the law of Moses was to rule, was to govern.  Now all of this would show that they were thoroughly acquainted with God's word and they were to lead by it. But when you are, here's something for you, when you are thoroughly acquainted with the word of God, you are also under stricter accountability. The more that you know, the more you're responsible for. That's just a fact. The Bible tells us, for example, speaking in a New Testament sense to teachers, James says it like this in chapter 3 verse 1. He says, Dear brothers and sisters, not many of you should become teachers in the church for we who teach will be judged by God with greater strictness. So the more that you know, the more you're responsible for. So the king who was to have a scroll of the law, the prophet who was to know the law and declare the mind of God to man, the priest who was to do his duties, his priestly duties under the law in regards to the law and holding fast to the law, etc. All of them had tremendous responsibility and thus God would speak to them when he was bringing judgment. You see, in this passage, Micah brings a word to the leaders of Israel and Judah and he says here that he is calling them to hear. That's how he begins, which is an interesting phrase. He said, Here now, O heads of Jacob, and you rulers of the house of Israel. Is it not for you to know justice? Here now is what he's saying to them, listen up, listen to what I have to say. So he's bringing a word to the leaders of Israel and Judah. He's calling them to hear. It's a call for them not only to hear with the ears, but to obey the word of correction as well as to heed the warning. Now this term here is something that you find very often, especially in the New Testament. For example, remember how Jesus in Mark 716 says it like this. He says, If anyone has ears to hear, let them hear. Or you see in James 1.22 where it says, Be doers of the word and not hearers only deceiving yourselves. So when God calls them to hear, it's not simply so that they can say that they have heard something said. It goes much deeper than that. It's to hear and to obey. It's a putting to practice what I'm saying. So it's not enough for me to be able to repeat what's been said. It is that I, in my change of behavior, demonstrate that I believe what's been said. So you will always demonstrate your genuine faith by the life that you live. So there, you know, because a lot of people say, Oh yeah, I'm a believer in Christ. Talk is cheap. There's this old saying that we used to use practice what you preach. It's what you are. Speak so loudly. I can't hear a word you're saying. And so you preach the gospel at all times and whenever it's necessary, you use words. Because my children, for example, in my home will worship the God that I serve, not that the God that I say. Because they watch the way I live and that helps them to determine what is truth. And so if I say you shouldn't lie, but then the phone rings and they say, It's Grandma. And I say, Tell her I'm not here. Right. They see it. If they see me at the store and they give me more change than I'm supposed to get. I gave them a 10, but they give me a change for a 20. And I put it in the pocket and walk out with the extra money and my kids see that. And I say, Isn't God good? He just blessed the family.  And rightly so, because that's what I am. So I can say all day long, love the Lord, love one another. But if I'm unkind to them. The only way they'll define love is how I treated them and treated their mom. And for them in their mind, that is love. That's how it's done, right? Daddy said he loved and this is what he did. Mama said she loved and this is what mama did. And that's how you learn. And so in Christianity, it's always transformation.  I will give to you a new heart. I will move from you that that heart of stone. I'll replace it with a heart of flesh. I will write on the tablets of your heart, my law. So from the inside, you'll be transformed, you'll change. And so that's what it's all about. And that's why when Micah's bringing a word of judgment, he begins with the leaders.  That was under the law of Moses. And he's saying to them, you need to hear. You need to hear what I'm saying because you're not listening and you're not obeying. Listen how he says this in verse one. He says, is it not for you to know justice? Again, it's not like you go to law school, you learn the laws and all. This is, this is a justice that you're thoroughly acquainted with. It's a justice that flows from you. To know justice is more than simply to be able to cite a certain section or a certain coat. To know justice is to have a personal acquaintance, to know it from the inside. So you're to be judging according to the law. But the problem is, you are violators of the law, is what he is saying. You're not administering judgment impartially. Because you do not live by it yourself. You are above the law. The law is for other people. Now, I don't have any current illustrations to use as it relates to that. I want you to think about that for a minute. So you have found people guilty. But in fact, you are guilty also. You are making a judgment. Like Paul says in the book of Romans when he says to the Jews, do you say to someone, don't steal, but you steal yourself? You're saying one thing and doing another. And so what he's doing is he's saying this is pointing out your sin. This is why you are guilty. Look at what he says in verse two when he says, you who hate good and love evil, who strip the skin from my people, the flesh from their bones, isn't that gross? That sounds, that is such a powerful picture of causing pain to others. You see, he says the leaders of the day, they loved evil. And because they love evil, they don't judge righteously. You see, you're supposed to love righteousness to judge righteously.  And they shall judge the people with just judgment. You shall not pervert justice. You shall not show partiality, nor take a bribe. For a bribe blinds the eyes of the wise and twists the words of the righteous. Again, it is difficult to judge righteously when your standard is unrighteous. He speaks of stripping, stripping the skin from my people, flesh from their bones and all. That would speak of robbing the people of everything that they have. It speaks of basically devouring them. He says in verse three, who also eat the flesh of my people. It's all a portrait of devouring people, of robbing them of everything that they have. Psalm 14, verse four says it like this, do all these evildoers know nothing? They devour my people as though they're eating bread. They never call on the Lord. If he was to be saying something in a way that maybe we would say it, he'd say a good leader is to share the sheep, not skin them. And that's what you're doing, you're ripping them off. Jesus, and for this I will bring judgment. See, the judge was to judge impartially. They were to render true justice, but instead they were using their positions to devour the poor. It reminds me of what we saw in Amos in chapter eight, verse four, where there it said, hear this, you who swallow up the needy and make the poor of the land fail. So they were devouring the people. They were taking advantage of them. They were becoming rich off of the poor. And he says, this is why I'm bringing judgment. Now in verse four, then they will cry to the Lord, but notice what he says, but he will not hear them. He will even hide his face from them all that time because they have been evil in their deeds. Well, in the Old Testament, you also say, God also says they will cry out to me and I will hear them and I will deliver them. But in this case, I want you to notice, he says they will cry to the Lord, but he will not hear them. These people are going to be judged. See, the land is going to be invaded and God is not going to intervene on their behalf. It's interesting how he says he will not hear them. He will even hide his face from them.  He will not be found by them. In Isaiah 1, 15, when you spread out your hands, I will hide my eyes from you. Even though you make many prayers, I will not hear. Why?  Ezekiel 8, 18, I also will act in fury. My eye will not spare, nor will I have pity. Though they cry in my ears with a loud voice, I will not hear them. I'm not going to act on their behalf. I'm not going to respond. Psalm 66, 18, if I regard iniquity in my heart, the Lord will not hear.  Someone said, just as those oppressed by them had cried for mercy and they would not hear, so shall their prayer be rejected. It's rejected because it is the cry for deliverance from pain, not that of repentance for deliverance from sin. There are people, I've had more than one conversation over the years with people who got caught doing something and before you know it they become very spiritual. Can you pray for me? For what? Well, and then they tell me, and of course I'm always ready to pray for somebody. Of course, and I want God to move in your behalf. But is it because you want to be relieved from some pain so that you can go back to it again? Or is it because you're repenting and you want to turn away from it forever? That matters. That matters a lot. Because sometimes people, simply because they got caught or they're gonna go before a judge or because they have some problem that they're having to deal with. They want to be delivered right now. It's just another way of saying, take me out of my pressure so I can go back immediately to what it is I used to do. It reminds me of that thief on the cross. There were two thieves on the cross next to Jesus. You know the story? You have the one who is saying to him, Lord, remember me when you enter into your kingdom. But you have the other, and they're both actually saying this at one point. He was saying, if you're really the Lord, take us down off this cross, save yourself and save us with you. And when I go through that passage, I often will say this, that that was a cry to be delivered from pain.    Well, yeah, if you have the power, take us off. But for what? So that I can continue doing what I did before. I'm an unrepentant sinner. And I want to go right back to what it is that I used to do. I've spoken to many people over the years. Many people, I've prayed with many people. That's how you learn these things in a practical way. Who will say, I really need to be delivered from such and so. And we pray with great sincerity. And there have been times when we've seen what appears to be the mercy of God on their behalf. And three months, four months, five months later, they're back in the same sin they were in before. Because they never repented. They just got caught or they were afraid. And sometimes people can become real models of Christianity until that court date. Then they hear the, OK, you're getting probation. Then they go back. Because they didn't repent. They regret it. They were concerned. They were afraid. There are some very strong Christians in jail. But when they get out, that's a test of whether they're really free in Christ or not.  We call them J House Christians. And I'm not knocking them, to be honest with you. I'm glad that they go to ministry there in jail. I'm glad that we have a prison ministry here. They go to various prisons. And I thank God for it. But they can tell you firsthand that a lot of the fellows who are going to chapel just want to have a good appearance before those who are in charge so that they can have a little less pain, will say, in prison. That's just the fact. And so it's not just regret that they're supposed to have. It was repentance.  But he's not going to hear. He will even hide his face from them at that time. Why? They have been evil in their deeds because they're unrepentant. And they're going to continue in such a way. Verse 5, thus says the Lord concerning the prophets, who make my people stray, who chant peace, while they chew with their teeth, but who prepare war against him who puts nothing into their mouths. Therefore, you shall have night without vision. You shall have darkness without divination. The sun shall go down on the prophets. And the day shall be dark for them. So the seers shall be ashamed. And the diviners abashed. Indeed, they shall all cover their lips, for there is no answer from God. So let's look at that for a minute, shall we? Notice in verse 5 how he says, thus says the Lord concerning the prophets, who make my people stray. When you get into the New Testament, one of my more favorite portions of scripture, and we'll be there. It's in Matthew 24. We're in Matthew 19 right now. It's not Sunday mornings. So when we get to Matthew 24 in two years, you'll see this. If I rush, Jesus in Matthew 24 says, take heed that no one deceives you. Take heed that no one deceives you. He repeats that three times, maybe up to four times, in one chapter. If he says something once, listen. If he says it twice, three times, four times, there's an emphasis. He says, take heed. And he speaks about deception. Because when he says, take heed, he is putting responsibility on my shoulders to make sure I'm not deceived. That's a real problem today in the 21st century church, because many Christians are not taking personal responsibility for their own spiritual well-being. And they're not listening with discernment. They're not listening to Bible studies with the intent of understanding the ways of God. Many Christians today, sadly, but this is true, go to church for the entertainment factor, for the enjoyment factor, for the self-satisfaction, it's all about themselves. They're not going to be equipped with the word of God because the word of God isn't really rightly divided. What they're getting is the personality of the proclamer, or the methodology, or the environment. This is a fact, and you know this to be true. That's what's happening. And so somebody who is entertaining can cause people to be led astray because the people are not in the word studying, praying over, meditating, they're not growing. See, Christianity is not school. Sometimes people say, you know, I feel like when I go to church, I'm going to school. It's not school. What Christianity and what Christian teaching is is equipping the saints for works of service. When you get into the word, you actually find the ways of God. So each one in this room, starting with me and all of us here, and any who will listen to this study right now, every one of us has a personal responsibility to spend time in the word in order to be able to be equipped for works of service, to be able to give an answer concerning the hope that lies within you with meekness. And there are those who will question you, why do you believe in Christ?  There are so many different doctrines that are out there right now, saying that they're giving you the truth, and then you get confused, and I understand that, because there are some things I've never heard myself, and they'll say something, and you go, I've never heard that before, that's a new one on me, though it's never really new, it's just repeated error from a different time, that's what it is, always. And so the bottom line is, these people were not well trained, they're not getting the word of God, the priests were not teaching properly, the children of Israel are not equipped, they're not discerning, and so he says, these people, Israel, are being led astray by you.   I had someone write me one time, and he was upset at me one time, but this one time. And Christians don't need to repent. And why do you say that we need to repent? Christians don't need to repent. We repented, we're saved, and everything's covered by the blood of Christ, and our lives just proceed, and I said, no, repentance is a daily thing. Repentance is an ongoing lifestyle. The word repent is that Greek word metanoia, it speaks of changing your mind, because it's the change of your mind that results in the change of your behavior.  Your life is a life of repentance, it's not a one time only, but it's a habit of it. That's how it works. And it isn't bad, by the way, to wake up in the morning and say to God, God be merciful to me, I am a sinner, because I already have sinned the minute I opened my eyes. And I will go through the day dealing with my flesh. So it's an ongoing thing.  That's how it's gonna work.  They calm people by telling them that good times are on the horizon. And what they did is they produced a false hope and a false peace. In Jeremiah 8, 11 it says they have healed the hurt of my daughter, the daughter of my people, slightly saying peace, peace, when there is no peace. Lamentations 2, 14, your prophets have seen for you false and deceptive visions. They have not uncovered your iniquity to bring back your captives, but have envisioned for you false prophecies and delusions. They haven't said to you and cried out to you, get right with God, get right with God. And he says, because they're not, you have a false peace. These false prophets were actually making the people stray instead of repent. You see, as Syria was preparing to invade from the north, but the people were being lulled into a false peace through these false prophets. Now they have a longing for peace, but not the heart to repent from the evil he practiced. And that would keep them from experiencing any true peace. Isaiah 57, 21 says, there's no peace there, my God, to the wicked. There is no peace to the wicked. You see, peace cannot be experienced as long as man is at war with God. Peace can only come when man is reconciled to God. And then when he's reconciled to God, he's reconciled with other people. So you can't have peace with man until you at first have peace with God. In Romans 5, 1 it says, being justified by faith, we have peace with God through our Lord Jesus Christ. And so do you wanna have peace with others? You come to peace first with God. And that's how it works. He says in verse five, while they chew with their teeth, but who prepare war against him who puts nothing into their mouths. The false prophets were biting. It's interesting the word chew there is also the word bite and the word bite speaks of a serpent. The serpent, a snake that would latch onto you and inject poison. And so the false prophets bite with their teeth. What they're trying to do is they're trying to take out the true prophets. You see the true prophets were at war with the false in the sense of the messages were combating. And so they didn't give in, cater to them and didn't cater to their lies. And that's what they're upset about that you're not giving us honor. Therefore, he says in verse six, you shall have night without vision. You shall have darkness without divination. The sun shall go down on the prophets and the day shall be dark for them. So the seer shall be ashamed and diviners abashed. When's the last time you used the word abashed? Did you say it today? No, I was really abashed. The word abashed means confounded or confused, abashed. I don't use that word, but I actually had to look it up because I don't know what the word abashed means. I shouldn't have told you that. I should have pretended I'm a genius, but I'm not. Indeed, they shall all cover their lips. When it speaks of covering their lips, that's a demonstration of mourning or sorrow and he's speaking concerning the judgment that will come upon them. They're gonna make fools of themselves is what he's saying. Now, when he says in verse six, they shall have night without vision, they're gonna have judgment. They're gonna be dealt with. When it says without vision, they have no prophetic revelation that's gonna come to them. So these false prophets will be judged and God is not communicating to them. They're not gonna receive light from God. They will be kept in the dark because God withholds revelation. When he says in verse seven, the seer shall be ashamed and the diviner's abashed. The word seer speaks of... The word seer is a word that is very often used as a synonym for a prophet. So you can have them called seers in scripture and they're also referred to as prophets. But one of the things about a seer and what would designate them as seer is that they saw visions and that's why the word seer is used here instead of the word prophet. They were ones who saw visions. Diviners. Now diviner's speaks to us of false prophets because they're divining or trying to find the will of God when God is the one who freely gives his will to his true prophets. So a diviner using the practice of what is called divination, trying to divine is a false prophet because God is the one who reveals to the true prophets his word.  They're going to become ashamed in front of the people instead of being prospered by them. In verse eight, this is very powerful here. This to me is, well, I'll just share with you. Micah says, But truly I am full of power by the spirit of the Lord and of justice and might to declare to Jacob his transgression into Israel his sin.  He's contrasting a true prophet with a false prophet. False prophets have no light. False prophets have no truth. False prophets have no revelation from God. False prophets have no power from God. So what he's doing here is he's contrasting the true with the false. And his declaration to me is extremely powerful. Listen, his message was not popular and his message was not welcomed but he knew that it was a true message from God.  This is one of those places where I kind of like wax for a while. There was a time years ago, years ago that you didn't have Christian television programs. You had occasionally Christian television programs. But they were usually maybe on a Sunday and they were usually something like just a denominational program you may have, you may have something on the radio. When I first got saved there was nothing like what you have today at all when I first got saved. You didn't have one. You didn't have what is called contemporary Christian music. That did not exist. We didn't have that. What you had was old time radio programs with what we used to call a dollar a holler preacher.  You pay the dollar, you can holler. That's a dollar a holler preacher. They paid for radio time and they'd holler for a half hour and that's what they did. So we call them dollar a holler. And so you could drive and you could look all you wanted on the radio dial because nobody listened to FM. There's no such thing as satellite. And so I'm speaking ancient history now for a moment but I'll get to the point in a second. There just was not a lot for you to find. And some of you are old enough to remember going through the hunting through that. You're AM single speaker radio trying to find something. And you just couldn't. But on Sunday nights, you might be able to find some program. And when you listen to that program, you would say, oh, I'm getting studied. But every once in a while, you'd have somebody like an old Roberts or somebody like that yelling out about healings and things. And he might be on a Sunday afternoon or whatever. And that was pretty much it. You didn't have a lot of selections. You didn't have a lot of opportunities to get a lot of things. And I can remember when I first got saved that I was a brand new Christian and I was driving in my car. It was an evening. And I said, you know, God, I just want to sing. Because I was taught you sing to the Lord. You know, I got saved and was going to Calvary. And there's a lot of worship and praise. And I just wanted to sing. It was just me and Jesus. It was night. Nobody was watching. And I said, Jesus, I want to sing to you. But there's nothing. And I was going through the radio trying to find a program that I could stop and hear songs about the Lord. And there was no contemporary Christians. So you would hear an organ and this and that, a 90-year-old woman warbling, you know. And that was about it. I mean, so there was nothing there. And so you know what I landed on? I landed on George Harrison's song, My Sweet Lord. That was it. So I just sang along with him. And when he said, Hare Krishna, I said, hallelujah. I just kept on, hallelujah. And no, Krishna, we don't need Krishna. It's just hallelujah, Jesus. That's how it was. What has happened in our day? And I've been watching this as a minister of the gospel. I've been watching this. And hopefully, this will make some sense to you that. And I tried to share this with people recently. What happened is I got saved. And I was brought into the Word of God. I actually believed that God actually could speak. I actually believed it. I believed it to this day. Of course, the Bible's God's Word. I was brought into a place that taught me the Bible. That was Pastor Chuck Smith. And that was a guy named Lonnie Frisbee. And they would speak about the Bible. And I would listen to what God's Word said. And I was equipped for works of service. I began to believe that God's Word is truth. So the things that I didn't find in scripture, I just no longer held fast to.  I just said, well, if it's not in the Bible, I'm not going to believe it. And that's kind of my journey. That's what my journey has been up to this point. But it had to be found in scripture. To this day, if I don't find it in scripture, I'll put it on kind of like a shelf to be discovered later. And I go with the things that I know. That's how I am today. With that said, things have changed. And this is not a complaint. This is an observation. This is the truth. And some of you understand, and others perhaps won't. Some care, some don't. But it's truth. You don't go to church to be entertained. You go to church to be equipped. You don't sing songs because they're cool and all, though I do like contemporary music. You look at the words of the songs, and you determine whether those actually bring honor to God or not. You're discerning in all things. And what we have today is popular preaching by those who give 40% of the message about themselves, and then the last 60% don't actually even handle the word of God. That's what's big today. It's the look. It's what I personally get out of it. And what happens is you lose Jesus in the mix. When Micah says in verse 8, truly, I am full of power by the Spirit of the Lord, that's a strong declaration. It reminds me of what we had earlier read in Amos when it says, Amos answered and said to Amaziah, I was no prophet, nor was I a son of a prophet. I was a sheep reader, a tender of sycamore fruit. The Lord took me as I followed the flock. The Lord said to me, go prophesy to my people Israel. You want to be used by the Lord? You need to know who you are in him. You need to know who you are in him. You who don't know me don't know that I was a 23-year-old preacher. I was 23 years old. And I'm teaching my dad who's 47. And I'm teaching my mom who's 43. And I'm teaching a group of adults when I'm just a young boy myself. And so I didn't know a whole lot, but I knew this. I knew God's word is truth. And I knew I was called by God. And I taught and preached with a confidence not in myself, but in him. And that's what's really important. And that is missing today in many pulpits. There needs to be a sense that what is being said is not that person's testimony every time you hear them. And it's not the joke, and it's not the fun or the fun facts. It is Jesus Christ. And that's why when you walk into our main sanctuary, that's why you see we would see Jesus. Because that's what sets you free, and that's what keeps you strong. That's what does it. And we don't understand that today.  If it isn't fun, it isn't worth the effort. If it isn't in a 15-second soundbite, my concentration only lasts eight minutes. Those things are true. That's what's going on right now. Because of the usage of so many technological devices, the average attention span for people today is down to eight minutes. Did you, that's a fact. Did you know that in seminaries today, they're teaching the new preachers who are going to take the church into the further on in the 20th, that they're teaching them to give 12-minute to 15-minute messages? Did you know that? 12 to 15 minutes. Because they're saying that you can't listen for more than 12 to 15 minutes. Now some of you can't. I catch you sleeping all the time. Or texting somebody. How hungry are you? How hungry are you? And how important is God to you? That's the question. And you've got Micah saying, I'm full of the power of the Spirit. These false prophets are liars. They walk in darkness, and they want me to change my message. Do you think that John the Baptist, if he were alive today, would be invited on some of these TV programs to preach? Think about it. Do you think Joel Osteen would ask Paul to come and speak? I'm not kidding. I'm serious. I'm serious. Do you think they would be asked? They wouldn't. They wouldn't. Why? Because expounding a passage of Scripture is not entertaining. And if you give the whole counsel, there are blessings and there are buffettings. There are God saying I'm going to give you this, but I will also deal with you in this way. That's why I am so grateful to the ministry of Pastor Chuck Smith because he taught us to go from A to Z, give the whole counsel of God. Because that way, I can be one day like Paul, I'll tell you this. When I finished this book, I can say what Paul said when he said, I have not shunned to declare unto you the entire counsel of God. This is the last book of the Bible I've never taught. And I can finally go like this and say, I did it. I kept the faith. I did it. Jesus, I taught the whole counsel of God. And to me, that was the most important thing for me as a young pastor, teach the word of God from cover to cover. And that's what we do here, to know the Lord. That's the bottom line. That's the bottom line I am telling you. And that's what he is saying here. He's basically simply saying that, I know who I am filled by the power of the Spirit of God and of justice and might. And what does he do to declare to Jacob his transgression in Israel, his sin? And he's saying, and I am not shunning to declare to you that there is sin in the camp and it has to be dealt with. Now hear this finally. You heads of the house of Jacob, rulers of the house of Israel, who abhor justice and pervert all equity, who build up Zion with bloodshed, Jerusalem with iniquity, her heads judged for a bribe, her priests teach for pay, her prophets divine for money, yet they lean on the Lord and say, is not the Lord among us? No harm can come upon us. Therefore, because of you, Zion shall be plowed like a field, Jerusalem shall become heaps of ruins in the mountain of the temple like the bare hills of the forest. What a powerful thing to say.   Under you, the guilty have been declared innocent. The innocent have been mistreated. In verse 10, he says, who build up Zion with bloodshed, this speaks of the southern kingdom, Jerusalem. And he's condemning them for their greed. These people built mansions on the wealth obtained through the condemnation of the innocent. In verse 11, when he says, our heads judge for a bribe, every class was corrupted by greed from the rulers to the priests. They were all doing everything they could for personal profit. This existed during the time of Christ, by the way, the Pharisees were guilty of profiting from their positions. In Mark 1240, it says they shamelessly cheat widows out of their property and then to cover up the kind of people they really are, they make long prayers in public. Because of this, their punishment will be the greater. That has continued, by the way, to this day in the church. In 2 Peter 2, the apostles said in verses one through three, there were also false prophets in Israel, just as there will be false teachers among you. They will cleverly teach their destructive heresies about God and even turn against their master who bought them. There's will be a swift and terrible end. Many will follow their evil teaching in shameful immorality because of them Christ in his true way will be slandered. In their greed, they will make up clever lies to get hold of your money, but God condemned them long ago and they're destructing us on the way. There was a guy who was caught in his quote unquote healing services. His wife was reading prayer requests from people and they had descriptions of the people and she would say this person is seated in a certain row.  Is your name Beverly? Yes, do you have leukemia? Yes, and then he would say, well, God, this is your moment and he would take the money and he was exposed by the amazing Randy. That was his name, the amazing Randy. He exposed this guy and you know what that guy did? He's back at it right now. He's back at it right now. You can see him on TV, I won't tell you his name but I will tell you he's not far from where we are right now. Yeah, doing it to this day. They make merchandise of you. They will use your money. That's what he's saying. That took place then, it take place now. Paul gives us insight into his ministry. What is a genuine minister? 2 Corinthians 2.17, Paul said it like this, we're not as so many peddling the word of God that as of sincerity, but as from God, we speak in the sight of God in Christ. He said in Acts 20 verse 33, I have coveted no one silver or gold or apparel.  This meant that they didn't believe that Micah's promise of judgment would harm them. But according to verse 12, destruction will take place and it did. He said Zion shall be plowed like a field, Jerusalem shall be plowed. The destruction took place under Nebuchadnezzar. It was completed by Rome. The Roman army reportedly plowed up the foundation of the temple with a plowshare. Complete judgment fell upon them even as God had declared. He said it would happen. Jesus said, do you not see all these things? I say to you, not one stone shall be left here upon another that shall not be thrown down. And that's what happened.  There is a tremendous responsibility on the part of teachers. We stand before God to receive stricter judgment. The one whom God says I will look at is the one who trembles at my word. This is the one I will regard. This is the one I will look to. You want to be used by the Lord? Love his word, spend time in it and determine to do it.


Sure, my name is Steve Melner, and this is... Hi, I'm Jeff Liggett. Nice to meet you. We both are part of the Red Hat CoreOS team at different levels here. But we're going to be going over what is Red Hat CoreOS. So before we get started, I have a couple of questions for the audience. So first off, how many people here have used Container Linux, a.k.a. formally CoreOS? Okay, we've got a handful of folks. How about how many people here have used Atomichost, whether that's Fedora, Rel, Centos, okay? Slightly more people. Cool. Now how many people here have installed OpenShift? Okay, so not used, but actually installed it. How about Tectonic? How many people here have actually played around with and or set up a Tectonic cluster? Okay, we've got one person here. Okay, so these questions are just a gauge to see how deep we should be going into some of these topics, but... We'll move on forward here. So what we expect you to learn from this talk, when and why did Container Focus Distribution start appearing? A history of Container Linux plus Atomic, so the acquisition, what kind of happened after that? The use cases or use case of Red Hat CoreOS, depending on how you look at it. And foundations for the following talks that are going to happen in this room. So we have them listed up here, of course you can see it on the schedule, but there's going to be a couple other ones that come off directly after this talk that are going to be in the same room around Red Hat CoreOS slash immutable host-related items. So we're hoping to set up a foundation so that you can follow those talks as well. And the plan is for this one to be very high level, hand wavy and not too midi-gritty. And then later on, other speakers are going to be going into deep detail about things like ignition and other technologies. Exactly. So this is kind of where it started here. So before there was a CoreOS, most people were utilizing containers, and at that time that was pretty much docker on general purpose operating systems. So we're talking about RHEL, we're talking about Fedor, we're talking about other distributions. And this makes total sense because that's what we had at the time. However, once CoreOS popped up, it became really, really obvious that having this larger infrastructure to essentially run your applications on a host that you really don't want to be modifying, kind of makes sense to have this. So if you think about it from the point of view of travel, you have a lot of options. So like here in Brno, you can get to and from the conference a lot of different ways. You can take public transportation, you can get in a car, you can even ride a bike. All those things make a lot of sense. But you have other options as well. You could use a seaplane, you could drive a tank. These things are options, right? But they kind of don't make sense for those use cases, or at least they don't make as much sense for those use cases. And that's kind of what we have here. We have container focused operating systems, the first being here of CoreOS. So I believe this was actually the very first release of CoreOS. Version is 94.0.0. Does anyone know why it was 94.0.0, not 1.0 or 01 or anything like that? Any ideas? Other than Andrew, but I will call on Andrew if nobody else knows. Go ahead Andrew, why is that the case? That is correct. Day 94, this was the first release, right? And it really, I mean it took off pretty quickly. Once people started playing around with the idea of having this immutable host to run their containers on it, have very light management that they actually had to do, it was simple. I mean it really blew up. It just went everywhere. So we're going to talk a little bit about the projects and the tooling that went around with CoreOS. So not necessarily directly in CoreOS, but either directly in or around those things. So the first one I think pretty much everyone is going to be familiar with, and that's at CD. Distributed key value pair, used a lot with Kubernetes OpenShift and a heck of a lot of other stuff. I won't jump too far into that. Ignition, how many people here are familiar with Ignition? All right, cool. It's actually extremely foundational to both Container Linux and Red Hat CoreOS. There's going to be a talk on this a little bit later, but basically think of cloud in it, but way earlier in the process. So you can actually provision disks and do all that kind of good stuff that you can't do post installation with Ignition. And as we go through a update flow, you're going to see where this shows up in terms of updating Red Hat CoreOS and how it could be used for other processes as well. Mantle was also a big one, and this is one that we use in Red Hat CoreOS. It's kind of a collect all of tools that are used to either build items, to test operating systems, or to push images around. So I think there's about seven or eight projects underneath the Mantle GitHub repository. They share each other's code. That's why they're all in kind of one location. COLA is a big one. OR is a big one. COLA is kind of the unit testing of the operating system. OR is a big one moving these images around into different clouds and that kind of good stuff. FLEET, anyone here use FLEET? Have used FLEET, I should say. No, okay. So FLEET was an early one with CoreOS, and this was kind of scheduling system D units into a cluster so that you could run things, so it wasn't just like, you know, Ansible running something and getting system D units on all your machines and running something, but you could actually schedule where things were running within that cluster. And it also acted as a distributed system for system D itself. It is not really used much any longer. Part of that was having a stampeding herd type issue, where if you had a whole lot of machines, they'd all be checking into SED, and even though SED is really resilient, if you have thousands and thousands of machines just hammering and watching and holding on, even SED can't always keep up with that. So there was moving away from that. Torx is actually a really interesting one as well. If you think about OpenShift and Kubernetes, they are tied very strongly to the container runtime. Now if your focus is on that, running Kubernetes or running OpenShift, you need a very specific version of, at this point in time, it was Docker, but whatever that container runtime is. Users who wanted to run container workloads outside of, say, OpenShift or Kubernetes generally wanted a different version of Docker, the latest supported version from either REL or from maybe the company itself or some other application, while Kubernetes tended to hold back on older versions of Docker. Now, Torx was a way within CoreOS and Container Linux to be able to essentially say, okay, we're going to ship with a version of Docker so that our customers will be happy, but if they install Tectonic, we can turn it over to an older version, which can be used by Tectonic so that the cluster works at that level. So Docker was not the only example. There were some other things that you could also move around for that, but that was kind of the biggest thing. On the Tectonic side, we're going to see that the Tectonic team actually figured out a similar process but using a different tool set. Rocket, another container runtime. Toolbox, since most of these types of operating systems don't come with a lot of developer tools on purpose, or I'm sorry, not developer tools, but debugging tools on purpose to keep the size small. Toolbox is a container system to pull down debug tools on these systems if that becomes necessary. And the last two here that we're going to talk about, Flannel, just essentially network, fabric for containers and operators, which came later, which there's actually been a decent amount of talks here in DevConf on the operators, but the high level of operators is essentially extending Kubernetes or OpenShift, providing more functionality through the Kubernetes API and then being able to action on those items within the cluster itself. So obviously there's a ton more, but those are kind of the more relevant items for that. So then when Corus was releasing this, there was plenty of concern that this new way of doing immutable infrastructure was going to be so exciting and so interesting and everybody would want to do it that Red Hat really needed some kind of response. And the direct response that came out of Red Hat was Colin Wolter's OSTree project that created Atomic Host. And as Atomic Host got a little bit more mature, we added more and more items to the umbrella organization of Project Atomic, and that just became everything that's container-related should kind of be there. And this was also kind of at the same time as OpenShift was really getting really big there were container-related things, but it wasn't really a good idea to put OpenShift under the Atomic umbrella and so they kind of merged and became together Atomic OpenShift, which is a little bit closer to where we are today. So for Atomic Host, we've got three different versions of it. We've got the Fedora Atomic Host that we spin out upgrades every two weeks. We've got Rel Atomic Host that has a faster life cycle than Rel itself, and it comes out every six weeks. And we've got Cent Atomic Host as well that is basically a rebuild of the Rel Atomic Host stuff that we're already doing. And all three of those all share OSTree as their base foundational immutable infrastructure solution where it will lock down the packages that are there and only allow you to install new ones with RPM OSTree. The big thing about OSTree is, next slide, transactional updates. If anybody went to Colin's talk yesterday, his dream of an update is as the host is updating, somebody can pull the power and it's still going to be okay. You will still have a host that you can boot into just by rolling back to the previous installed operating system. And we like to throw around the metaphor of get for the operating system where you have snapshots of the state of the entire OS as an image, and you can move between those snapshots and move to different sets of packages and things that are there. And on top of that, you can also add new things to make a dirty treat, right? You can add in a couple different versions of Docker if you'd rather run with a very, very recent version of Docker somehow if you can get the repo pointing at it. And at any point, you can always roll back to a previous version that you know works, right? So real quick, I wanted to go back to this real note how fast the response was. So this was Tuesday, 15th, April 24th. So I believe that was roughly seven months after the first CoreOS release. This was announced, and we were already trying to compete with our current coworkers. You were that big of a threat? Yes. So Project Atomic did a lot of the same type of stuff. So we just talked quickly about OS Tree and RPM OS Tree, but System Containers was very similar to Torx and that the attempt there was to be able to switch to different runtimes or use a different cooblet or things like that, but instead of doing it through moving files around or pointing something to a link somewhere else, it was through using Run C, this was before Podman, using Run C and not using a direct runtime other than Run C itself. The point to really hit there is that anytime you're using a mutable infrastructure, it's not that you want everything locked down over time. You want controlled mutability, right? You want to be able to change a couple small things and it's a matter of what you want to allow people to change and not really turn the entire thing into back to vanilla rel where you've got everything installed that you could possibly use. But there are cases where sometimes you are going to need one more thing for one more customer. So other things we had there, the Atomic CLI which sort of just sat in front of a lot of other tools and this was meant to be a management type tool. So an administrator would log into these nodes and run Atomic CLI to do something like pull down installations of pods or things like that, be able to pull images, be able to install System Containers, be able to do an update of the system, et cetera, et cetera, et cetera. Builda, I think people are pretty familiar with that, building OCI Containers, Scopio, moving them around. Cryo, which is actually an implementation of the CRI or the Container Runtime interface for Kubernetes. And Cockpit, which actually still is a really awesome UI and integration into management of systems. And again, lots and lots of other tools that our teams owned and still own, but we won't jump in too much of that. So last year, like two days after DevComp, we flew home and we started getting calls saying, hey, did you hear the news? Because we bought CoreOS. Yeah, there was a guy we saw on Twitter that cut off his beard, it was pretty interesting. But after this happened, so did this. OpenShift 4.0 development started going down and what we knew is that for the Atomic team and for the Container Linux team, we were coming together and we were going to be helping significantly on OpenShift from that point forward. So no longer was there going to be an operating system that somebody's going to install OpenShift on, but there's this combined thing. So actually, before we go forward anymore, the steps to install OpenShift previously or currently is you have hardware or a cloud, right? You install your operating system, whatever that might be. We'll use rel as the example.  You get OpenShift, you install OpenShift using, in this case, an Ansible installer. So you have OpenShift now on top of rel and your next move is, well, day two, I need to provide some updates. I need to get this machine updated because there's some CVE or there's some update that I'm required to have. Well, great, you update the operating system, you verify that works, you update OpenShift, oh, something's not matching now. Or maybe it is and everything is great, but there's a lot more management overhead for that. A big jump here was that we were going to start combining these things together. So no longer would it be a multi-level, you have to manage everything at multiple levels, but instead it was, you got your cloud or your hardware? Great, install OpenShift. Okay, great, now manage OpenShift. It's all one thing now. There's no more of these multiple layers that you're going to have to deal with. Everything is managed through the cluster, not at every single level, you must manage those things individually. And that really came from the outstanding work that the CoreOS folks had done with Tectonic of making sure that the version of Tectonic really defined what version of Container Linux was running underneath. They were really married and they were really in sync because the developers would just go downstairs and say, hey, I need this from you guys. So as the team started jumping together, we had to come up with a philosophy. So a lot of the philosophies that we already had just matched right off the bat. So the first three there, the minimal, the immutable, and the effortless management. So minimal in this case are things like dynamic languages are not desired to be on the image, nor is extra development tools or extra debugging tools or anything else that's not going to be used on a at least monthly basis. It's just not needed. And all it does at least for the updating and installation and management is cause more download time, more packages that need to be revved, more things to track, all that kind of stuff. So just like Container Linux, just like Atomichost, we wanted to keep those things as small as we possibly could. Again, a big thing too is the simplified install. So with Red Hat products, we tend to have Anaconda, which is actually really, really awesome. However, when we pulled CoreOS and Atomic together, we realized we have a very, very specific use case for installation. And that use case is get it on disk, don't touch it. And giving people a lot of options to mess with the installation was counterproductive for us, at least at this time. And so we actually ported the Container Linux install. Is that what it was? Or is it CL install? Yes, CL install. And to CoreOS install, which is a shell script that essentially DDs the image onto the drive. The biggest portion of this installation script is a GPG key. Pretty much everything else is five, maybe 10 lines of code to get this onto the disk. Now, that might sound a little weird, right? Okay, you've got it on the disk, but the disk might be significantly larger than what you just DD down. Ignition is there for. So ignition, once it starts doing its first boot, it will start provisioning those disks, growing things. Basically, it takes over from that point. If anything needs to be laid down, if anything needs to be modified, that is where it happens. And you're going to see a little bit later how that ties back into OpenShift as well. The immutable thing, as Jeff said and Colin said yesterday, it's more about controlled immutability, not everywhere, but in certain locations. You can still change configuration files. You can still update SSH keys. But if you're trying to mess with system-related items, it's not going to work. It's not going to allow it. Unless you take extreme measures to do so, of course. At which point, it's kind of sort of not right at CoreOS any longer. And the effortless management. So for those who have used, or not used container Linux in the past, it's kind of like a continuous rolling release that just keeps on going. And you can opt-in slash opt-out of these updates being applied to your machine. So you're not sitting around going, oh, well, I just read the release notes of XYZ and it had this issue and I need to go ahead and install these updates. But instead, it's either automatically applying or you're sitting there going, okay, I got an email or I saw this thing happen in terms of a CVE was posted online or whatever. So I'll go ahead and say, yeah, make this a little update. On the atomic host side, we ended up adding in a timed update mechanism which acted similarly but not exactly the same. So you could also opt-in and consistently get new OS tree updates at that point. And the last two here, opinionated actually came from container Linux. So atomic host was not nearly as opinionated as we should have been. We started off pretty opinionated, but over time kind of allowed it to become a bigger set of things. So we were focused and then over time it was well, okay, yeah, for that use case, we'll go ahead and add those libraries. Yeah, all right, for that other use case, we'll go ahead and add that service. All right, and so instead of being focused on either running the cluster or container workloads directly, it started becoming a container optimized operating system that does a lot of other things, but maybe you shouldn't, but you probably will operating system as opposed to being very, very opinionated on it. So when our friends from CoreOS came over, they nicely called us out on it and we made sure to stick with that from that point forward. So everything that we choose is for a very specific purpose and we're not choosing multiple items. So when you see the runtime, we've got one. When you see the cluster, we have one set of cluster components. When you see any of those items, we have one for one specific reason. And then the last thing here is focusing on the cluster. So we embed the OpenShift packages and as I said before, we're pushing management to the cluster, but it's not just that, we're also versioning with the cluster. So both Tectonic and Atomic had issues when trying to work, I'm sorry, both Container Linux and Atomic had issues when trying to work with the clusters above. And part of this was for their own operating systems with their own use cases with the schedulers on top with their own use cases and so the runtimes didn't always match, the libraries didn't always match. Basically, there was a lot of hoops that would have to be jumped through to get these things to work. And essentially by versioning the container runtime, the operating system and the container scheduler all together, we can continuously ensure that they always work lock step and they all feel like one entity, not multiple entities that are separated. So a lot of the things that we had to choose from were things that came from both Container Limits as well as Atomic Host, right? So one of the choices that we were forced to go down was are we going to stick with the roller mechanism that CoroS did with the initially Chromium work and then eventually became a fork of that or are we going to stick with OS Tree that was a little bit more native to Atomic Host? And luckily we both came to the decision fairly early on that OS Tree was a little bit easier to use and we had a little bit more control over what was going on with OS Tree and we really liked the ability to have multiple different images on the disk, not just the two. There had been cases where CoroS had updated and then updated the other partition and then at that point you kind of run out of backup plans if both of them don't work so you wind up with the brick machine and with OS Tree you can lay down as many updates as you need and as long as you've got the previous one that worked you can continue adding more on until one new one works, right? So a little bit more flexibility and a little bit more restuability, right? The toolbox we thought was amazing and it had some really cool tools and so we really wanted to move that over in and we made some other decisions about the stack where we're going with Cryo for reasons that I can't really get into but I'm sure other talks have covered why we love Cryo. Ignition was kind of the way that Tectonic installed and since OpenShift and Tectonic were working very close together very early on the architects said well we're going to go with the Tectonic installer route and therefore Ignition had better beyond whatever host you choose, right? So that was kind of laid down and we've learned to love it and it's come a long way and it's super, super useful. Build is what we use for building container images but we don't actually have the builder package on the host it's actually up in a library inside of the OpenShift packages, right? And let's see. Scopio I believe also is included and it's been entered in in Podman so it's there but it exists as Podman, right? So we picked up a couple other things here so Podman which there's been a lot of talks around that obviously the Kubelet and other tools from OpenShift CoreOS assembler which is what actually creates Red Hat CoreOS as well as its image payload which we'll talk briefly about later. We also, we don't include the machine config daemon but it does run on and it's specifically made for Red Hat CoreOS which is something that senses changes and we're going to go through a flow of that shortly and is able to intelligently create updates as well as changes to configuration files and all that kind of good stuff and then Pivot which is sort of a layer in between a lot of these other items which we'll also see in that flow. So we can talk very briefly here we'll go through these pretty quickly of differences here. So container Linux, right? Designed for general container workloads just like Atomichost, Rails, literally all the things you can do whatever you want but it's on you to make sure that those things are updated there set up the way you want them to work and Red Hat CoreOS which is designed to power a container scheduler in this case it's OpenShift specifically but this is kind of what it boils down to as being part of OpenShift now. So we are really the abstraction layer between the hardware and above. So we are a component of OpenShift no different than the cooblet is a component of OpenShift no different than the logging layer is a component of OpenShift it is all together it is all included obviously hardware cloud is not but everything from Red Hat CoreOS up is all one unit when it comes down. So we have some other items here container Linux provides multiple runtimes Rails whatever you install is there Red Hat CoreOS is specifically one in its cryo which is an implementation of the CRI like I said before so it is specifically meant for Kubernetes slash OpenShift not for other container runtimes or workloads I should say. So some of the other choices for operating systems a lot of people really love the Gen2 based kernel and the content that came in container Linux we are a rail shop so we ship rail content and so Red Hat CoreOS is going to have the rail kernel in it it's going to have rail RPMs it's going to have OpenShift RPMs and it's going to be managed if you need more packages by RPMO Street but generally nobody gets to choose that except for OpenShift yes that is one of the things that we are working on yes sorry are we going to have support for bare metal for Red Hat CoreOS and yes that is one of the things that we are working on that is one of our items for us to deliver yes for so this is basically the comparison between the three operating systems and we've got a couple more to go so this is an interesting one in my opinion so container Linux used Omaha for its update mechanism is anyone here familiar with Omaha or the Omaha protocol of course yes alright so it is actually the same thing that is used from Google for updating Windows components in some cases as well as it was used I don't think it still is used or at least it's not used as much as it was with Chrome OS for delivering the updates to Chrome OS box so essentially it was reused along with Core Update to keep container Linux updated we know how rail is updated right it's the tried and true rail repositories and or satellite etc for CoreOS it's something called the MCO and the MCD MCO is machine config operator which again it's that operator word that you've probably heard a heck of a lot while you've been here it lives up in the cluster and it's sort of in charge of passing off items to the MCD or the machine config daemon which will then consume that information make an intelligent decision if it's possible to actually action on this information and if it is it's going to update the system update files, update configuration and restart so you have a brand new system coming well not brand new I should say an updated system coming back in to the cluster or to say no I can't do that respond back up to the cluster and say you need to destroy me and reprovision me because it's not possible for what you're asking so security the security model for Red Hat CoreOS comes from RO the main reason we're using the rail kernel and rail content is we want to make sure that all the features that we support in rail are going to be available in Red Hat CoreOS we want to make sure SC Linux is in enforcing mode container Linux has SC Linux on it but it's impermissive so it might as well not and we want to make sure that any CD updates that we're getting from rail can be easily applied and having to do all that work twice for every CD that we find just seems like max so that is another reason for the content choice that we went to another reason to keep the package set small one of the reasons that we want to get things off the host as much as we possibly can is to make sure that we've got a minimal footprint for places where somebody can come in and try and find one of those CDs we want to make sure that the cluster is in control OpenShift is in control of exactly what goes on to the host and allowing them to have one big button that appears in the OpenShift console that says you are able to update when you are ready push this button and everything just rolls out to every single node on the cluster is just the experience that we want to deliver to customers we know that's a big deal because some updates for OpenShift in the past have gone a little less than ideal and so we know we have to land this correctly and this is our mechanism of making sure that we are going to get it right and so real quick before we move to the next slide SSH access is something that is more so optional for Red Hat Core OS the only time you should ever actually SSH into a Red Hat Core OS machine is if something has gone terribly wrong that is you need to go in there and debug something related or you need to go in there because literally there is some unknown weird thing occurring that just could not be foreseen other than that there is very little reason to actually go on to those machines and that is by design if you do SSH into this machine or these machines that's okay nothing bad is going to happen to you or to your machine here but what will occur is it will note in the cluster this machine has been accessed so it may not be exactly the way that it should be in the future what we are looking at doing is starting to taint these machines after SSH access because hey something was wrong right so it shouldn't be part of the cluster you are accessing it you are looking at something that went wrong we don't want to destroy the machines but we want to make sure that it is not actually continuing to run with pods in the future if people are accessing them so we actually already went through that one so we will hit this so this is how essentially the updates work is a very simplified version and I am going to hand wave at the higher level OpenShift stuff so essentially at that point think of what I refer to as the MCO or the machine config operator that is a operator running within OpenShift that is in charge of getting its hands on what should be on these hosts what should be on my workers what should be on my masters it ends up providing this document down well it ends up holding this document I should say which is called machine config and this is an ignition configuration file and this is an OS image URL now ignition configuration is going to be things like dropping files on the system modifying configuration adding or removing SSH keys that kind of thing the OS image URL is literally a OCI container reference using not a tag but using a hash so that we know exactly what we are getting to a container that is in a container registry that houses the OS tree content that the machine is supposed to update to so this is not we are going to get some repos and do something or hey there is some files sitting on some server somewhere we are going to download no this is updates coming through OCI containers to the host itself so it is basically containers all the way down with an openshift from this point forward so the MCD or the machine config daemon senses hey this machine config that I am attached to has changed can I actually lay down these can I make these ignition changes that I am being asked if the answer is no it just essentially tells the cluster I can't do what you are asking it is impossible for me get rid of me reprovision me the way you want and then continue from there and the answer is going to be yes so when the answer is yes it is going to make those changes to config files do whatever it needs to do and then it is going to pass to the pivot which is a command which is going to pull down the image that is referenced in the OS image URL explode that content pass it to rpmos tree which will then make a basically it will install the OS side by side with the current running one so there is no modification to the current operating system it updates the boot loader hits the reboot reboots back into the new system which is side by side with the old one so if there is anything wrong it can go right back to the original one and then report to the cluster I am sorry I couldn't do it you need to reprovision me but if everything goes as planned which it does it reports back up I have switched over I have made the changes that you wanted I am in the state that you want I match that machine config in terms of what I am it updates the cooblet that could be security updates that can be all sorts of things now the information coming down is what comes down from what we talked about earlier Cincinnati which has a graph of all the components and the versions that match each other and that essentially gets distributed down to different operators flows down comes down through here and the updates occur from that point I know it is as simplified but this is as simple as we can get it down so another way to look at it so over the year updates the way that they work is as we explained in the last slide basically we have a running system here and we pull down a container out of Quay and inside of that container we mount it and there is no process running inside of it it is basically just a flat container with some files we pull in the updates that the operating system needs for the next version and then we point at that next version and reboot and simple as that we call it the pivot and it is super easy and fast so we only have a couple minutes left here before questions let's jump through the roadmap that we have quickly so some things that we want to do that we haven't got to fully yet streamlining which is always happening with every project and product we want to remove more packages that we know are not absolutely required at this point what we have is Python and other dynamic languages that have snuck in through other packages I'm a big Python fan but we don't want people trying to SSH on the machines to run Python and you can run Python on Red Hat CoreOS in a container that is what we would rather if you are going to do that and of course continue to shrink these images down the operating system image I should say down so we want to get to Fedora CoreOS we want you guys to be able to use the same technology that we are using in a community supported version that does what the community wants so Fedora CoreOS is trying to fulfill those roles that other people were using container Linux that were not just running tectonic some people actually had it as their desktop for a little while there were plenty of other use cases like Mesos and general runtime just a general container runtime and we want to enable that for people in the Fedora space and allow them to help guide some stuff that will then eventually become part of Red Hat CoreOS this is that entry point and we want to make sure that we form a bridge with the always ready OS work that Steph Walter and Don Zekas are working on because the way that they are gating packages and the way that they are building things is really amazing and if we get it optimized for the smaller package set of Fedora CoreOS we think we can really build something bigger from that plug in support for Kola we want to make sure that there are other things that you can do with the testing framework that is part of Mantle and we want to increase the usability of ignition Colin called out that it would be nice editing a couple hand files are going to throw a big configuration problem down the road because it's human readable versus machine readable and the last items we have here obviously there's more things that we want to do but essentially we want to continue expanding to other platforms so bare metal obviously is a really big one but we also want to expand that CI and OS level testing so right now we mainly are testing in AWS with some kind of prototype testing in other clouds and other locations but we want to essentially get that a lot stronger and a lot better add more tests to Kola as well we do a lot of testing with Kola but there can always be more and on top of that we want to port Kola to more clouds so it does have a limited set of clouds that it tests on so we do have some work around that as well so this is just some sources this is a tiny list of things that we own we own probably about seven more pages of these things these are some of the ones that we talked about today and really do we have any questions we covered a lot we didn't go super deep but yes sir so the question was do we have a real release date for bare metal for core OS yes, we do have that release date I'm not sure that we can talk about it at DEVCOS though sorry sorry that sounds like a fancy party I hope I get to be there other questions yes sir for best way to provide like the next eGIS S access solution so my three departments is actually not doing it at our times my new machine that's actually not scanned what would be the best way to do it that's a really good question so there's two things that come to mind so the question is you've got an ancient security scanner that you need to run on the host and it uses SSH access for it today how would you suggest or how would we suggest to you the best way to go about doing that and that's a great question so first off I would say with the diversion that we're pushing out now they can still do that and that's okay it's going to show up as an alert that someone access these machines it's not going to stop anything obviously in the future we want it to be tainted because it's been accessed but there's a couple things so number one it sounds like is this opportunity to have a security scanner that doesn't rely on SSH or other things like that however obviously it's not totally realistic so even when we go down the point of tainting these machines and pulling them out from the cluster temporarily tainting does not mean the machines are unusable out it just means that okay it's been accessed it took a base of maneuvers that's not really supposed to so the security tool actually set off a security event and now to get it back in let's say you did it to have your cluster it takes a command to get them untainted and back in the cluster in fact you could do that quickly along with the scanner if you needed to do so I would guess that and since this is a future thing that we're tainting today but my guess would be too that there would be a way that you could temporarily disable the tainting for a certain amount of time for something like that but we definitely want to push people towards don't access these these are only for the cluster any SSH should be some sort of a security and the other half of that is we need to be very vocal and talking about the fact that we're going to be doing it this way rather than just hiding it in a dock or a ship to admin doesn't just run into it down the road and can't figure out why no new containers are scheduling to that node and the taint that we're applying is a no new containers no new scheduled containers to it so the running containers will still be there and that's even more mysterious to somebody that so we want to make sure we're very vocal we talk about it a lot and it doesn't just come out of nowhere which is why it's future work never forget about it sorry we have to repeat the question so the question was is there any support or thought for supporting things like VMware in this case it's an internal installation of OpenShift and things like AWS external cloud is not an option so we do have on a roadmap basically a roadmap to continuously be adding new clouds whether that is external or internal and I don't know exactly where each one fits down in terms of the roadmap but VMware is very popular so I tend to think that it's it's going to show up but I don't really know exactly when if you talk to a PM that would probably get you there that would definitely be a better answer than we can get anybody else okay? thank you