Live from Las Vegas, it's theCUBE. Covering Informatica World 2019, brought to you by Informatica. Okay, welcome back everyone. It's theCUBE's live coverage here in Las Vegas for Informatica World 2019. I'm John Furrier, host with Rebecca Knight, who's on the floor getting some data, getting some reports. She's my co-host here this week. Next guest is Bruce Chisholm, board member of Informatica. O.G., original gangster of the tech scene, been there, done that. Welcome back to theCUBE. Great to see you. Yeah, great to see you, John. Keep up alumni. Yeah, I love having you on because you're kind of historian through experience, still active in the industry, obviously Informatica. Four years private. Historian, that's scary. You've been around the block. You've seen more waves than I have, and that's a lot. But you've been, you've done a lot of things, and you've seen the waves. You've run companies, you've been on boards. You've been on Informatica board. Four years private. A lot of great things can go on. Michael Dell proved that he took Dell, Dell computer, which is now Dell Technologies. He took it private. I asked him, he wanted a retool and didn't want to do the, you know, the shot clock of being a public company, the filing and sarbanes out, all those regulations. Because he knew it was coming, the wave was coming. Informatica did the same thing. So I'm expecting an IPO or M&A, big deal happening. But four years with great product people, you're on the board. Data, our original conversation, four years ago on theCUBE, hasn't changed. It's the same wave, now everyone's jumping on the wave. The good thing for Informatica is as a private company, we got to do things that we could not have done as a public company. The level of investment we made in R&D, the transition from perpetual or on-premise to subscription, the investment in the sales organization, couldn't have done that as a public company because the shareholders tend to be too short-term focused. And also I will add, just to get your reaction to, is that my observation looking at these situations when you have smart people, the board, like yourself, and the product team, which I've been complimentary of Informatica as you know, some other critical analysis, but that's different, but great product engineering people. When you don't have the pressure of time, you can watch things gestate. And when you're early, you have an advantage. Talk about that because that's a strategic thing that not people are talking about, but you've had early lead on data. You've had product engineering leadership and you had time. Yeah, it's not as easy as you make it sound. Keep in mind that Informatica is owned by financial sponsors, private equity, CPP. And it's up to people like myself on the board, the other independent board member, the management team, to continue to remind the investors that if we make early investments and they pay off, the company will be worth more and they'll ultimately make more money and they're limited that the partners will make more money. I made a sound like you're on the beach drinking wine. A great example is what Informatica did with the data catalog. That was an early investment. No one really knew whether it would pan out, sounded good, but it required a significant investment that came out of the pockets of our investors and we were able to convince them to do that. Another great example is Clare. AI is hot. Well, had we not invested in Clare three, three and a half years ago, Clare would not be in existence today. Couldn't have done that as a public company. And that gives you a little bit of lead again. There's no shot clock on public, but yeah, the private equity there, I'm going to let you sit around and hit the beach and clip coupons, you've got to work hard. But I got to ask- The other thing you've seen, the company has gone from a great point product company, great products to really developing a platform and architecting a platform, which requires a significant amount of engineering. I was going to ask you about that. I'm glad you jumped the gun on that. Platform is the key. Speaking of platforms, I was just at Adobe a company you're very familiar with. They're rolling out a new platform. Platforms are now back in vogue, but it's not the old way. The old day was build a platform, have a competitive advantage, lock in your nested solution in emitability. Now it's platform open, different twists. How is that different? Cause you've seen the platform where you got to own it, various entry, proprietary technology, to platform that's open, extensible. Yeah, customers have gotten smart. No customer wants to be held hostage to one individual platform. SAP being a great example, Microsoft Windows being another example. They want to make sure that if they choose one platform, they could easily migrate to another. It's one of the reasons why Informatica is in such a sweet spot, because we allow our customers to choose which cloud infrastructure providers they want to put their workloads on, and they could use multiple cloud infrastructure. I got to ask you about the competition now, that's not competition, but you know, co-opetition, well, just marketplace in general, everyone's jumping on the same wave that you guys have been talking about.  I've done here with the team, you, on four years ago, look up some things we were talking about. Not a lot of many people talking about data-driven, hardcore analytics, NextGen. These are kind of topics in AI, machine learning. Now everyone's talking about it. What's different about Informatica as the noise level increases around some of these things? Certainly it's pretty obvious, AI is going to be hot, multi-generational, cloud multi-generational things can happen, operations, you know, AI and automation. But what's different about Informatica? What should people know about Informatica that might be unique that you could lend some insight into? So when I think about the competition or the co-opetition, I put those competitors in two buckets. There's a whole slew of smaller players. They have some really good point products. Fortunately for Informatica, they don't have the scale to compete. And when I say scale to compete, not just on the go-to-market side, but they can't afford to invest $200 million a year in research and development, building a complete platform. So even though they're kind of ankle-biders and occasionally I feel like the company has to slap them around and their annoyances, I don't think they're a big threat. The cloud infrastructure players, the platform guys, Google, AWS, Azure, will continue to provide data tools that are developed for their stack. They will do some things that will be good enough. The good news is Informatica does great as it relates to enterprise cloud management. So if an enterprise really cares about their data and they really care about having choice in the future and they don't want to be held hostage to any one platform, Informatica is the only game in town. You know one of the best is doing the Q this our 10th year and I remember telling us from NetApp people because they invested in cloud early too. They don't get the credit. Is it, this is another example of Informatica invested early on in cloud. I mean I talked to Amit and Anil years ago, again the cloud, they were well down that cloud path. So Johnny come lately to kind of jump on the cloud is an advantage so props to Informatica. And plus it's not cloud only. Most of the large enterprises are hybrid. They will be hybrid for many years to come. In fact, if you look at workloads today, the majority of the workloads are still on premise. Scale's come up a lot. You know my commentary in the cube, everyone who watches me knows I like to rap about and it was the first to call Amazon the trillion dollar opportunity because of the scale. Scale's a new competitive advantage, I've said that. I've said open is the new lock in. And this is, values the new lock in is what I said. So now you got scale. So the question is how does a startup compete if scale is table stakes? Is it race for funding? Is Snowflake's got a three billion dollar valuation? Are they worth three billion? We're going to analyze that in the cube later. But they raise almost a billion dollars in cash. Do you scale up with cash and go? Great technology. It starts out with really great technology. So I look at an organization like Snowflake. Great technology. I look at Databricks. Great technology. So I look at the great new startups. What makes them great is they have an innovative technological solution that's hard to replicate. Then they get the funding and they're able to scale. That's what it takes to be a startup. And that's almost the OG, original gangster, venture capital model. That's correct. Agile, iterate your way to success. No craft, no scale, just speed. Is the world going back to the old? It's going back to innovation. To technical innovation. Especially given that you have so many scale players. You can no longer just come in there as a startup. Money alone is not going to enable you to be successful. All right, I want you to pay it for it for all the young people graduating. I just was at my daughter's, Cal Berkeley graduation yesterday. Although she wasn't in this class, but Cal just graduated their inaugural first generation class of data science. Databricks was involved in that. They donated a lot of software. They're very calorie oriented. People graduating high school, elementary school. This is a new field. Not enough jobs. I mean Berkeley, a leading institution. First class ever in data science. What skill gaps are out there that need to be filled that people could learn now to get ahead and get an advantage in the workplace? My view, John, it starts in middle school with math. If we could help our kids who are in middle school to get through algebra, studies have shown they will move on to undergrad and then many of them will move to graduate work. We've got to start early. Yeah. There's some simple fixes. Help people become coders. Help people do other things. But the reality is, if we're going to solve- Get the algebra done, you're not going to code. We have to solve the longer term problems. So when I think about jobs or the future, we've got to create people who are creative but at the same time understand the basics. Yeah, math, stats, great stuff. Final question, you going to run a company again soon? So I get that question quite often. First of all, I love doing what I do today which is kind of a lot of little stuff. I do miss running a company, but as I've told a whole bunch of people, I have no desire to ever report to a board again. So unless I own 51% of that company, I will not be running a company. Now you know the deal terms. Anyone's watching for investment from Bruce's partner and one of them, great stuff. What's missing? What's going around the corner? What are people missing in the news these days and the trends? What's coming that's exciting that nobody's talking about? I think what's happened and this happens each wave. There's been so much excitement about the movement from on-premise to cloud, about AI and machine learning. I don't think people really appreciate how early it is that we're this much into it and we've got a long ways to go and the old workflows that are on-premise, the amount of advancement in artificial intelligence and machine learning has so far to go that people need to be patient and continue to invest aggressively in what's going to transpire 10 years from now, not six months from now. And then you add things like 5G, fastest speed, Wi-Fi, that also is going to have this huge impact. Great insight, Bruce. Thanks for sharing that insight. Get the kids learning math in middle school, gateway to coding, gateway to graduate work. Next 10 ways, a lot of ways coming. Bruce, thanks for sharing the insight. It's good to see you again. It's a pleasure. Give us coverage here at Informatica World 2019. I'm John Furrier with theCUBE. Thanks for watching. We'll be back with more after this short break.


One of the most memorable experiences I had in medicine was part of a tour that I had of a hospital that I was working at. The lady who was giving us a tour of this hospital took us all around and then she took us to a lower level of the hospital and there she said that we needed to pay special attention to this particular room. The room had a label that read Special Orthopedic Procedures or something like that. And she told us as we were standing outside the door she said actually this is the room where we do all of the abortions in the hospital and we intentionally mislabel this door because we don't want people to know that there are abortions happening in this hospital and We also want to avoid all of the protesters. So we keep it a secret. It bothered me a lot. I would work in that hospital for many years and I knew that a couple hundred yards away from where I was working with cancer patients and trying to save lives lives were being taken. I Spent a while working in the neonatal ICU or NICU of this hospital and we would spend hundreds of thousands of dollars or even millions of dollars to help save the life of a baby who was 22, 23, 24 weeks of gestation and again a couple hundred yards away the exact same gestational age there would be babies who had their lives being taken away. I can remember thinking the only difference between the babies in this NICU and the babies that are in that secret room is that some are wanted and some are unwanted. They're the same age. They have the same physiology. There is no difference except some are desired and some are rejected. I felt haunted by this. I felt haunted by this juxtaposition. I wondered if I or anybody else could hear their cries. Ultimately, I left that hospital frankly with a sense of failure. I felt that I had failed in my duties. I didn't know what to do. I didn't know what I was supposed to do but I know that being a doctor meant that you were supposed to be saving lives and I felt like I was silent while lives were being destroyed. But ultimately this is a call upon all of the people of God. I am going to only read two short passages of scripture this morning, but I ask that you please read them with me. The first passage comes from the book of Proverbs chapter 24. We'll read Proverbs chapter 24 verses 10 through 12. If you faint in the day of adversity, your strength is small. Deliver those who are drawn toward death and hold back those stumbling to the slaughter. If you say surely we did not know this, does he does not he who weighs the hearts consider it? He who keeps your soul, does he not know it? And will he not render to each man according to his deeds? So let's look at this passage together. In verse 10 it's the author says that some people will faint. They'll give up on this day of adversity. We heard a good talk last night from Brother Kurt who talked about the adversity of our day. Specifically with abortion, who isn't overwhelmed by this problem? When you hear the millions of lives that are lost, it's hard to even begin to fathom these kinds of numbers. Most of us I think feel numb when we hear these sorts of statistics. We just want to throw our hands up in the air and give up. How do we even begin to address a problem of such magnitude? But this is the path of those whose strength is small. Verse 11 gives a general statement. It talks about those who are stumbling in Hebrew. It's literally slipping to death. I like how one translation renders this verse. It says deliver those being taken away to death and hold back those slipping to the slaughter. This morning I want to make three points to you. The first point is that it is incumbent upon God's people to rescue those who are being slaughtered. That's simply what verse 11 says. Verse 11 is a general command. It doesn't seem to have any one particular situation in mind. This seems to be a general command that God is giving to his people. How will we, how will I stand before God on that great day of judgment? How will we stand before the one who weighs our hearts, before the one who renders to each one according to their deeds? And to give a testimony as to how we obeyed this, how have we delivered those who are being taken to death? Note the response in verse 12 is basically to plead some kind of ignorance. It's easy to do that, but God says that he knows our hearts. If somebody were here at this camp abducting children, taking them to some room and having them killed, we would not be sitting here. We would be running around. We'd be coming up with some kind of strategy. We would be doing something to help stop this killer. We would not be able to rest because of the burden of our innocent children being delivered to the slaughter. Why then are we inactive? Are we attempting to plaster ignorance over the walls of our heart? Or have we fainted on the day of adversity? For the second point, I would like us to turn to the New Testament. Please turn with me to the book of Ephesians and let's look at Ephesians chapter 5. We'll look at just one short verse. Ephesians chapter 5 verse 11 reads, And have no fellowship with the unfruitful works of darkness, but rather expose them. I'll read that again, and have no fellowship with the unfruitful works of darkness, but rather expose them. So I believe that this verse could actually be a theme of the whole weekend. I know we've chosen the verse in Isaiah about lifting up a standard, but I believe that just as relevant, we could have chosen this. To explain the significance of this verse, I want to cover a concept about how often God gives us things in pairs. They often come in pairs, and when you contemplate the design of these pairs, you realize that you need both elements of the pair and if you just have one without the other, something becomes dysfunctional. Something goes wrong. Let me give some examples. A famous example would be faith and works. So we all know from the book of James. It says that if you have faith and don't have works, your faith is dead. Similarly, if you have works, but don't have faith, that's legalism. You need both. You need both of the two working together. It's in a sense almost two sides of the same coin. I'll give you another example. Repent and believe. So if you repent without believing, that's a form of moralism. It's a type of self-improvement philosophy that misses God. But if you believe without repenting, well, that's disobedience. The technical term there would be, that's the antinomian heresy. This is the plight of the modern American churches. It's obeying without repenting. I'll give you another pair. Teaching and discipline. If you teach without discipline, you actually harden people to the truth. Have you ever seen a scene of parents telling their children to do something? The child totally ignores what they do and they keep saying it and the child keeps ignoring. There's no discipline. There's no consequences. When you see a situation like that, I don't know how you feel, but I feel like saying if you're gonna say these things and not follow through with discipline, just don't say it. You're better off not giving these instructions to your child rather than give them without providing the discipline. Similarly, if you discipline without teaching, will you fail to win hearts and minds? There are other pairs that we could look at, mercy and justice, or even male and female. We need both. If we lack one or the other, we have disastrous consequences. Now Ephesians 511 contains a very powerful pair. Please don't miss this. Paul is telling us here that we are not to first fellowship with those who are, sorry, he says we're not to fellowship with unfruitful works of darkness, but rather expose them. Okay, so let's think about this. Let's imagine what it would be like to have one half without the other half, one part without the other part. Okay, so let's think about what it would be like to be exposing the works of darkness, but still have fellowship with them. So what would that be like? Well, there's a lot of cases like that. There's a lot of examples of pastors who get up and they preach these thunderous messages to roaring crowds about how wrong adultery is or pornography or immorality, and lo and behold, they're caught in an affair. They're caught in adultery. They're caught in some kind of immoral situation. What do we call that? We call that hypocrisy. There are many examples we can think of those who have exposed evil while yet have remained in fellowship with evil. But what about this? What if you do the first part of the verse without the second part of the verse? What if you're not having fellowship with evil, but you fail to expose these works of darkness? There is a term that someone has coined for this. The term that has been coined is it's an avoidance ethic, an avoidance ethic. An avoidance ethic is basically where you say, I'm going to practice my own personal private religion. I'm going to do the right thing, but I'm not going to stand up in a public way and suffer the consequences. I'm not going to be this very visible, prophetic, persecuted witness. This is something which I believe is probably more relevant to all of us. Have we fallen into this trap of only doing the first part of the pair, but neglecting the latter element there? So my second point is that personal holiness and a public witness against evil should go hand in hand. They should be wed together. When I was asked to speak on the subject of abortion, my first reaction was, doesn't everybody here already believe that abortion is wrong? I didn't want to preach to the choir, as it is said. But as I wrestled with them more and thought about it more, I began to realize that with this topic, and I believe with every other topic that we're going to be looking at over the course of the weekend, we have generally embraced an avoidance ethic. We have not exposed this evil to a watching world. We've been content to practice a holiness privately and have not served in the prophetic office that God has called his church to serve. You see, we are supposed to be the conscience of the world. We are supposed to be the light of the world that exposes evil. Even that term, light and exposure. Note how those words connect. Jesus says we're supposed to be the light of the world, and what does light do? It exposes things. It exposes darkness. It exposes those things that are hidden away. John the Baptist, he lost his head because he served as the conscience for Herod. This avoidance ethic that I believe has plagued many of our churches is also a form of disobedience. It's very comfortable. It's very easy for us to practice Christianity privately. It is far more difficult. It is far more taxing to make a public stance. And let's be open about it. There is so much to expose. There is so much wickedness to expose with this particular evil, this particular plague of abortion. We should expose the hypocrisy of a system of laws that makes it illegal to crush the egg of a bald eagle, but yet allow for the destruction of a human life. Look this up. It is illegal. You can go to jail for five years for destroying the egg of a bald eagle. We should expose the deceit of an industry that causes more psychological harm to women than can be measured. We should expose the anti-child worldview that we live in today. Children are viewed as burdens, not blessings. I read an article in a medical journal recently that was talking about a survey that was conducted where they asked women who had had an abortion what the number one reason was that they had for doing this abortion. And the number one reason that women gave, 40% of women said it was because they feared the financial consequences of having a child. Children are seen as being financial drains, as party poopers. They're seen as threats to the rainforest, and that what takes away our fun. When I first came into kingdom churches, I knew that I had encountered something special. I knew I was in the right place because I saw parents that love children. I saw the large families. Who can't miss that? That was something that moved me greatly. And one person told me something. He said this question to me, and I'll never forget this question. He said in a talk, he said, imagine I give you two options. Option A is I give you a million dollars. Option B is that I give you a child. Which do you choose? A million dollars or a child? And I can remember wrestling with that because a part of me thinking, wow, a million dollars, that sounds pretty nice.  Children are far more precious, more valuable than money. What would you choose? Our world chooses pleasure, our world chooses money. What do we choose? There's much to expose there. When you go to the grocery store and someone says you have your hands full, right? How many women here have had this experience where you go with your five, six, seven children and you get all kinds of comments. We get this on a nearly daily basis. And I know my wife has come up with various answers to give when people say you have your hands full or how do you do it? She says, I have my hands full with blessings from God. Do we expose the anti-child worldview? Or do we partake in the jokes about children being irritating our pests? Do we expose the worldview that is so common today about all of these matters relating to the burden of having a child? We should expose the apathy on this issue by praying. How much do we really pray about this? I was convicted myself to think about this. Often when there's times of war, we pray and that's tremendous. We are called as a church. One of our main ministries is to be priests that petition God for peace on earth. Praise God for that. But abortion kills far more people than war. Do we agonize in prayer over this or do we forget? Do we plead ignorance? Do we expose apathy by working in crisis pregnancy centers, by adopting, by showing love, not just in word but in action? My sister-in-law just recently adopted a child who has Down syndrome, a baby who has Down syndrome. And I was so convicted to see this. About 90% of the time in the United States when a couple finds out that they have a child who has Down syndrome, it ends in an abortion. 90% of the time.  And to see my sister-in-law do this, it made me think, would I be willing to have that kind of sacrifice to make that kind of commitment? That is probably one of the most powerful witnesses that the world can see. We should also expose the hypocrisy of those so-called pro-choice advocates. These same advocates are nearly always in favor of things like gun control, welfare, minimum wage, and affirmative action. They want choice restricted on all those issues for the sake of protecting a broader population. This is absolute hypocrisy. We should expose the hypocrisy about this being about women's health. So often you hear about this is an issue about women's health or women's rights. When everybody agrees that far more women, far more girl babies have been killed than boys, the reality is simply that people don't generally want, in the world, they want boy babies. They don't want girl babies. There's rampant gender selection in places like India and China and even in the United States. I remember when I was in medical school, we had somebody come in and give a lecture to our class on OBGYN and this person worked right in Boston at a Harvard teaching hospital and he frankly came right out and said gender selection in the United States is rampant. People don't always make it as explicit as we might think, but it happens all the time. This is absolute hypocrisy to have this be an issue about so-called women's health. Even the most staunch defenders of abortion squirm when they contemplate this issue, have a dialogue with somebody about this, expose the fallacy of that argument. If an abortion is about removing excess tissue, just like cutting off your toenails or removing your hair, ask them, do you have a problem with sex selective or gender selective abortion? Most people, like I said, even the most staunch pro-choice advocates, they struggle on that issue. They understand that. There have been people who have estimated the so-called missing girl number. So how many girls have been lost due to pure gender selection? So they find out it's a girl and they decide to have an abortion. So there have been three or four serious estimates that have been made in the secular literature. Estimates are somewhere around a hundred million girls are missing today because of abortion due to their gender. Think about that. A hundred million girls have been selectively killed because of their gender. Somebody has rightly said that the three deadliest words in the entire English language, do you know what they are? It's a girl. The three deadliest words in the whole of history, in the whole of English language are it's a girl. If you hear those words pronounced at an ultrasound, there is a significant chance that that baby will be killed. How tragic. I have personally four boys and a girl.  I can't contemplate. I can't think about how wicked that is. May God have mercy on us. There is much to expose about the works of darkness. Let's go to the final third point. Good works supply the attractive power of evangelism. So it's very interesting as this subject is raised in all kinds of churches all across America. Do you know what a lot of people say? They say this, this is not an issue for the church to talk about and the main reason that's given is they say this is not about evangelism. They say our job is to do evangelism and then all these other things will get sorted out. It's a very common protest that a lot of churches of all denominations all stripes raise. They say it's not an evangelism issue. It's really something that we shouldn't be spending a lot of time in. This is a tremendous mistake and I want to tackle it head on. The first thing we ought to say to that kind of foolishness is that we ought to declare the whole counsel of God. Murder and bloodshed, taking the lives of the innocent is one of the dominant themes of the Old Testament and the New Testament. Innocent blood, this is something that we should not shrink to declare. Chronically though, I believe that speaking about this issue is actually a great assist, a great help to evangelism. It's a great aid to a discipleship and let me explain what I mean by this. Jesus actually tells us in his teachings, there's three things that are the fuel, they are the attractive power for what draws people into the kingdom. There's three things and only three things that I'm aware of that he gives as having persuasive force to draw people in. The first thing he says, we won't look at these passages. I think most of us know them off the top of our heads. The first thing he says is in John 13, he says, by your love for one another, the world will know that you're my disciple. So when we love our fellow brothers and sisters, the world knows, hey, these are really God's people. That's the first thing. The second thing that Jesus teaches in John chapter 17 is unity and he says, may they be one, as you and I are one, so that the world will know that you have sent me. If we have unity, that has tremendous attractive power to draw people in. Jesus himself praised for that. And last year at Kingdom Fellowship, we had a whole panel just about that. The third element that has this persuasive power is good works. And Jesus talks about that in Matthew chapter five where he says, let your light so shine before men that they may see your good works and glorify your Father in heaven. So when we do good works, when we are out there, that brings people in, that light is radiant and attractive and winsome. And so we ought to ask ourselves first about our respective churches, those three things, love, unity and good works. How are we faring with respect to those three dimensions? You see, the world now focusing on the third element, good works, the world marvels when they see sacrifice, when they see people who are committed to the cause. Think about people like Mother Teresa or Martin Luther King. The world gives them tremendous respect because it recognizes the value of those who give selflessly, who are committed to a cause. And that is a form of social capital, if you will. It's a form of persuasive power that helps you be known to them as a person to be taken seriously. How can that kind of sacrifice not command respect? The church can purchase social capital and evangelistic capital to use those terms by doing good works like adopting, serving in crisis pregnancy centers, taking leadership in these issues, speaking out on these issues and being that prophetic witness. One of the criticisms that's often made about non-resistance is that it's this criticism of, hey, you people are just shirking your duties. You're not doing what you're supposed to be doing. All of us are doing our duties. We're fighting for the country. We're doing this. We're doing that. Why aren't you willing to do your part? Well, this ought to be a natural offset and in fact dampen that criticism and in fact completely squash that criticism before it even begins. We ought never to be thought of as leeches or parasites or mooches because the world ought to see what we are doing on behalf of the most innocent and weak members of our society. At many levels, being active on this issue will serve the church with respect to evangelism and discipleship. We're going to have another talk that gives more concrete examples on this. I want to close by reading a poem that I believe captures some of the human dimensions of abortion. I'm actually not sure who wrote it. You can see it up on the screen. Month one. Mommy, I am only eight inches long, but I have all my organs. I love the sound of your voice every time I hear it. I wave my arms and legs. The sound of your heartbeat is my favorite lullaby. Month two.  If you could see me, you could definitely tell that I am a baby. I'm not big enough to survive outside my home, though. It's so nice and warm in here. Month three. You know what, Mommy? I'm a girl. I hope that makes you happy. I want you to always be happy. I don't like it when you cry. You sound so sad. It makes me sad too, and I cry with you even though you can't hear me. Month four. Mommy, my hair is starting to grow. It is very short and fine, but I will have a lot of it. I spend a lot of my time exercising. I can turn my head and curl my fingers and toes and stretch my arms and legs. I am becoming quite good at it too. Month five. You went to the doctor today. Mommy, he lied to you. He said that I'm not a baby. I am a baby.  I think and feel. Mommy, what's abortion? Month six. I can hear that doctor again. I don't like him. He seems cold and heartless. Something is intruding in my home. The doctor called it a needle. Mommy, what is it? It burns. Please make him stop. I can't get away from it. Mommy, help me. Month seven. Mommy, I am okay. I am in Jesus' arms. He is holding me. He told me about abortion. Why didn't you want me, Mommy? God, please have mercy on us. Father, please have mercy on us for silence, for not exposing the works of darkness. Father, may we not shrink from our call. May we be people of courage. May we rescue those who are being handed over to the slaughter. Father, this is a great day of adversity. We need your Holy Spirit. I pray for wisdom. I pray for courage. I pray that you would give us the ability to act wisely with boldness, that our hands would not be stained with the innocent red blood that is flooding our land. Please, God, help us. Help us, O Lord. We pray these things in the name of Jesus, your Son. Amen.


Hi. Just get started. So I'm Derek. This is Diego. We both work at Pivotal. I'm a software engineer. Diego is an engineer lead at Pivotal. And we're here today to talk about pipelines and how you should write your pipelines, or at least how you should try to. So our agenda for today is we do a briefly introduction on continuous integration and continuous delivery. And we'll talk about some things you need to have before you start to do this thing at home. We'll do a very brief description of what concourse is. If you don't know what concourse is, this is not going to be enough. But just to remind us a little bit of a few concepts. And then we're going to jump straight into some demo. We'll be writing some pipeline here today. We'll do a recap of the principles that we talked about when we are doing the demo. And then we open for questions. Cool. So just to define the terms, we'll talk about it today. We talk a lot about continuous integration. And then continuous integration for us, when we talk about it, we are talking about the practice of building and testing our application in every commit. So every time someone pushes a code, should your repository, something will happen in your pipelines. And CD, or continuous delivery, is an extension of that. So it enables your team to release your software quicker in a more sustainable and automated way. To some extent, the goal of both CI and CD is to releasing software boring. Because in this case, boring is good. You don't want surprises when you're pushing code to production. You want it to just go smoothly in a reliable process. Cool. So what do we need before you start doing CI and CD in your organization? You need to have an automated, or at least an automatable build. So you need to be able to, from the common line, to run your builds and compile your binaries and then deploy your software. You also need some version control, especially if you plan to use concourse or anything like that. And more importantly, you also need your team to buy in. CI and CD are not our tools for delivering software. If your team don't want to do it, so there's no point in trying. All right. Why concourse? We choose concourse to be our tool for continuous integration and continuous delivery. Because it treats your pipeline as a first class resource.  And also gives you reproducible and debuggable builds. Each concourse run will run in an isolated container. And given the same set of inputs, it will produce the same set of outputs in general. It's also extensible. But it don't extend it to plugins or some configuration in your UI. You extend it to custom resources. And in a nutshell, this is how a concourse pipeline looks like. This is a really simple one. On the left, we have the input resources. Resources are the things that your pipeline use that come from the outside world. The green box is a job. A job contains a set of tasks. And each task will consume these inputs. And then it may or may not produce an output. So you can see on the right side the binary is the output of the build, of the build job. So just a disclaimer for those that are already familiar with concourse. In this talk, we are assuming that there are people in the room not familiar with concourse. So we'll be like sometimes explaining some context, like giving some context and explaining what concourse abstractions are. So just a disclaimer as we go. Yeah. And that's all we had for slides. So we're going to jump straight into some YAML. This can cause anxiety in some people, but stay firm. The pipeline we're building will have four jobs. Builds, deploy to staging, test, and release, release to production. And Diego will be facilitating it. So cool. So as Eric mentioned, we are going to call the pipeline. And the goal of this pipeline is going to give examples to exemplify six best practices of continuous delivery. So we are going to call this pipeline and we are going to call out the best practices as we go. So this is our YAML file. I'm sure that you love YAML, all of you in the room. Very excited about YAML. So we are going to create a concourse pipeline. So concourse, all the pipeline information is in a YAML file. And that YAML file is nice because you can commit, you can push that to Git. So you can keep all your configuration from your pipeline. So what we have in here, we have a resource. It's already in here because there is no point in typing this. We are just consuming this project from Git. It's the Springpad Clinic. So it's a project that's already there. So you can just use that. And we have already a container here. So this container is not like a concourse notion, like a property here. It's just here because we are going to use that. So first thing, we are going to create a job to get started. So in concourse, we can do this by creating a job that we are going to call build. And that's it, right? That's how we get started. So let's try to set up this job now. Want to explain the fly command line? Why I typed this? Yeah, you interact with concourse through the command line. You cannot upload your pipeline using the UI. So for some reason, concourse decided that everything will have airplane-related names. So fly is the CLI. Yeah, this set pipeline command sets the pipeline. DashL loads variables from a file. So in this case, it will be loading the secrets. DashP is the pipeline name. And the rest is the configuration of the pipeline. I hope you can all see my screen.  And there you go. So what's happened is that our command line is talking to a concourse server that we have deployed somewhere. So we are creating a pipeline. So nothing exists. And what this CLI is saying, well, since we are creating this pipeline, this is the diff between what we already have and what we are creating. So here we are just creating a new job. So there is an error here. Just got the resource. Yeah, we need to get the resource in here. There you go. All live coding, we already made an error. That's going to happen a lot. Right. So what we are doing here now, we are saying, well, there is this job. And we want this job to consume this repo, right? So this is the only thing we're doing right now. We are just instructing concourse you're doing that. So let's try that again. There you go. So we created our pipeline. Let me go back here. I might have the pipeline. Let me put this in here. There you go. Hopefully, you can still see my screen. This is the pipeline we just created. It's called CFSubmit. And there you go. This is just like a build called job with GitHub resource reading code from GitHub, right? When we create a pipeline, the pipeline is posed by default. That's the way concourse works. So we need to unpause that. There we go. So we are going to introduce our first best practice now. And our first best practice is, well, every time I push a change to my pipeline, I want this change to be propagated, right? And that's not happening right now here. I don't know if are there people familiar with concourse that can tell me why that's not happening yet? Just looking at the pipeline is not triggering, right? That's correct. So I'm going to set up this again because we created some files just to make life easier for us. This is the task. Yes. Cool. So now, if I go back to my code, so I'm going to jump some files in here just to avoid doing some typing. There you go. So we stopped here. And I just added a new task. And this new task is just going to compile the Spring Petty Clinic code. And that's the only thing it's doing. So if we go back in here, we can maybe trigger this build. I'm not going to do this right now because we are going to implement our first best practice. And our first best practice is just to make sure that we're triggering the build, right? So there you go. We're triggering the build now. I'm going to set this pipeline again with my new file. And this is the only change we have right now. All right. So let's apply this change. Let's go back here. And I am expecting this build to be triggered automatically because it's the first time I'm setting this pipeline. And there you go. So since this build was never triggered before and this resource is triggering this build, the build is running out. So it's just compiling some Java code now and generating a package. Right. So now that we are compiling this code and building this code, we are going to deploy this code somewhere, right? And we are going to jump to our second best practice, which is, well, we need to deploy this code in another environment, which is not production yet because maybe we're not ready to production right now. But we want this environment to be as close as possible from production, right? And that's the second best practice. And why is that? Well, to avoid some problems of, well, I'm deploying my staging environment. I'm using a MySQL database. But in production, I have an Oracle database. And then when I deploy to production, things go wrong. I'm compiling things in a different OS. So I test this binary. And then when I deploy that binary to another environment, to the production environment, well, there are some libraries missing or some stuff can go wrong, right? Another example could be, well, an OS, right? Let's say that in staging, I'm using, I don't know, Linux. And then I'm going to deploy into production using Windows. That's not like every time possible. But we should try to do that as much as possible, right? Keep all the environments as close as possible from production. Right. Just for you're not familiar with maving, we are not skipping unit tests. The package command that you're using to build also run the unit tests. Right. So we are cautious of time. So I'm not going to type the second job. So here we have a second job ready. I'm just going to explain that to you. And we just set the pipeline. So what the second job is doing, the second job is reading the code. The second job has a task inside. And this task is going to compile the code. The same thing the first job is doing. But once we have the binary, we are going to deploy this code to staging, right? And in this case, we are using pdubs. We are using Cloud Foundry. And we are just going to push that to Cloud Foundry. The bit where we want our staging environment to be as close as possible from production, it's somehow guaranteed by Cloud Foundry in this case, right? Because we are deploying to the same Cloud Foundry deployment. And things tend to be the same. Right, so let's try to set this pipeline here. There you go. We have some changes here. We added a new job called Deploy. And then this job has two tasks, the package one, and the deploy to staging. There you go. It's there. Let's go back to our pipeline. I can go back in here. There you go. We have two jobs now, right? Right. So there is something still wrong with this pipeline, or at least not ideal, right? And we are going to introduce our third best practice. And to do that, we are going to edit another file. So what we want to call out here is that we want this job here Deploy to fail if the build job does not go well, right? So let's imagine that in this job build, we are building something the tests have failed, and then we are going to deploy a build with broken tests, right? So we want to have a sequence, a linear sequence of steps here in order to guarantee that what we are deploying has actually been tested. So let's go back in here. I will try to set this pipeline here. This is our, they should be the same. Cool, cool stuff. So how to do that? Well, if we go to our Deploy job, we are consuming this source code thing here, right? And when we come back to the pipeline, both jobs are consuming that. I would like the Deploy job to consume the source code that was used, or the same commit shot that was used to build the code, right? So the only thing that we need to do here is to use this past property from concourse and say, well, this resource here is coming from build, right? So I'm going to set the pipeline again. And the only deep should be the line I just added. There you go. If we go back in here, if I refresh, there you go, introduce our third best practice, right? So that means that if it fails on the first job, we're not going to trigger the second job. Yeah, cool. So moving along to introduce our fourth best practice, looking at this pipeline, we can see that in the build phase, we are compiling the project. We are creating a package. We are creating a binary. And in the deployment job, we are doing the same thing as well, right? So what we need to do here is to avoid like building the binary more than once, like we can build many times. And that's because we want to make sure that the binary that was built by the build job is the same binary that's going to be deployed by the other jobs in the pipeline. So let's do that change. So the goal would be to remove this code here that's actually building the binary. So we will need this code here in the task that we're using in the build step, right? And now what we are doing here, we are well copying the jar that we're building into this directory here. This directory is going to be created by this output property in the task, right? Also, when we have inputs and outputs, the path does not tend to work correctly, as far as I can tell from my experience. So we need to see into this source code here. This is where we are cloning our GitHub repo. We are compiling that. And we are copying that into this release jar. So now this job here is going to output this jar. The problem is that concourse doesn't have the concept of sharing resources between jobs. So in order to achieve that, we need to move the result of the build that is the jar from the local container to an external place. And we're going to do that by just putting it into an S3 bucket. There you go. So as Derek mentioned, we have a resource that we need to put in here. So we have three main properties in this job. And we are going to create a compile jar here. And we need to pass some prams. And the pram, I'm going to remove this line. And the pram here is going to be the file that's going to be in the release jar. There you go. It's removing some white space from here. Right. So what this job is doing is it's compiling, creating a binary, and making this binary available inside this resource. And now we need to use this resource in the second job, because we're going to deploy that. So let's grab that compiled jar. And let's use that compiled jar. It has to be all through through. In here. Well, we don't need this one anymore, right? And there you go. Am I missing anything? I don't think so. I need to change the name. I need to change the name. There you go. Script as well. You can also extract this and put it into an external file. You don't need to type all of your tasks and configurations inside the pipeline. It's just simpler and requires less file movement. That's why we're doing it. There you go. So if we look at the diff, we are removing that package task in there. And we are creating some logic here in our first build job in order to export a compiled binary. And we are also using that compiled binary in our second one. So there you go. If we come back in here, we have two compiled jars. Let's go back to our conf. Let's try to spot what went wrong. We need to say, well, this one is actually being passed by the job build. With the trigger. And we need to add the trigger as well. Good call. The left one. So as the build will actually build the jar, we don't need to trigger this job on source code changes anymore, but rather on new versions of my binary available in my bucket. There you go. So now this jar is being compiled here. This job build is going to pass the compiled jar. So we make sure that we're not building the binary twice. So we implemented this best practice as well. So the next one is going to be to add a smoke test to this deployment. This is our fifth best practice. Every time we are deploying to a new environment, we want to make sure that our deployment was successful, that everything went well. So in order to do that, we are going to add a smoke test, what we call a smoke test, which can be like a very lightweight test. In our case, we are going to perform a curl call. So I'm going to reset the pipeline to the next one. I have some changes here, but these are some small changes, just not dealing with the paths and the directories. There you go. Coming back here, we're going to pick the next one. So this is what we've got so far. In our deployed job, we have this deployed to staging. And as I mentioned, we want to add a task here that we're going to call smoke. I also like using verbs in tasks and jobs, because this specifies an action. So I'm going to do the same thing here. I need to set the config. Actually, I need to specify a container I'm going to use here. It doesn't matter. I just need curl. So I'm going to use the previous one. And I'm just going to run curl here. So I'm going to use this dash f option, because I want this command to file if I cannot call the URL. Thank you for the indentation. That's bad programming, but it's OK. Right. And now I need to specify the URL for my app. So that's going to be, I don't remember the variable. So I need to check on my secrets, because I have everything in here. So I need to use CF domain and the app domain. Sorry? I'm going to use CF domain. Yeah, I have an example here. I'm just going to copy this down. There you go. This is the right URL. Just getting rid of some spaces here. There you go. Changing from prod to staging. All right, these are very, very basic smoke tests. I hope you have a better suite of smoke tests. Some indentation problems here. It's the joy of YAML, right? There you go. Going back to our pipeline, we can trigger that once again. And hopefully, our smoke test will run. Can trigger that from the beginning, actually. All right. So the last principle we are going to talk about is deployment to production. And when we deploy to production or to other environments, we want to do it using the same scripts, right? The same way. Sometimes we have different scripts and different tools and different procedures to deploy to environments that are not production. And then we use other stuff to deploy to production, right? So this is also a source of problems when it comes to pipelines. So we are going to use the same script. In this case here, just to make things simpler, we are going to use the same script. But the script we are using is actually these lines of code here, right? This is an example in a more elaborate project. We would have maybe several scripts. So the goal here is to use the same script to deploy to various environments. And we just need to pass the variables, right? We just need to pass the configuration. The configuration is going to be different. But the scripts and the way is going to be the same, right? So I am going to set the next version to six. Some change in the naming. That's fine. There you go. So what I'm going to do, I'm going to copy everything here, because I'm lazy. And I'm going to call this one release, right? So we are creating a new job. And this job is called release, right? Yeah. Good spot. So this job is coming from deploy. This source code is coming from deploy. This compiler is also coming from deploy. So I'm sure that I'm passing all my artifacts through the pipeline. The question was, why do we need source code as an input? We don't. So yeah, we decided to skip one important test, one important job on this presentation because of the time. That is the test. So between actually pushing to staging, you need to run some acceptance tests in your staging environment, like to test the load and test things like that. And in this particular case, the test suite is on the source code. That's why we were passing it through. But yeah, for this example, we don't need it. There you go. So I just added the release job. I'm going to set my pipeline. I'm going to get back here. And then now we have the release. And if you come back to Veeam, we are using the same scripts. Now it's hard coded in the pipeline. This is an example. But this would be a script outside in some GitHub repo. And the things that are different here are the CF space, for example, and the app name. So we just changed the configuration instead of how we deploy. Yeah, now we're deploying to production. Hopefully that will go through. What else are we missing? Is it? All right, so we got through it. I don't know how much time we have if we can show the example of the test. Right. Let me just set the final pipeline so you can see the example. You show the one that's already set. We have one that is already set to the S or not. Not in this space. That's fine. I'm going to set anyone. Right. Oh, we don't have anyone here. No, we don't have that one with the test. Actually, in the full pipeline, we have, for this example, we have another box between deploy and release. And that box is just running acceptance test, performance test, other tests that we have. So all the tests that are running side build are sort of a unit test that are related to the code, so not black box test. And the test that we would have between deploy and release would be black box test. So this test would be using an environment. Maybe the staging environment we are using to deploy. We are using the deploy job. So maybe we should come back to the presentation. There you go. All right. So just to recap the things we saw really quickly today. The chains that propagate through the pipeline immediately, that means you don't run your pipelines every week or every night or every couple of hours, but you run on every commit. And you do that on concourse by using the trigger true. Deploy into a production-like environment. You can do that with CF. It's a little bit easier. But I know that most of us have more than one foundation. So make sure that you have a similar foundation, at least. You also want to stop in any failures. Otherwise, it's not even a pipeline.  You want to only build your binaries once because you want to ensure that all the pipeline is testing the exact same code and not that someone is compiling the binaries somewhere with a different flag or a different config. And that's why it was passing. You want to run smoke tests in all of your deployments. One thing we may have missed is that we are also running smoke tests in our production environment. It's exactly the same suite, which is a curl. But you also want to test all of your deployments. And if you can, deploy the same way to every environment that ensure you that you're not testing your deployment script when you actually want to release your software, but you're testing it every day or every time that someone checks in. And with that, we open for questions. Any questions? There's one here, one there. I guess it's just out of this talk, I would guess. Normally when you push, maybe you don't want to trigger every time because you may have a lot of push. And you want to have some other branches, maybe. So how usually it's organized, like development branches and then the actual pipeline? So generally, we use trunk-based development. So we try to avoid branch as much as possible. And all the commits are going to the same branch, the same master branch. And then yes, this can happen. We can have a lot of commits in the day triggering the pipeline. But if you look at your pipeline, there are some jobs that are slower than the others. So at some point, if you have fast tests and if your build job is fast, even if your build job is going to be triggered first, the build job is going to trigger the other one. Let's consider the other one is a bit slower. So at some point, all your commits will get through until normally the bottleneck on your pipeline. So the pipeline is not going to run for the same commit like all the things. Maybe the second job that's going to take a bit longer is going to run a batch of five commits that you have in your first job. Does it work well when there are a lot of developers working parallel on different features of the same application? Yeah, sometimes it does. Sometimes it doesn't. One thing you could do if you want to use branches is to set up multiple pipelines. But as soon as you start to diverge, you cannot guarantee that you're running the same things. That's why we try not to do that. That's what we usually do is try to use more lightweight tests. And if people are submitting pull requests, we run these tests. And this gives us a first glance of, OK, so we can try to merge these. And maybe we have some more heavy tests. And we try to merge and run them. When you switch back to your pipeline, can you do that? What we normally have is a user acceptance test in between what you call deploy and production deploy.  Then you say, OK, this is fine for someone to go over it manually and do user testing, manual testing. And then at a certain point you say, OK, this chart that we built before, this is fine for release. So how do you now trigger the release stage with this four-built chart that you take from some artifact store without going through the whole pipeline again? So yeah, it is possible to not trigger the release in here. So if you want to have manual tests at some point, like a point in here, you can have a box that is not triggered. And you can run that box with the version. So the way that Conquest works, the resources will fly through your pipeline. And every job will run with the latest available version of that particular chart. If you want to control which one you have, you can use the resource page and then maybe control it in here. But we can talk about it, I think, after we are a little bit out of time. Yeah, so in this example, very quickly we are using F3 buckets to store the binary and pass the binary. But you can have other concourse resources that talk to more elaborate repositories. And these concourse resources will keep a version. And if you upload the binary, the concourse resource will keep a version, for example. And like that version, when it goes to another job, the other job will say, OK, so I'm going to ask this concourse resource to download this artifact from the repository using this version. So this could be an example. The pipeline is available on this GitHub page. And here are the resources we use for this talk, so give it a go. We went for the branching approach and for all pipelines.  And you only realize it in staging, which takes one or two hours. And then other devs couldn't push their code. Can you do that with concourse as well? There is a GitHub pull request resource. I've used it with mixed success in the past. Concourse is not really ready to deal with branches and pull requests, but there is an effort from the community to make that a little bit easier. There is a resource. It's useful. Give it a go. My question is more regarding the branching. I have understood that you recommend to having one branch would be recommended, but how do you deal with the hot fixes if you have just a small correction to be released to the customers? But the other piece of the software is not planned for the delivery? Yeah, the way we deal with it at Pivotal, with some of our products is we have, when we release a version, we branch out and then we leave a branch and a pipeline running from that branch. And then if you need a hot fix, you push to the branch and that will push that, like run that pipeline from that branch, and then you can merge it back. That means you are working more with the toggling process, right? Toggling business has to be delivered, and the rest of that should be, is that safe? Is that safe? Well, it depends what you want to do, but I think the takeaway for me is try to avoid branches as much as possible. So we also always try to do some sort of trunk base. If that's not possible, we try to leverage pull requests and plug some steps from our pipeline into pull requests and try to avoid situations where things can break down the road in the pipeline because in our pipelines, at least in one of our team's pipelines, more like the artifact goes through, more like the builds are going to take time because they're doing more elaborate things. So we try to avoid having a bottleneck down the road. There are many ways to do that. It depends on the limitations we have at hand. I think we're a bit out of time, so Derek and I, we are available to have a chat with you. If you want to know more, if you have more questions, we want to go deeper. Thank you very much.