I know a lot of folks had kind of a late night last night and it's still rather early in the morning today so I have a little poll I want to take but you don't get to raise your hand you have to stand up so get ready to stand up. I've been to all eight Ruby comps and all of them have been fantastic this one included but I'm kind of curious like what the makeup is as far as how many people so first one I'd like to do is I'd like everyone who if this is your very first Ruby comp stand up. All right that is sweet. All right.  If this is your first Ruby comp you can sit down. No Rails comp doesn't count I'm just I'm just counting Ruby comp this morning sorry. I've been to every American Rails comp so far so maybe we can do that sometime at Rails comp but so now I'd like everyone who's been who this is your second Ruby comp doesn't have to be in a row but let's see how I want to do this yes so actually if if you've been to two or more Ruby comps this one included go ahead and stand up so this should be everybody else stand up now if so now if you if this is your second Ruby comp then go ahead and sit down if this is your third Ruby comp go ahead and sit down okay if this is your fourth Ruby comp go ahead and sit down if this is your fifth Ruby comp you can go ahead and sit down same set of people if this is your sixth Ruby comp go ahead and sit down seventh and now Paul and I are the only ones left standing so the amazing thing to me is those guys that have been to all of them are still in the room but as I've spoken at every Ruby comp so far as well some reason they keep coming back for more I'm not sure why but this morning having yet another opportunity to get to talk to you about whatever I want to what I want to talk about is the fear programming so let's back up a little bit this is a talk about making stuff it's a talk about the act of creation so we just had in this room if you were in here we had a great talk about Aristotle and the art of software development and that was really more focused on philosophy but I want to talk about coding as art and as creating I'm not gonna debate with anyone whether software is engineering or art I'm just gonna say that a significant portion of what we do when we write code is art and if it is art then there's a lot of interesting things that we can learn from artists in terms of how they approach art and in terms of in particular their emotional relationship with the art that they're creating we're all software development guys we're like we're geeks analytical rational we don't necessarily talk about emotions very much but artists they don't have any trouble really they just very touchy-feely this is how I feel about what I'm creating etc and I like to step back for a minute I hear a lot of talk and have heard a lot of talks that sort of talk around this issue of how emotions and particularly the emotion of fear affect us we talk about procrastination and how to be more productive and how to hack our mind but to actually go and deal directly with the emotion of fear and how it affects our development I haven't seen very many conversations about that and that's what I want this to be I want this to be sort of a conversation and I hope that everyone walks away with this certainly with I hope I can offer a few answers but even more I hope you walk away with some very key questions that you can apply as you're going through software development and that we'll begin having a conversation about more more about how fear as an emotion affects our development so to start with everyone has fears fear affects all of us we have this emotional reaction to what we're doing I would say on a daily basis and usually it's it's in the background we don't even necessarily notice it and and many times it's a healthy relationship that we have with it it's something that we deal with we move on and it doesn't even affect us but at the same time we all have fears that do affect us that affect our productivity how many of you would like to be spending more time working on a side project that you have that you're really passionate about okay I would say that all of us if we think about that side project there is fear that affects our working or not working on those side projects and the amount of time that we spend on them it's certainly not the only factor certainly we have other constraints on our time but at the same time we make regular decisions about what we're going to spend time on and I think that fear is a very significant emotion that affects whether or not we do those things so everyone has fears everyone has different fears and fears affect us in different ways let's talk about a few different fears that might affect coding first one would be say the fear of a blank page so okay I want to start working on this project but I'm in a fresh directory I have an empty editor with a blinking cursor how do I get started and the interesting thing about fear a lot of times is you actually won't even necessarily get to that point because you already are afraid of getting to that point so you don't even go to start the project because you know you'll have to encounter the blank page another fear might be a fear of existing code how many of you have seen existing code that you're afraid of well actually there's two ways so the obvious way is legacy code yucky code that you're looking at and you're thinking oh man if I have to maintain this it might even be code that you wrote it man if I have to deal with this if I have to maintain this I'm like you get that feeling in your gut like oh no but the flip side of that you can be afraid of really nice existing code and not want to contribute to a project because you're afraid that you're going to get in there and mess things up so you can have there can be really beautiful code and I'm afraid that I'm going to get in there and and write bad code and so that's another way that you can be afraid of existing code and along those same line just in general the fear of writing bad code the fear that you might sit down to create and you might create something that you don't respect that you don't like or that you think that when you show it to other people they will not appreciate it usually if you are afraid that you will show it to other people and that they will not appreciate it's because you yourself somehow have an interaction with that code emotionally that you don't completely respect it and a lot of times again this is something that can affect you before you even write the code so you never write code because you're afraid that the code that you write would be bad another fear might be a fear of not finishing so if I start this project I've started five projects in the past and none of them are finished and if I start another project it's just going to be another failure I'm not going to finish it so you can have this fear where you never start anything because you're afraid that you might not be able to finish it now an interesting corollary to that and I see this especially as sort of an organizational fear in large organizations is a fear of finishing how many six month projects turn into 18 month projects because everybody's afraid to actually call it and say we're done and we ought to be done and we need to push this out so there can also be a fear of finishing even just you working on your side project it's easy to hack away on a side project for months when you could have released very soon after you began but be there's that fear of having to push it out and have other people encounter it and and what they'll say and and the different issues that will come up when you actually get real with it so now what I want to do is dig into this a little bit more what other fears have you encountered in when you approach code so I'm sure I'm afraid that I'm not excited about programming anymore kind of burnout okay so if a fear that programming is just not gonna be fun anymore who else I've just a few minutes at a time during a day I guess the problem I have is sometimes I'm afraid I'm not gonna finish what I start in those few minutes and I don't want to start something and then leave it when the test is failing or when it's incomplete mm-hmm is this my car yeah so that was a fear of sort of not finishing but in the small so you've got a few minutes but you don't start because you're afraid that in those few minutes you won't be able to finish fear that what I'm doing doesn't actually help anyone or isn't interesting to people that what I'm doing is actually meaningless right fear of commitment something getting a clips three seconds after you release it and then like do I maintain this do I you know start recommending something else or how does that go excellent we're all on the couch and tend to be afraid that I'm gonna do something not the right way that I'm gonna waste time re-implementing the wheel and not finding like the one method call that would make it all work perfectly right absolutely I'm often afraid that when I go to investigate the patches that people have given me these are things they felt passionate about they've taken time to make a patch I'm afraid of saying no to them so often I'll avoid going through my patch list for that reason gotcha I can totally understand that I had the fear of committing time to the project on a consistent basis anybody else burning fear that affects you when you're going to code I'm afraid of all the extra work aside from programming that it takes to make a project successful excellent one more I'm afraid that my imagination outreaches my ability I've definitely felt that before so there's all these different fears that affect us and keep us from causes to avoid what we know what we want to do what we believe would be the most satisfying I mean you wouldn't you wouldn't even encounter the fear if you didn't want to do the thing right and so you must believe at some level that that activity would be satisfying and yet these fears keep us back from approaching that so and the other I guess the other point that I have about fear and this is something I read I didn't come up with this myself but you there are both fears about others which is the one that we often think of well if I publish this I'm afraid of what others will think of it and think of me but there's also fears about yourself there's those times when you you want you go to do something and you're afraid of sort of whether you'll respect yourself after you do it well I'm if I write this code it might be bad code nobody else might ever see it but I'm still afraid that I won't like it and therefore I'm not going to approach it so fear about others fear about self here is the key point of this whole talk so if you walk away with nothing else you can just like grab this piece this is what I want to communicate all fears are legitimate okay they are legitimate because what you have to understand is what fear is fear is a warning mechanism that's all it is it's a flag it's it's an emotional warning mechanism it's basically your memories you have your memories of past experiences and those build up to a point where you've learned different things and part of the way that that comes out okay I experienced pain in the past due to something or something around this thing and so now I am afraid to approach it the key the the the thing that we have to understand is that they are just a warning mechanism they're really great as a warrant fear is really great as a warning mechanism it's really pathetic as a decision maker and so it's not that the fears are illegitimate or that we want to have no fear rather it's that we want to take fear elevate it to a level of conscious thought think rationally logically about it and then make a decision versus approaching a situation feeling that gut feeling of oh no and just decide just deflecting a different direction without even ever thinking about that reaction and so often our fear happens at this low emotional level we don't even necessarily recognize that we're making a decision simply based on this emotional reaction and so what we need to do is is look for this and watch for it and elevate it up to the point where we can make a conscious decision about the the thing that we're afraid of so one analogy to help us think about this and it really helps me think about a lot pain or I'm sorry fear is to your mind as pain is to your body right so there is an actual medical disorder in which people cannot feel pain they they have no ability to feel pain it's a very incredibly dangerous disorder to have because you can be in a situation where you basically stick your hand onto a hot stove and like lean on it for minutes at a time and never realize the damage that you're doing to your body until you pick up the black and stump right it's very dangerous to not be able to feel pain and pain is an important mechanism that you that we use on a regular basis to inform our decisions about what we're going to do and what we're not well in the same way fear is this important mechanism that we have that warns us okay well last time I took on this ugly legacy code from someone and tried to maintain it it was really painful I shouldn't do that again right or to take it back to a much more basic level a child touches a hot stove for the first time and goes out that hurt right if they don't learn fear from that they'll continue to do it over and over and over again continue to injure themselves so fear does serve a very valid purpose in the same way that pain does to warn us at the same time pain makes a very bad decision maker because one of the best things that you can do for your body one that most of us myself included probably do not do nearly enough of is exercise and what what does exercise do exercise hurts right if you spent your all your time working to avoid pain with your body there's a lot of wonderful things that you would never experience you would never go mountain biking or biking at all because you might fall off the bike and you might get hurt and in the same way we we don't touch the stove over and over again but we do choose to do some things regardless of the pain involved such as exercise or such as do things that have the risk of pain and with fear there are some things that you're afraid of and you ought to be afraid of them and you ought to let that fear be a very strong factor in your decision-making to avoid doing that thing ever again but at the same time you want to make those conscious decisions and not simply something where you're making your decisions completely out of your gut and saying oh that that's I'm afraid of that so I'm just I'm not even going to go close to it so what are the tools that we can use to move beyond making these decisions based on our fear and make elevate them to a rational logical process that we're making with our mind I think there are two key tools that we have in order to do that the first tool is more learning more knowledge more wisdom one of the the biggest I'd say one of the most detrimental fears that we can have and one that we ought to fight in every way that we can I don't think it's ever I won't say that it's ever a legitimate fear but it's almost never a legitimate fear at least and that is the fear of the unknown and so many times and we're at where and we're afraid of something it's not even like we don't even know what we're afraid of it's just this big amorphous blob and I'm afraid of that that thing right it's an emotional reaction it's not a rational reaction and so you you haven't differentiated yet what you're actually afraid of and so if you can basically learn more about that so the first way that knowledge helps us is by learning more about the thing that we're afraid of so when you first decided to take on Ruby I'm betting there was some fear involved with that move what if I go after and you probably don't even remember all that but what if I decide to take on this new language and it it's not successful I spend a whole bunch of time and nobody else is using it and it's just me all by myself what if I take on this new language and I'm no good at it or I hate it there's all these fears around it and so but if you want to learn a new language what do you do you start experimenting with it you dabble with a little bit you don't have to commit the next five years of your life to it what you're trying to do is get enough of an understanding to eliminate the unknowns well okay I'm afraid that nobody else is using Ruby so what could you do you could go and learn and do a little bit of research to figure out who else is using Ruby and all of a sudden that is eliminated as an unknown and you can continue to focus down and learn more so very often knowledge is direct antidote to fear particularly if you're dealing with a fear of the unknown which is one of the most detrimental ones the other way that learning really helps with dealing with fear sometimes you're going to encounter a fear you're gonna elevate it to your your conscious thinking you're gonna go oh I'm afraid of this and I know exactly what I'm I've I've learned enough about the fear the thing that I'm afraid of to kind of know what it is that I'm afraid of and I'm I've decided that it's work doing so I'm gonna pursue this thing regardless of the fact that I'm afraid of it well the the thing is the the fear emotional reaction it doesn't just instantly go away because you've decided to disregard it and plow on it's still there and there's a whole bunch of techniques that you can use to get through that regardless because it's still going to continue to affect you so what are some of those techniques well as coders one of the ones that probably most of us I hope all of us are familiar with is testing right testing is a wonderful antidote to fear when you've decided to take on that ugly piece of legacy code and you have to go in and change it what's you can take tests surround it with tests train it with tests to the point where you're confident then to go in and change it without worrying about breaking it nine ways from Monday also testing is a great way and it sort of ties in with another technique testing is a great way to break a problem down so a lot of times when you're afraid of a problem right it's this big hairy thing well big hairy problems are made up of little fuzzy problems and the little fuzzy problems are much easier to tackle when you're looking at that blank page and going oh man how in the world am I going to get started on this it helps a lot to say well could I write one test well yeah I can I could write one test so you write one test and it breaks could I make that test pass well yeah I can make that test pass go and make the test pass and so you incrementally move through breaking the problem down into small steps and it becomes much more approachable I mean how do you eat an elephant one bite at a time and so Jen and and not just in testing but just in general when you're afraid of something breaking it down into smaller and smaller pieces is a great way to tackle that fear so break it down until you're dealing with pieces and with problems that are small enough that you you can't really be afraid of them right or you're not nearly as afraid of them and so you at that point have enough motivation to get past that and as the thing is once you take that approach and you start working through those smaller problems all the sudden you'll go oh wow I'm not really afraid of this anymore you start to build up momentum you get in the zone and all the sudden you're through the thing that you are afraid of another one would very coding specific that I've used before is to prototype something so I'm afraid of this thing I'm afraid in particular that I don't understand the problem well enough to write good code around it I'm afraid that I'm gonna write code that is gonna be really ugly and then I'm gonna end up having to maintain that really ugly code well my one technique that I've used for that is prototyping something and forcing myself to throw it away when I'm done basically saying okay I got four hours I'm gonna hack on this thing maybe I'm not even gonna write tests I'm just gonna hack on this thing understand the problem better and force myself at the end of that to chuck it and then all the sudden that frees you up from the fear of creating bad code the fear of not understanding the problem well enough etc you just have given yourself freedom to to tackle the problem in a constrained way and an interesting thing so I gave this talk at the local ruby brigade last month and we had just a fantastic discussion afterwards about things and one of the things that kind of came out of discussion is a lot one of the real antidotes to fear is setting yourself boundaries right it's the big things that make that can be so scary it's that amorphous thing of oh I should work on the side project but you haven't really broken it down into what you should do on that side project and setting yourself constraints basically building your own set of fences it gives you the comfort then to move into that thing and prototyping with force trashing would be a way to do that another one and this is this is one of my favorite techniques actually and I use it not just in coding but in other things as well and that is doing worst case scenarios what is the worst thing that could happen if I if if this fear was realized well you know okay I'm afraid that I'm gonna start working on this project and I'm not gonna finish it so what's the worst well I'd spend a whole bunch of time working on something and it it would it would basically go for nothing well I mean that's that's kind of bad I mean it would be better to work on something that you would get more results out of but at the same time it's not gonna kill you right you won't be dead so if you can take that worst case scenario now the one thing I will say is you're all pretty creative people so you don't want to get too creative when you do worst case scenarios I mean I could probably work up a scenario under which not finishing a project might result in my death so you don't want to get quite that creative but at the same time you know apply apply a good amount of reasoning skills to saying what is the worst thing and it might be really bad right it but it probably doesn't result in imminent death I've heard a lot of people in the business arena very successful businessmen say the best thing that ever happened to me was losing everything right they were in business and they lost everything they lost their shirt but it was the best thing that ever happened to them because then they knew in business nothing like they survived the worst thing that could ever happen and so they were free then to go and take risks because they knew that even if their worst fears were realized they would come through they would learn something and they'd be able to start over so doing worst case scenarios and then the other thing that I would recommend is there are some really good books there's one book in particular the war of art an excellent book by a writer about art in particular he talks a lot about writing and and what he has found is necessary in writing it's an excellent book it's like this thick it's really quick read I read it on an airplane one time but it will change your thinking I'd highly recommend it the other book that I read the book called art and fear also good if you read the war of art and you still want more than I'd suggest reading art and fear it is it's basically it's from two artists collaborating and talking a lot about the fear that we encounter when we go to create something and ways to mitigate that fear okay now I'd like to hear from all of you what some of your techniques are to go through fear when you decided that something is worth doing how do you how do you move through that I know there's some really productive people in this room way more productive than I am so I want all your tips and tricks honestly pair programming is one of those things that helps me deal with it with the first time I paired I sat with someone who had more experience than I did and I thought that they were just gonna see me for the fraud that I was and when I was able to get past that and get past that acceleration at first or you're afraid of coding with someone I was getting feedback from them and I was learning and I realized that I didn't know a few things that this other person didn't know and that helped me build confidence in myself and in my work and build confidence in the other person so I think that pairing really helps so I recommend that absolutely I found that as well so here what I tend to do is I'll try and explain my idea to someone and especially to someone who doesn't understand programming at all and like my wife I spend many many hours explaining a problem explaining 20 different solutions and through the repetiveness of it I'll teach myself what is the best way to approach this problem excellent yeah what one thing I do to address the fear is try to identify what is the scariest or the biggest most uncertain part of the project and try to prove a solution to that first and then you feel like everything is more downhill after that right couldn't be any worse than what I already did yeah this is not specific to programming but I find it extremely useful to write down my fears on paper and ask myself why I have them excellent and sometimes talk about them out loud with somebody else too excellent just gonna keep working back this way I saw some hands back here back here way way way it's interesting actually that everyone is answering this question is coming up with kind of upbeat positive ways that they've handled their fears I think there's also a dark side to this I mean one thing for example that I think of is the I mean I hope it's more just kind of an impulse and I don't really act on it too much but for example with musical projects name you know sort of upcoming concerts this feeling that you know I can at some level handle my fear by not practicing so I'll have kind of an excuse of course that does that's not really you know the people in the audience won't know that but there's this kind of weird kind of inverse comfort to thinking well you know of course I won't play well I haven't been and of course then you have that's the solution to the fear but then you have to fight that in other words not every solution to the fear is itself productive and and positive and that's something to watch for too right absolutely yeah I mean the goal is to to do the things that satisfy you that you know at some level will be satisfying so who over here we have that one up there you know I do two things if something's like really intimidating me or frustrating me I get up and take a walk and usually by the time I get all the way around the block it's not a big deal anymore the other things if I have a really short period of time and I'm worried that I'm gonna get into something and not solve it I write some failing test cases like I focus the time only on writing the test cases I don't try and solve the problem and I see you know sit down to code like I used to read the train all the time so I have 30 minutes right right failing test cases during that time and then when you get into the office you know exactly where you were because it says so right there on the screen right so what do you do when you don't know how to write the test like if you don't know what tests are right in other words you're not sure how to test the problem if you haven't designed the issue yet then usually it's time to sit down with a piece of paper and just play with it and try and get some idea of you know what's the flow going to look like what you know if it's user interface what is the wireframe look like if it's you know a system level interface like how is it really kind of function and just do the mind map and get everything out and get an idea of like what does it look like from the user's perspective or what does it look like from the interface perspective cool nobody else okay one thing I'm afraid of is that my code after I first written it won't work which typically it doesn't so what I find is that that helps me is to actually just take a deep breath and press return and just do it yeah there's definitely something for just leaping it's interesting that you mentioned worst-case scenario because in a corporate environment there's actually a very formal process what they call risk management which involves documenting the risk evaluating the the chances the probability of it coming to fruition and then documenting how to mitigate the risk but also how to respond to it if it does in fact occur yeah I definitely didn't come up with worst-case scenario it's cool and borrowed it's kind of weird to me that I'm more accountable to other people than I am to myself like if I have a deadline at work I tend to meet it but if it's something I want to do for myself I'll put it off and off and off and I've tried a lot of these things the one that you mentioned the one about breaking it in the little steps that worked the best for me until I started getting even smaller and smaller and I tend to stop right it's the same thing as what the gentleman back there said about the music I guess it's in high school if you take a test you don't study it doesn't reflect on you because you didn't study for it so you don't feel bad it's the same idea with music but I don't really know what works that's why I'm here but I love some of the ideas and and I don't I'd love to understand why I'm more accountable to other people than I am to myself that's so I'd have a theory about that actually I so one of the ways to overcome a fear is to be more afraid of something else right so if you you reach that point in a project where you're procrastinating on something because you're afraid of it and then all the sudden it's like oh the client is gonna come after me like this is really gonna hurt so now I'm gonna work on the thing right but seriously it's this it's this scales and all the sudden the fear of the client becomes greater than the fear of whatever it is that you're trying to do but I think that that is somewhat it works into in some sense but I don't think that behave like acting out of fear is the right way to act because you're still at that point like we don't even recognize that that's what's happening and you're still at that point making your decision based on the gut feeling and not actually bringing it up and making rational decision and it means that you don't do your most satisfying work so Carl has something back here soon he has to say oh yeah all right so I've been reading a book great book I would recommend everyone called the road less travel by Scott Peck and he talks about he quotes Carl Jung who said that neurosis is always a substitute for legitimate suffering so what we do is if we're afraid of something we create some sort of neurosis to cope with that fear but what often happens is that the neurosis actually becomes worse than the original thing that we were trying to avoid the original pain we're trying to avoid and sometimes you can actually create layer upon layer of neurosis in fact almost everybody has this and he recommends that we should have no fear of going into therapy because probably because he's a psychiatrist it makes money off of it but also because like everybody no matter how well adjusted they seem to be actually could benefit from being able to analyze and discuss their life with someone else and try to see ways in which they do things that cause cause themselves harm you know and I think that that it was brought up here a little bit of that where like the not practicing was sort of a neurosis for you know actually preparing for the event and I think that if we look hard we can see a lot of ways in which we do that in our daily behavior just to comment yeah absolutely last one one of the things I do is pray I mean when you lift these things up to the God of the universe it's a fear seem pretty small so that's right so there are lots of techniques for that you can learn to help to overcome fear to do our most satisfying work right I mean the the the real problem that I see with acting out of fear is that well well and it really gets down to the next tool so more knowledge more learning but the other tool that we have for combating fear and I actually think the more powerful tool for combating fear the most powerful tool we have is love I was so pleased when Moss brought up love in his keynote because now I feel a little less funny saying it but seriously like I think that love is actually the ultimate antidote to fear because when we make decisions based on fear it's much better to make decisions based on passion for what you're doing right what happens is like you were saying you you don't you work on the projects at work because you're accountable to other people but you don't work on your side projects what you're completely missing the fact that you what you want to do is is put your poor yourself into the things that you love into things that you enjoy into things that you're passionate about and if you have that passion for something it allows you to go through it gives you the motivation to go through the things that you fear if you're not passionate why would you why would you even bother with going through the things with that you that you're afraid of what's the point and so to do your most satisfying work what you want to do is be motivated by your passion by the things that you're excited about by the things that you love and so more love is is sort of the the ultimate antidote to fear and you know this is sort of if there was a hill that if if we were fighting over hills this is one that I would die on and that is that I can't think of anything more tragic well I can think of a few things more tragic okay one of the biggest tragedies that I can think of is someone going through life not doing the things that they're passionate about because I believe people were made to do specific things and if you're not doing the things that you are made to do that you're passionate about then what's the point why are you doing anything so and if you do find that thing that thing that you're passionate about I mean in particular we're talking about sort of your your work life you're the and I think that that work is a good thing if we're talking about the things that you choose to put your hand to you on day in and day out you want to be you're not going to have overpowering issues with fear if you stay focused on your love for the things that you're doing if you're passionate about what you're doing then you go oh that's scary but I don't care because I want to do this I love to do this and I'm gonna keep doing it and nobody's gonna stop me and a lot of times the issue is that our fears build up and we don't even recognize that we've lost our passion for something and if we'll again if we'll if we'll recognize those fears saying oh you know those are great indicators that you've approached your passion probably in some wrong ways in the past ways that didn't work but at the same time what you want to do is recapture that passion this is what I love to do I love to code I love to create things to make things to solve problems and so recapturing that gives you a powerful force to just run right through the fears and that's something I I actually found last year I mean I've done a lot of exploration into into business and it's still something that I'm really interested in and I'm learning a lot about but I'm also passionate about making things and particularly writing code and I've kind of lost my passion for everything because I'd allowed myself to move away from that and so rediscovering that passion reinvigorating my passion for code actually reinvigorated just about everything that I was doing and allowed me to move past fears that I was having across all kinds of different areas so finding your passion and pursuing it is key the other part of more love in conquering fear is having people around you that love you and that you know love you I mean what's the cliche I made a website and my mom checked it out and she loved it right but it's true I mean it's a cliche because it's true we our moms empower us I hope you had a mother that empowered you by loving you no matter what you did succeed or fail and in the same way that is a powerful antidote to fear knowing that there's somebody whether you succeed or fail whether your feelings completely realize knowing that somebody loves you and regardless of that and also having those people around you that you know respect you and will tell you when you're off the track but you know that they're not attacking you personally they're trying to make you better so all fears are legitimate they're just warning signs and we ought to use them as such to make decisions we need to move beyond them make decisions decide when we ought to abandon a path based on a fear and also decide when we're gonna go through them and the tools we have are more learning and more love so that is fear programming I hope it's helpful I'd love any feedback that you have I'm not sure let me check the time here so we have some time no we're out of time so thank you


Hi everyone, my name is Ranki Magica-Dumi and I'm here to talk to you all today about innovative leadership is product management and why that is. I want to thank Product School for giving me this lovely opportunity to talk to you all today. And once again, I want to thank you for taking the time to attend this webinar. So as I said, I'm Ranki Magica-Dumi. I am a Senior Product Manager at PayPal. As most of you know, PayPal is an American company that operates an online payment system where you can transfer money. And this is of course, in majority of the countries around the world, that accepts payments transfers. And this is of course, in view of money orders and checks. And also as most of you know, PayPal is double-sided. There is the consumer side and then there's the merchant side. And at PayPal, I help large enterprise merchants to manage and grow their business. For most of my career, I've worked in FinTech. I've helped to build B2C and built B platforms. So thank you all for joining today. I always like to start my presentations by talking about my PM superpower. I've thrived in finding opportunities being on the side of others. The reason I like to share my superpower as a PM is because, well, it's gotten me through some tough days. And as my PM always, my superpower always reminds me of why I am here, which is to build the best products for my customers that would change their lives. And so I implore you all to use your PM superpower because guess what? That's how we're all going to build products that are going to continue to change humanity for the better. We got to use our PM superpower. So let's start, shall we? What is leadership? So in my perspective, leadership can be characterized as that person, that individual or group of people that have this ability to not only affect, but guide and support and lead our own entire organization. They can provide that thing that we all rally around and believe in that cause, which is to build products that would change our customers' lives. That to me is leadership. So let's talk about innovation. To me, innovation is something different. It is something novel. It is that brand-new thing and it's something unfamiliar. And innovation can be adding an existing feature to your product. You could also be building a completely different product. That is innovation. And that, in its essence, is part of our job as product leaders to innovate. So what is innovation or leadership? Well, to me, it can be characterized as a model of leadership that employs innovative, creative, and unconventional ideas. Meaning, my cross-functional partners and I are free to go dream big and come up with really big ideas so that we can again build products that are going to impart our customers' lives for the better. That's innovation, innovative leadership. So during our discussion today, we're going to cover a couple of topics that I believe help to enhance innovation. So we're going to talk about embracing diversity and neurodiversity and what do I mean by that? Well, so let's start with diversity. So I build products for global markets. The global market is ever changing every single day. The demographics in our market changes every day. Their needs change every day. So how do I succeed? How do I get them what they want? I gotta go meet them where they are. So that means in order to be able to figure out what that diverse demographic needs, I need diverse people in my cross-functional teams. So that means I need people that are diverse in gender. I need people that are diverse in thoughts, diverse geography, diverse circumstances because I need their viewpoints, even diverse geographical experiences, because I need their viewpoints, their perspective when we're talking about building a new experience, a new feature for a customer because we all represent different ethnic groups and different perspectives that we can build the best product. So I need to embrace diversity. When I talk about neurodiversity, what I mean is we all have team members in the organization who are quiet, who kind of keep to themselves, who don't really say much. And what tends to happen is those people get ignored. And I'm challenging all of us to not ignore our colleagues just because they are quiet and they keep to themselves and they don't really say much. But rather, go meet them where they are because those folks that are quiet do have best ideas to help us bring our product vision to life. Well, one of my successes is to embrace diversity and neurodiversity because then that gives me a cross-functional partner team that just come from different walks of life. They have different perspectives and they can bring those perspectives into the products that we built. And that means that we can build for an emerging demographic and we can meet them where they are. That's why diversity and neurodiversity are super important. Another thing is organizational storytelling. What do I mean by that? Well, so I have a diverse team. Cross-functional partners are diverse. My leadership is diverse. But there's one thing that binds all of us together. There's a reason why we all wake up every single day and we come to work. The core value of our organization. Because our values are aligned to that core value. So therefore, we want to come to work every day and make a difference in our customers' lives and build products that's going to help them to do their job and change their lives. And it starts with our organizational storytelling, which is, well, first we all align on the reason why we are here. Then we tell the story of our product. Now, I tell the story of my product in two ways. One, internally within my organization, with my cross-functional partners, we align on why we come to work every day, which is our core value. But then we also tell the story of how our product is a hero. Our product is a hero that's going to help our customers.  Externally, the way we tell the story to our customers is our customers are the hero and they are using our product to solve a problem in their organization. So, it starts with our organizational storytelling. Making sure we're all aligned as to why we come in every single day. Why our own value ties to our organizational core value. And that, by the way, is where the love comes from, the love for our products comes from. And once we've established that and we all aligned on why we're here, then the next thing is, okay, now let's tell our story. So, within my organization, my product is a hero. My product solves problems. Externally, for my customers, my customers are the hero and they're using my product to solve a problem in their organization. And this is super important to have that organizational storytelling. I want to talk about encapsulate alternative solution. What I mean by that is, when my cross-functional partners and I get together and we think about how the customer problem at hand and how we're going to solve it, we don't just come up with one idea, we come up with several. We come up with multiple ideas. We dream big. We don't let all technical debt be the thing that prevents us from coming up with great ideas. We look outside of our organization sometimes. But our goal is to come up with as many ideas as possible because we're not just solving that problem for the now, we're solving that problem for the future. Because even though we might even solve that problem for now, it's going to generate ideas of other things that we can build, that our customers haven't even told us yet. But because we're so close to our customers, we can anticipate they're going to have that problem. But those things happen, those ideas happen because we encapsulate. We look for alternative solution. We don't settle on one. We don't settle on two. We bring multiple ideas to the table to solve customer problems. And this is why that is important. And I want to talk about challenging our assumptions. Studies have shown that our brain does not want to do any work. Our brain wants to kind of take shortcuts. So it's easy for us to just kind of go, well, that's how it works. It's always worked that way. And I challenge the status quo. My cross-functional partners and myself, we make sure that we hold ourselves, we hold our feet to the ground, and we make sure that we're constantly challenging our own assumptions. Yes, there may be technical challenges, but that's why we'll work with our engineering lead to figure out how we can get around it. Or what we must take care of with those technical deaths to be able to bring our product vision to life. But we don't just take status quo as is. We look for alternatives. And the reason is because when we challenge our own assumptions, once again, that's when we're going to come up with those ideas, those multiple ideas to help to solve our customer problem. That one thing I always ask myself every day, how might we? How might we solve this problem? How might we perform this function? So I recommend challenging your assumptions, but challenging your cross-functional partners to challenge their assumptions so that we're not just settling for status quo for the way that things have always been. That's how we're going to build innovative products. And then I want to also chat about leading with empathy. Not just with our customers, but with our cross-functional partners. So I want to start with our customers. Why leading with empathy is so important. As product leaders, especially I'm not sure about other people, but for me, I know that I have biases and I know that I have blind spots. So the only way for me to really get around those and really understand my customer is to put myself in their shoes and have that customer empathy. So I'm going to give you an example. If I've just met a colleague and they invite me to their house, I'm going to go out and I'm going to get a regular bottle of wine, a white or a red, because I don't really know them that well. But if I really knew them that well, I would know what their favorite color is. I would know what their favorite flower is. So I would bring that with me because I really know them. Well, we need to know our customers just as closely as we would a really good friend so that we can anticipate what their needs are. But also means that we can put ourselves in their shoes and we can try to understand the pain that they are having, the pain points that they are having with our products. Or how many of the tools do they have to use to get their jobs done? What are their aspirations and motivation? We need to lead with empathy. And with our own cross-functional teams, same thing. We got to lead with empathy. We need to understand what our cross-functional teams, aspirations and ambitions are. What is their professional ambitions? What is their personal ambition? And it is our job to help them reach those. It is our job as product leaders to make sure that they are genuinely happy and they can bring their authentic self to work. So if you have cross-functional partners that are that you're leading with empathy and they can bring their authentic self to work and they are happy to be there, guess what? They're going to innovate. They're going to build best-in-class products because you're a leader that leads with empathy. And our customers, well, if we continue to build the right products for our customers and we genuinely care about them, they're going to love us. And because they love us, they're not going to leave us for a competitor. We must lead with empathy. So I want to end my presentation by talking about keeping pace with trends. It is important as product leaders that we know what's going on in the world, that we listen to whatever podcast we need to listen to to see what's happening with the market, that we understand how decisions being made through politics and environmental disasters and climate change and so forth are impacting that demographic that we are building for. That way we can be ahead of it. That way we can anticipate what the products and features they would need. Because guess what? Our competitors are paying attention to what's going on in the world and they're going to beat us to it if we're not careful. So we must keep pace with trends. We must know what's going on in the world so that we can anticipate our customers' needs and so that we can innovate and bring those big ideas to the table. So I end my presentation today by talking about product leadership. Being a product manager to me is one of the most humbling opportunities because our actions not only can inspire and motivate our cross-functional partners, but it can encourage them to dream big and not settle for status quo because they themselves can blossom into more than they are. They can grow professionally and personally. And that to me is the embodiment of leadership and why I love being a product leader. So I want to end by sharing with you all my favorite books. My favorite books about leadership, which is something that's very important to me to always grow as a leader, to make sure that I am not only growing as a leader, but as a person. So these are those books. I recommend them all. And I want to thank Product School for having me today to share my insights with you all. I want to thank you all for taking the time to attend this webinar. And this is how you can reach me.  Thank you so much for having me.


Well, hello everybody and welcome again to another OpenShift Commons briefing, this time an incredibly timely talk on security, open source security and containers from our friends at BlackDuck, Tim Mackie's with us. We're going to let him do his presentation and talk all about the goodness of what BlackDuck's doing there and some of its, with all the patchy strut accusations from Equifax as that's the problem of the world, BlackDuck's going to help us figure out how to solve those and prevent that stuff from happening and give us their insights into this space. So, Tim, without any further ado, I'm going to let you go ahead and introduce yourself on your topic. You can ask questions in the chat, I'll try and answer them or there's a couple other folks from BlackDuck that are on too, they may answer them, but we'll have live Q&A at the end, so we'll go for it, Tim. Thanks. Well, thank you, Diane, and welcome, everyone. My name is Tim Mackie, I have a senior technology with BlackDuck software and I'm going to talk today a little bit about managing container risk and this talk is going to be at a little bit higher level than just OpenShift specific for probably about three quarters of the talk and then we're going to kind of dive down into how this is all going to benefit an OpenShift environment and the container infrastructure and so forth is actually what we all want to be using and deploying and being successful with in an OpenShift environment. So, I'm going to start out with a set of assertions and my first assertion is going to be that from a development perspective that we have for practical purposes baked security into our SDLC and that we have effectively followed security-driven development and deployment model has developers being empowered with security information. Security-driven release policies, trusted components that we have as part of our CI loop, security testing is baked in that binary artifacts only get created if those policies are met and that we are signing things that we're supposed to sign and that images are being stored in trusted container registries and that they're only deployed from those trusted registries and that's my assertion, that's my starting point and if that's those nine things are actually met, the question then becomes, well, what could go wrong? The sad thing is is that quite a bit can actually go wrong and so CSO magazine earlier this year had an article out by Maria Korolov that said the easiest way to get fired in 2017 was to have a security breach. I don't know exactly how many people at Equifax are now potentially looking for jobs but this is the reality that IT lives in today and part of this is born on regulations that are global in nature so for example in the EU there's GDPR in Canada, there's a proposed legislation called PIP EDA and they're basically setting regulatory requirements for organizations that have personally identifiable information on their customer base around how they disclose, how quickly they disclose and the details under which they disclose with a set of penalties and in the case of say GDPR there's actually a percentage of revenue that's associated with those penalties and other scenarios we've seen execs like the CEO of Target actually get fired as a result of the breach they had a few years ago. And to kind of like focus this down a little bit IBM and the Pondamon Institute annually put out a cost of data breach study and there were three items in there that really caught my eye the first being the average cost of the data breach being a little over 7 million and that the lost business from it was a little over 4 million but the shocking number was that the length of time that it took to identify and contain a breach was a little over 6 months. Now if we put in perspective with the Equifax breach from last week it came out overnight that the attack vector that was used was an exploit of the Apache struts vulnerability that was disclosed in March and Equifax made a statement that they discovered this in July 29th of July to be specific and that the data exfiltration that occurred was from May through to then if you take the March timeline for that struts disclosure and move forward to the 29th of July well that's under six months which means that as horrendous as it is to make this statement Equifax actually did a better job than the average person or the average organization so this is the type of world that we're actually living in. Now a lot of people go and say well you know what from an application perspective it doesn't really matter our infrastructure guys have thought of everything so if I look at a build out I've got some users some shiny happy people on the left I've got some perimeter defenses and I've got my data center and if I build out what's a compute node inside of that data center I may have a hypervisor in place that has its own set of services which includes an SDN network for ensuring that only the traffic that's supposed to be there is there. I have some form of security service in place that is looking for malicious activity within the virtual machines themselves. I have obviously a virtual machine that is going to be containerized and this allows me to have multi-tenanted segmentation. I'm going to have a minimal OS like Red Hat Atomic and I'm going to have some number of containers in here and because this is a virtual environment I'm going to replicate it to however many container VMs are necessary. If one of these containers happens to have a component that is vulnerable things get interesting quickly so we change our shiny happy people into a malicious actor and now let's assert that that malicious actor was able to compromise that vulnerable container well they're now on exactly the other side of all these perimeter defenses and so they're now in a position where they could potentially mount an attack from a compromised container to another compromised container despite all of these structural rules that are in place notwithstanding the fact that a lot of the vulnerabilities that we see in web-based frameworks require a reconfiguration of perimeter defenses in order to even detect the patterns of attack. So all when trying to put security in place in a large scale infrastructure is truly to question everything and continually reevaluate the trust of what's out there but we should be looking at things like where does your base image actually come from if you're building it locally when was it brought down what is the health of that base image within the Red Hat Container Catalog we have now a health index that is showing how up to date is does it have any known CVEs in it when what's the patch interval what is truly the health of that image if you're running through a set of build servers before who trust them do you trust them themselves is there a way that a foreign container can start in your environment are you allowing say an OpenShift template to come in are you allowing users to go and access containers that are referenced from Docker Hub directly do you have the provenance in there if you're building base images from outside what happens if that registry goes away what happens if the tag that you're beholden to goes away or changes because after all tags are immutable what's the process to determine the impact if there's a new security disclosure all of these things are part and parcel to the trust model that needs to be put in place as we grow and as we mature and if at this point you say wow my brain it's in this is an awful lot of work right is and yes most people would be right in saying this is going to hurt their brain so let's let's take a little bit more deeper dive into how we can better manage some of this because at the end of the day we don't want anyone to get fired as a result of the data breach that these these things are manageable if you understand the information flow and so one of the things that challenges enterprises and I'd be willing to bet that this is something that challenged Equifax is that open source doesn't play by the traditional commercial proprietary software rules if you take a look at a pure upstream project compared to commercial code when a project decides that a given version is end of life there's no opportunity to pay a boatload of money there's no dedicated support team with SLAs there's no staff of security researchers there's no transactional relationship between a procurement house or a procurement team and a quote unquote vendor now obviously as you move from pure upstream you can get organizations like Red Hat who are going to very nicely provide support services and curation for all of this but when we look at the infinity of open source it is truly community based activity it's truly a scenario where if you forked if you've done anything that's deviating from that distributed component ultimately you're the one who's responsible and you need to establish that relationship so just kind of a little bit of a point on it but I look at this media wiki maintenance release announcement for versions 126 25 24 and 23 there are two key pieces that are in here there's a security disclosure that says various special pages resulted in fatal errors that's in the first yellow blob that I highlighted that's the nature of the security update so if you are a media wiki admin looking to determine whether this is appropriate or not to deploy at this particular point in time that's the information that you're working with the other thing is is that there's also a note about end of life and it says please note that the 124 6 marks the end of the support for the 124 series of releases technically this ended a few weeks ago however 124 5 had issues along with other versions so it's thought fair to fix them so the community was trying to do the right thing but they're baking the information and interesting in different ways which makes it a whole lot harder if you are on a security response team to determine if you are impacted by a specific notification and we're seeing today is that attackers themselves are getting incredibly resourceful and you need to be just as resourceful if not cunning in how you respond to it so here we have a potential attacker and this attacker has a job to do the job is to determine whether or not there's a set of vulnerabilities against a specific set of configurations or platforms they create their attack when they test it against platforms and chances are a couple of iterations don't go so well but they're very sovereign and they're going to iterate and eventually they're going to find something that's successful now that success might not necessarily be uh something that supported the original thesis but a success is a success is a success and they're going to claim victory and move on and so in order to move on they have to create a deployment vehicle or utilize one of the multitude of deployment vehicles that are out there to take this attack that they've now created and package it up for utilization now they have a trust issue themselves they need to be able to demonstrate that this in fact works so in all likelihood they go and they create a video that goes up on youtube showing exactly how their attack was able to compromise this system and gee whiz isn't this a wonderful thing you should be using it too and if this looks a little bit like an sdlc that's because it is this person has a job to do they're working for someone who has a an end goal of being able to build something out but this is something that also happens a little bit um off in the shadows and occasionally things like equifax or uh as i referred to it the pr department gets involved and nature of their vulnerability that's out there ends up getting a ton of publicity and so that increases their credibility increases the value of what they've accomplished this is their business and this is what we're collectively fighting against so talked a little about stuff that's theoretical um i'm going to actually make this incredibly real and decompose a specific vulnerability and i'm going to decompose a vulnerability that was uh made public through the process known as responsible disclosure and so under that process a security researcher uncovers an issue goes to whoever the project owner is and says gee whiz but if i do this bad things happen and i assert that that's a security issue uh collectively they work together to determine exactly what the scope of it is and create patches uh those patches are then brought downstream into distributions and the idea is that until those patches are actually released nobody outside of those core team members actually knows that the issue is happening or that that could be out there so indeed composing this vulnerability i've decided to choose a vulnerability from last fall uh that impacted a lot of the systems that we're dealing with on a daily basis specifically the linux kernel this was an embargoed vulnerability which is the term that's used uh when you're working through responsible disclosure it was given the name cve 2016 51 95 and cv stands for common vulnerability enumeration 2016 just happens to be the year in which was allocated and 51 95 just happens to be the sequence number that was associated with the block of numbers that this was run through nothing particularly fancy about it now the upstream patch was created on the 18th of october by linus and this is his commit message and i highlight a couple of important pieces in here first piece being this is an ancient that was actually attempted to be fixed once 11 years ago but that was then undone so what we've effectively established is a set of commit IDs that i didn't highlight that go back 11 years and represent the timeline for this particular issue now there's a whole series of forks that happened over that time period so there's going to be multiple branches of the kernel that are going to be impacted by the patches well the next piece that i want to call out is the last highlighted section which says also the vm has become more scalable and what was a purely theoretical race condition back then has become much easier to trigger and so what that really is saying is if we look back at the types of servers that we were working with 10 11 years ago they were single core machines maybe there's some hyperthreading we might have two or three sockets two or four sockets in there three ever actually worked two or four sockets in there so there wasn't a whole lot of concurrency race conditions love concurrency today you can get 12 18 24 core sockets and there's a ton of concurrency in there throw a second socket in and you just doubled the nature of the concurrency and when you're dealing with something that's a copy and write issue as this was race conditions can be particularly problematic so this is linus's commit message on the 18th of October on the 21st of October the embargo expires there's tons of media coverage um the silly dirty cow branding dirty cow ninjas created um where you can buy silly things in their shop including uh absorbed in the price t-shirts and um coffee mugs patches are available from uh all major distributions and the embargo expired and various people start to make uh assertions and the timeline moves forward so in the u.s and canada we have a lovely little fall festival called halloween where people love to dress up in silly costumes and kids dress up in silly costumes and go door to door and look for candy and grown-ups party in their silly costumes and we have a lot of fun at black da and um well this is madeline madeline's one of our uh field sales people and she decided that she was going to dress up as dirty cow as part of her team um and they actually won contest and that's the 31st of October now if you've got media coverage for this vulnerability the logical place where you would expect to have security information would be in what's known as the national vulnerability database the nbd sometimes people refer to as mitre um actually had no meaningful information on this vulnerability until the 10th of November so that's roughly three weeks of timeline from when the embargo expired patches were available people could dress in silly costumes and there was still nothing from a security perspective that was disclosed that's a pretty big time window for someone to mount a malicious attack but there's a whole series of point-in-time decisions and information pieces that play into this so when the embargo expired various media outlets were asserting that well this was not remotely executable it turned out that it was that information came out about six hours later when the researcher said well i figured this out by looking at my web logs so yes this is remotely executable there were initial assertions that virtualization meant that this was not exploitable that took better part of a day to say well that depends on how the hypervisor is architected and so some are and some aren't then there was about three days went by where people were asserting that if you were in a container that the namespaces effectively prevented that from occurring if you look about halfway down the page of doc's you'll see once is deadbeef.c that's actually a container breakout and that took a little over three days for that to come out use a very interesting way of manipulating the system in order to bring that forward at this point in time there's well over 80 such proofs of concept out there and so if you made your decision about how to go about patching this when the embargo expired you exposed yourself to a different level of risk than if you were continuously reevaluating the nature of this particular vulnerability and the exploits that were out there so all this timing is fundamental opportunity some organizations look at security analysis as you know what i'm going to go and do a pattern static analysis so fundamentally covarity scan for example others are going to do some injection testing others are going to do some fuzzing or some pen testing on the system in reality all of these techniques are focusing on the code that the individual is creating or the organization is creating but they're not focused on upstream and they're not focused on the dependencies that's where tools like vulnerability analysis which is what black duct out i'm into play so in a full end-to-end security model you want to do static analysis you want to do injection you want to do dynamic analysis but you probably aren't going to get buy-in from your leadership team to say run static analysis on the links kernel or run it on the docker engine or run it on open shift components or run it on your stn controllers that's going to be quote-unquote somebody else's problem that's where the vulnerability analysis comes in looks at the composition of what you've got and identifies vulnerable dependencies which is it turns out most of those dependencies and most of those vulnerabilities are actually found by researchers now one of the things that i really like highlighting is that we're all fundamentally researchers when we uncover an issue that is potentially security related we should be working with the project and the project leadership and their security team to go and get those patches out and get that awareness out there is a model for doing that with generic upstream open source projects if you go to iwanttocde.org or the distributed witness filing you can see more information about exactly how to approach upstream organizations with security issues that you see so that they don't remain latent and we don't have things like an apache struts issue that runs back eight years because the bug was there that someone potentially might have known about and said here's my work around i'm just going to work it around it as opposed to actually getting it resolved and that timeline and that lack of resolution is how attackers find weaknesses all the time so one of the things that we saw about this time last year was a 620 gigabit per second attack denial of service attack on crebsunsecurity.com and this was done through some compromised iot devices think doorbells thermostats nanny cams ebr's refrigerators dishwashers microwaves anything that's internet enabled which is these days becoming pretty much everything the vector was actually through an open ssh vulnerability from 2004 with a flag allowed tcp forwarding set to true if you look at that particular disclosure's description one of the things you'll notice is that it describes nothing like what we have today for a compute environment it does not describe iot devices it looks like it should be one of these ones where yeah this is legacy it doesn't really impact me if you dig just a little bit deeper you'll find that this allowed tcp forwarding flag is set to true inside the man page it says that well this is not a security issue and i certain anytime someone says this is not a security issue it probably is and you should be having your spidey sense going um thing yeah we want to work on that this is not a security issue because it needs to have a well-known password and be publicly connected in order to be exploited it just so happened that a lot of these iot devices had things like a password of say admin admin or admin password so all of a sudden something that shouldn't be an issue becomes a big issue that'd be a patchy struts vulnerability this actually impacted the canadian revenue agency cra which is the canadian equivalent of the irs right in the middle of the e-file tax season and this had a little bit of extra press around it because they were proactive and reached out to the media to say look we are turning this e-file system off because we are vulnerable we need to get this thing fixed as it turns out the same vulnerability from march is exactly what impacted equifax last week or became disclosed with equifax last week so vulnerability response times matter awareness matters when looking at things and there's an incredible long pale we may want to kind of point our fingers at equifax and say why did it take you so long but we can equally point our fingers at the roughly 200,000 websites that are still vulnerable to heart bleed a three-year-old vulnerability in open ssl today you would expect that that should be something that is just taking care of out of normal system hygiene one of the things that this actually plays into is what we look at as an open source development risk maturity model and this is going to feel really familiar to most people at level one this is when we're worrying about features and functions we really don't care what we're bringing in what our dependencies are the state of blissful ignorance we want to make something we want to hit that MVP so that we can actually get something out there and find out that it solves the problem because that's actually what we care about as we move forward and we find a few people forking it we find a few people who've actually downloaded and might be actually using it we get a little bit of an awakening to understand how the security implications of what we're going to be working with should be attended to but we're still very much focused on features at this point in time as we move forward we get a level of understanding we start throwing in some manual review processes some fairly basic tooling we might have some spreadsheets to keep track of stuff we might try and look at some free or low cost tools and do security scans maybe prior to each release as opposed to going on an ongoing basis this is where a lot of projects are today what we really want to get to is a state of enlightenment where we have automatic identification of all of the risks as they happen and as they're disclosed and we've baked all of this into our CINCD environment so from an open shift perspective that would be hey can I bake this into my build pipelines what kind of awareness do I have around my builders so this is the model that we want to take forward and we want to automate and there's a set of criteria that we put in place around what makes for successful automation and highlights and extracts the best information for the first thing we want to look at is the factors that impact risk so item number one is a vulnerable open source component where they where are all the dependencies coming from are the dependencies on the application side or on the the user space side that's inside the container image is a component we're dependent upon a fork or a true dependency how is it being linked in these are factors that go into the vulnerable open source side of things point-in-time decisions are another problem we all want to use stable components wherever possible but that stable might be end of life and if end of life equates to dead we might have a lot of responsibility and potential technical debt to attend are there change sets that are coming down the pipe that could make it a lot more difficult to actually update to a newer version if the need dictates api versioning issues what is the security response process for the project what is the commit velocity and who are the contributors and are they changing are they stable some of this information is actually coming out of a new initiative that the Linux Foundation announced on Monday around what they're calling the chaos project to understand exactly what the true health of upstream projects really is now when we look at historical data center operations one of the things that's a hot topic is well how quickly have you patched what's your patch process what's your patch tooling look well in reality we don't patch containers we go and rebuild them we might do some ab we might put a canary out there but fundamentally we're not in the patching process when we're working with containers but we do need to question that patch process so there are a bunch of it's in time where actually struts became more vulnerable due to the nature of whatever was disclosed against it if you happen to be at a version prior to that and upgraded to a version that was worse did you actually move from the frying pan into the fire are you the fish are you the person controlling what's going on question that process we want to make certain that we're supporting the eating of container build so that when something happens in the outside world and a decision that developer made was particularly legitimate this morning becomes completely problematic due to a vulnerability disclosure this afternoon that when the build occurs that we're performing a risk assessment crossing a ticket into a defect system like JIRA but among other things we're not actually creating that container image so that it can't actually be deployed because if you didn't create it in the first place well it's kind of hard to deploy it so make certain that these gating activities are actually happening we want to build a risk profile for every single container in the system even builders so if i'm doing a sourced image and my git environment changes and i've got the sourced image build that's going to toss it into the internal open shift registry and i've got my triggers and my deployment triggers in place say vulnerable when it comes in through a bit workflow it makes sense that you might have some vulnerabilities in your container but what happens if that builder container is similarly impacted you still have the potential for a vulnerable container deployment and so you need to have that aware same thing is true when you have a build pipeline and what steps that you have within your build environment pipeline itself has some issues can you actually trust that result in container if the various scans and staging activities also have issues they too can produce problematic results when you go to deploy the last piece of the puzzle is around ongoing changes in risk so if we assume that we've got every single test done and we've got a shiny happy object and a new disclosure happened say 40 minutes ago we don't want to be in the business of continuously scanning our running containers because that's going to impact the performance of scalability of our system what we want to do is take advantage of the fact that the container images themselves are immutable perform the risk assessment on the immutable object aka the container image monitor for the composition that's associated with that container image and map that back to the outside world and say look there's a new security issue toss a ticket into JIRA let's have that workflow in place and so have a more scalable system as a result of being able to monitor through now from my perspective we have a solution that historically has been geared towards the developer experience and the release engineering experience so being able to for example provide security information within the Eclipse IDE or Visual Studio IDE work with the various package managers integrate within CI tool chains from pretty much every CI out there integrate within the security the static and dynamic analysis tools so if you're for example going through a micro focus fortify that you've got the pieces in place wherever the artifact storage is that we can scan with that what we've done over the course the last eight nine months is bring all of this into the open shift world and that's where I'm going to go and look next so our initial design philosophy was that security response times are too long that from the point at which an actual disclosure is made to the time you determine exactly what portion of your infrastructure is impacted is far too long and with the rate of new disclosures coming out it's something that is going to be very very difficult to keep up with and so we wanted to make certain that we automated everything associated with and this is our basic architecture we have a knowledge base that we host partly because it's about 500 terabytes in size it's about to become a petabyte and most people don't really want to be in the business of hosting that every other aspect of the solution is actually customer hosted so our core application is called the hub and we have essentially it's a hub and spoke kind of scenario where various integration elements hang off of that if we look the open shift environment and this can be an enterprise deployment this could be an origin deployment they all work exactly the same way I have the potential for an integrated registry and I have image stream events that are going to hang off of that obviously I have the potential for an external registry that could be the red hat container catalog that could be darker hub that could be your own internal artifact for your nexus repository what we do is we actually in place some integration elements designed around being able to listen for activities that are happening in the system that relate to immutable container objects so a new image stream is created that's associated with an image that is within the registry's purview we'll see that create event we'll also see an update in the week as well we'll also see pod creation events so that if a container image is brought in from outside the integrated environment I will be able to see that when we see it we'll go and perform an assessment to determine whether or not it needs to be scanned if it needs to be scanned we'll go and we perform that scan scan results go up to hub and hub goes and takes a look at that bill of materials and maps it against our knowledge base to go and say here's what the risk is risks are assessed are assessed by our policy engine which then goes and communicates everything back into the scan controller and ultimately annotates the images and so those annotations are actually pretty interesting so with origin one six and open shift enterprise three six we actually have a spec in place where those annotations can actually be used within an emission control workflow to decide that you know what this image is okay to deploy this image was okay to deploy now it's not so great and of course the outside world is continually changing so the policy engine will update as the outside world changes and we'll get new notifications coming in which will also update those annotations ensuring that the state of the system is say no more than an hour out of date so effectively what this boils down to is that had theoretically Equifax been using open shift in this application that was the attack vector that we've all been talking about for the better part of the last week we would have been in a position to actually let them know exactly which images were impacted within an hour of that disclosure back in March and continuously monitor for any changes so that even if a developer happened to revert to an older version because that's what they needed to do that we could have flagged it as that happened that's our piece of the puzzle but we want to make certain that we are truly layering container security and the success criteria for a truly trusted environment starts with a platform so open shift as the platform with the project atomic host locked down with all the appropriate se linux enforcing settings in place using open scap to administer all of the various policy and governance rules to ensure that there aren't any misconfigurations within the user space or for that matter on the the Kubernetes nodes map that against the patch definitions for all the red hat products that takes care of all of the core red hat side of the equation but that still leaves the infinity of open source and that's where we take over and so we literally scan any and all container images in an open shift employment including our own providing visibility into the open source components regardless of the source annotating those images with the vulnerability information and automatically updating them with new disclosure information as that occurs without any need for rescan without any human involvement it's completely automated and integrated within the system so I've bored everyone with some slides I'm going to take a little bit of a risk here and try and do a little bit of a demo Diane how are we doing for time you're doing fine if you could do me one favor and hide the bar that's at the bottom of your screen that says stop sharing in the next here you go thank you apologize for everyone that was I thought it was only my side okay so let's go and so I'm going to show exactly how easy it is to install actually you know what I'm going to show what we've got make sure I'm logged in in the right place so here's my open shift console probably gonna want me to log in again and I actually have a project called black dot scan which I'm going to delete right now and the black dot scan project is our integration elements and so once it's actually completely deleted itself and all the container infrastructure underneath it we'll go and we'll refresh so I'm going to install and bring installation I'm going to ask for incredibly difficult questions like where is my hub server which just happens to be right here what's my username that's where the version of my hub server I happen to know that version 3.7.1 I'm going to go with two concurrent scans to make this go just a little bit faster for the most part this is the value that people use occasionally if the nodes are smaller we recommend going with one concurrent scan and in very large clusters we've seen three to be beneficial it goes and reads all the components and if I go back to my open shift console I'll see the black dot scan project in place and I have a total of five containers that are here now my infrastructure itself has four nodes and the way that this is architected we have a daemon set that is on each of the worker nodes and the daemon set is listening for any activity related at the node level to images being created and deployed when it uncovers a an image that might need to be scanned it will then go and ask for permission from the arbiter itself and so I'm going to go and take a look at our pods I'll take a look at the arbiter and we see right now he's going and assigning a variety of jobs to ensure that the scans of all these fully qualified containers are being performed if I look at a controller the controller actually consists of two containers one's a sidecar which is the scan engine that isn't being used and the other is the actual scans that are being performed from a usability perspective I could kill any one of these things off and they would restart themselves because that's what cloud native computing is all about at the end of the day these scans are being performed and the information is coming up into our uh which I'm going to log into and what we see is a set of projects that are created and in each of them there's going to be some amount of registry information so this 172 30 103 10 that happens to be the console that is associated with this open shift environment sorry we have multiple open shift environments that are coming in if an image is coming from docker hub it won't be fully qualified sometimes we get ones from docker io that are will also be scanning things that are coming out of the red hat container catalog as they're used and so I'm going to go and take a look at this this image here hub documentation the version is the first 10 characters of the pulse back so that's completely immutable and I see all of the components that are actually in here so there are 221 components that are part of this particular application there's some hibernate pieces in here there's some things that are coming along completely for the ride as a result of the user space uh and in this case there are 19 components that have a high severity vulnerability associated so I'm going to uh let's see what do I want to pick on I'll pick on uh tomcat let's see what we have from a tomcat perspective I have some new vulnerabilities in place I can see exactly what the record is get a deeper view of uh what's in here descriptions how exploitable it is see some references and a lot of the time we're going to be able to get to things like discussions around the particular fix being able to occasionally find the actual exploit code so that you can actually test it against it these are all part of the normal things that we we have with specific vulnerability but importantly if I go uh this one was a Tim Arpeggio so if I go to the Tim project and I go and take a look for images I'll find Arpeggio and I'm going to look at this snapshot this is some of the annotation information that ends up being put in place so I can see the server that this was running on the version that was there what the endpoint is and this quality.images.openshift.io.policy.blackduck this is the specification that I was referring to earlier where I'm now able to flag that this is not a non-compliant policy and so if I had a mission control in place that prevented non-compliant policies from allowing for container execution this would automatically be managed by that emission controller so we bake all this information in so that the entire interaction with the blackduck hub user interface is something that's not necessarily going to be a requirement for an operations user they can go and read out these annotations bring them into a sysdig or a data dog and do the right things based off of the the information that's present there and that's one of the the second key elements that we have is that we're not forcing people into our our UI we don't necessarily need to be there we're not forcing it from an automation perspective so it gives a very easy quick installation quick results as to what the state of these container images are and so I'm going to go back to my next line and I mentioned about the blackduck knowledge base um critical component behind all of this because at the end of the day it's pretty straightforward to figure out a way to scan an image you could just decide to go and attach to the registry and export them all um how you actually do the mapping that's where the magic comes in so our knowledge base has been around for a little over 12 years now it was designed with an understanding of how forks forks of forks parallel streams of development that might actually be merged back in as feature elements but those are actually that's normal open source that's what you want to see we enhance all that security information that we're collecting from the world with the security research team that today numbers a little over 50 people we're updating as the security issues occur so even when the struts update of s2052 and s2053 came out last week we had those in and fully mapped through within an hour and being able to map those to public exploits is really crucial at this point in time we're half a petabyte of storage but we're pulling in open source information from about 10 000 different data sources where all of github counts is one of them and oddly enough i've dated these uh these stats once a quarter in the last three months we've gone and increased the number of data sources by a thousand i really love to see exactly what those are and that's on my list to do when i get back all that we have is to be able to have full end to end visibility to be able to inventory those components that are in place math them to known security issues identify those risks manage them to your governance policies and alert when the world changes around and do this in a hundred percent automated way that requires no human interaction and that for practical purposes if a human tried to get in the way that we would be able to detect that the human is messing with the system and you're trying to prevent scans from happening as an example the end objective of course is to stay out of the news for the wrong reasons we don't want any of our employers to be in the news for what you see on the screen right now that's pretty bad that's not necessarily going to get anybody any awards but it actually could be a little bit worse you really don't want to be in the news for this having your employer go before congress to explain why they didn't do what they thought they were doing in the first place that's not going to help anybody so that that's it for my presentation but i want to take this opportunity to do a little bit of a promotion black duck has its user conference it's called flight flight 2017 is being held in boston at the seaport hotel and world trade center if anyone watching this recording went to a red hat summit earlier this year that's the same area it's being held the seventh of the ninth of this year and we've got some really really good content packed in here around devops and security research and innovation we'll have some of our researchers over who'll be able to explain some of the techniques that they're using to simplify the security well it sounds like a really good event too you know and and it's this is one of those things where it's you could go down the wormhole and ask tons of questions about each and every one of them but i think these kinds of sessions and and this event will probably be a really good way for anyone who's in the system and our security or even developers who are working on applications to get a better understanding of where the risk factors are too and where they're coming from so yeah completely and and the registration code they put down there tim 99 that's the special code that we put together for the open source events that we we do that gets you in for 99 dollars so that's a huge huge savings for anyone who has to justify to their boss that flying into boston is um a little bit pricey yeah well in boston's wonderful so yeah i encourage everybody who has a chance to get and come and hear this because there'll be lots of folks i think um one of our folks uh gordon half is going to be speaking at that event as well and that is going to do how gordon yeah and and we probably will be kicking off a security sig in the open ship commons shortly um so we'll try and incorporate some of that in the upcoming austin event in december um at coup con that we're having an open ship commons gathering there as well so security's been top of mind for a lot of folks here you know there there haven't been any questions in the q and a on the side in the chat because i think um everybody's just been sort of entranced and i think you get a very nice job of demoing how to deploy and use it all on on open ship but i i kind of wanted to make a comment about you're going to um the one thing you don't want to do is get fired for any of this stuff but if even with all of the automation if people ignore the warning signs um and and don't do it this is a perfect audit trail for why you would get fired because the auditors would have all of the information there to figure out when you were notified and um you chose to ignore it so there's there's a little back to forth and as someone who used to write auditing software auditing software uh this this up is really the metadata is now there all tagged and the annotations it's it's actually quite wonderful if you're a compliance officer in any company as well yeah and to your point about the the auditing dayan um one of the things that i assert is um uh one what makes for the lags that we're seeing such as what we have with Equifax is the sheer volume of what's coming in and the uncertainty of is this my uh do i have to for example scan my entire data center or is it only in this area and these types of analysis that need to go into it if you have a tool that can go and say these are the applications which are using this thing that has now become problematic even though it wasn't yesterday um focus here that should hopefully help with the velocity of fixes and even if you can't necessarily get it down to an instantaneous fix at least you can get to that runbook for how to resolve it implement that remediation plan a whole lot quicker yeah and one thing we we think that automations is going to is going to save the world and everything but it's also um like some of the stuff that we built in with OpenShift and the annotations from BlackDuck give us the ability to block an image from being deployed and i think that's really useful too or at least throw up some um steps before something could get redeployed or um respawn up so i think this is this is all um great stuff and i really appreciate that it's here because it saves our butt quite often um and i'm really happy that you guys are doing this and the work um so as much as the the Equifax folks have been blaming this on open source things it's really all of the pieces and parts are in place to have good automation of and and secure open source usage in your applications in your enterprise and a bit of its fud um and a bit of its um but there's a good dose of reality too especially when you were talking earlier about the dead ending of projects and there's nobody um maintaining them and if you're depending on an open source project that there is no maintenance or upgrade path um those kinds of things are big risk factors that need to be taken into account but the the same thing can happen with a commercial company too it's like commercial company can go under and regardless of the slas they're they disappear they disappear yep that's that's completely true and it's it really comes back to if your organization is dependent upon something you need to be engaged with that dependency in whatever manner um commercial relationship or community yeah so thank you very much tim for taking the time today to talk to us about this um they'll be um this will get posted on the open shift blog shortly um and we'll put up the links in here um that you found and um we'll shoot it out over the internet send over the social channels and hopefully we'll get you back again and um we'll see some of those lead times trunking down from under two under 200 days or so um and it won't be around a dirty cow or a heart bleed it's just hopefully we'll we'll have a good good story to tell um with some reduction in those times we can hope thank you Diane all right take care bye