so welcome everyone to the second episode of the questioning podcast and today joined by Juliano Calamari. No did I flush that up? Oh no! Come! Sorry! Sorry man! I think that's a food. Okay Calamari. Calamari? That's right. Calamari. Calamari. Calamari. Calamari. Like a baseball cup. Okay. Got it. Okay. Sorry man. My bad. My bad. Dilly. Dilly. Okay. I'll start. Here you are and welcome to the second episode of the questioning podcast. Today I'm joined by Juliano Calamari. Boom! Got it. An owner of a speciality coffee shop. Got it again. And I was about to say expert of coffee but you said what? Oh man nobody's an expert. Anybody who claims to be an expert doesn't know anything. Anybody who claims to not know nothing probably knows a little bit about some you. So yeah I'm really excited. Good to be with you man. I miss your face. It's been a long time. We're actually by the way speaking of it we're in Dublin close to SBG Ireland where I trained and that's kind of a short version of how we initially met but there's a lot of things I'm going to talk to you about. Oh man I'm a little bit nervous. I've never been in front of a camera before. We're just talking. The cameras are not there. But yeah so one thing actually the very first question I just feel like asking. So do you question yourself? Well every day every day. How does it work for you? Well you know essentially you know when I embarked on this journey you know in the kind of other lessons of when I started to kind of wanting to open my own thing and do my own thing I was like you know in the early stages like yeah it's gonna be great like I'm gonna do this and we're gonna do that and you know we're gonna nail it then you know high ambitions and then to to the point of you know where you're at this stage you know almost a year into your business you wake up every day just going like well what am I gonna do today? Like am I ready for today? Like are we gonna nail it out of the park?  Yeah yeah something you mentioned already leads me to another question and I think so many people think about opening a restaurant or cafe it's like it's kind of the old-school dream I think these days it's more like the younger generation they're like more like YouTube they're like I'm gonna be an Instagram star or YouTuber but I think our generation including myself I was like always like oh it'd be so cool to have a cafe I think everybody thinks of it and dreams of it but you know either nobody does it or some people try and a lot of like that's already my minority but I think quite a few of them fail because it's just such a tough business from what I know but so far you're making it work so so what do you but what do you think is the gap between where you start off with the idea of oh I mentioned like you want you want to have your own cafe but what were the realities what what do you face where you're like fuck this is not as easy as I thought well the the the reality is man is more of behind the scenes of running a cafe you know like if you run a speciality coffee shop you're more kind of like I have to pay this person you're now responsible for staff you know you're now responsible for for taxes and vat and you know you have to pay all these people you know your suppliers you know as in when you were when I was working in a speciality coffee shop I was you know you know oh look I made money you know I made people happy today you know I can do this like this is easy but not saying you know the other aspect of it and a funny story is that my boss man when I was working at the the other cafe because he he knew before I entered this job that you know it would be my goal to open my own speciality coffee shop and I wanted to kind of you know have an in-depth experience and work there which is very important in anything that you try to open kind of work in that area to get a better understanding of what's gonna come to you but every day he would laugh at me for wanting to open a coffee shop every day he was like he was just like walk past me like whenever he kind of had like you know a bill to pay or he had something from the government he would walk past me just fucking laugh and then you know walk away without saying anything and I was like what's this about and he's like oh you'll know when you're open your coffee shop every fucking month I come in you know I will get like I'll get a bill for this this and that and I'll be like oh Jesus I would remember him just laughing at me he's like you're an idiot for one thing to open this but man when you have passion I think you know that's the main key is when you open anything is if you don't have passion and that's why people do fail you know after six months or a year is because you know they weren't that passionate about it they're so so much into the idea that it would be really cool if I own this you know it would be awesome if I had this kind of you know coffee shop or restaurant or or so on and so forth but you know if they don't have the passion they're just in it for more the stature they they will close down after six months because this is a tough business it's very very difficult market to get into in any market and if you don't have the passion to be you know the best in the business and mindset straight away is like I'm gonna make this but I'm gonna make it top of the line like there's nobody that's gonna be better than me you know if you don't have that passion going in a hundred percent you're gonna fail a hundred percent but the thing is you are making it work I mean I come here from time to time it's booming with people who spoke about it that it is successful it is working it's tough work but it's working so what do you think makes it work for you you know I'm very lucky in a sense you know that I've had I've been in the specialty industry for about 10 years now you know I've made a lot of connections I've met a lot of you know mentors that would kind of you know educate me on certain things that I wasn't aware of and I've built a lot of you know because the specialty community in Dublin especially in Ireland is very small everybody knows everybody you know so having when I opened here having all the baristas and you know coffee shop owners support and backing me and you know putting on their social media you know Juliana opened unfiltered and check it out best thing ever you know having their kind of support really you know grew my kind of reputation in the industry and having you know certain type of people around me you know like barista world cup barista champions Daniel Hobarb you know having Arvind, Renata, the Latte Art Champion and of sorry Latte Art Champion of Ireland and Barista Champion of Ireland Wojtek as well works for Beulies having these guys kind of you know guide me in a sense and kind of show me like you should step away from this or move towards that you know because I'm still you know they're they're you know 25 years 15 years into this kind of business so I kind of look to them for a little bit guidance as well and they directed me to the right right approach and yeah but having that passion is definitely the number one driving force like I knew when I opened this coffee shop I was set mindset was that what I saw the speciality coffee industry kind of in Ireland was generating more you know or anybody in Dublin that has a speciality coffee shop they're not really baristas you know they're not they're not their accountants or you know lawyers or they had a bit of money and they thought they could make more money by opening a speciality coffee shop so that's what they did you know and other people that have a speciality coffee shop either like their parents are you know backing them or they had money from their family like for me it was it was my money kind of in there and I was like I'm gonna fucking make this work otherwise I'm broke you know it was all I had and and that passion to kind of be the number one in Dublin you know that's what's driving me to this day like I want to change the culture that they have here I want to show that you shouldn't sacrifice quality of coffee just because you know if you buy it from another person cheaper you know you could get more money out of it like if this is your job like at the end of the day you know if this business fail I'm still a barista like I'm not going to be a lawyer or an accountant or doctor I could you know but you know this is our profession at the end of the day you know so it's for me it's 100% it has to work like it has to work from day one day one running head start I'm like this this is going to be the shit you know what I mean what I like about your approach a lot and I think it's it fits to the main main keyword of the the podcast that I'm building up is that questioning that many people they're like I know the way and they set off to do it and obviously you can't know and foresee everything and then there's some failure and they're like oh so it didn't work out and then I bail instead of oh wait okay this didn't work out as I planned but let's change it and let's question it so so what did I do wrong what can I do better what can I improve or even if things are great I think that's that's part of what I like about your approaches from all the times we we spoke about everything you're always like it's even if things are good you're not like okay perfect let's chill you're like no but how can I make it better you're always doing that and there's a point man I don't think you were you were back in Lithuania but three weeks back man I almost closed the shop down what man I swear to god I was I was in a mind state where everything was going so good like I was everybody was loving what I was doing great you know but me personally I felt you know really depressed man I was I think you know in my kind of I'm I felt like I was conforming to some to please everybody but not doing you know what I really yeah what my what my my motto was what my agenda you know I felt like I was lacking in this you know this could have been due to the fact that I was working seven days a week you know for for the past seven months like in here every day I didn't see it but I wasn't happy so I took a step back and I was like I need an outside perspective right so I hired a consultant world barista champion Daniel Hobart he was world that was the guy who was here so Daniel came in and he worked here for two days right probably the worst two days he ever worked because there was elections going on here and there was two polls so we were hammered like we were extremely busy for two days so you know but I wanted his opinion you know his expertise to see like where what's stopping me from being great or what's stopping me like where did I hit that roadblock that I can't why am I not happy what I'm doing so you know after the two days he took he took me aside you know at the end then he would consult me he's like you need to change this you need to bring this to another level you know so after he left we changed things we changed the coffee we had a higher quality of coffee we're now rocking white star up in Belfast white star white star I think was the titanic was called white star so you're rocking white star meaning you're giving the that type of is that a type of coffee so white star is a is a specialty roaster up in Belfast so we before we were doing full circle full circle was great nothing against the guys but they were not they were not that next level in terms of what we wanted to achieve so we switched to more higher more higher product you know we got exclusivity around Dublin so nobody is doing white star in Dublin except me you know this was you know what we spoke with the owner and so we changed a bunch of things and like in the beginning man I was ready for no I don't like this type of coffee which is very understandable because not me when you made the change oh when I made the change it was horrible man like when we changed we changed from Brazil you know awash Brazil you know from chocolate chocolate nutty kind of profile dark chocolate nutty to natural nicaraguan which was wild strawberries almond biscuits like that's you know you know for people that be coming here on the regular to kind of switch them over to hey you know what smack in the face here's some fruity coffee you know what I mean not everybody you know what I mean so everybody was like oh the milk has gone off or this coffee tastes bad maybe your bins have gone off and you have to but in this kind of business in speciality you have to kind of educate them and you know why we did that you can't just be like oh that's what we do now you know you have to educate a little bit some people might want it some people don't want it so but if the people do want it they'd be like why did the coffee change you educate them we're doing this which leads to a higher brand so this is the kind of progression we're going to you know the price in the coffee changed but my price is in selling it didn't I want to keep that you know low cost down so people can afford it and try new interesting coffee that they never had you know instead of charging six or seven euro for this coffee you know my price is very low so people can come from everywhere around Dublin or Ireland and experience this new flavor that you know they haven't had experience here so this was the whole goal but the backlash was you know so people not even be like man this coffee tastes like vomit and you're like one guy told me it tastes like vinegar because it wasn't so used to the acidity of this kind of coffee he was always into that dark bean coffee you know so you know but he still comes in and drinks it he's like he said to me well I guess I have to get used to it so you know but you do get like this is this is one of the things if in your success man or in your kind of path there will be there will be times when you're feeling down or you want to change or you know you need outside perspective and you know if you're continuing with the mindset that oh it's going great but you're not happy then there's no point to do it for me those I never I'd rather close this down than not do what I set out to do in producing the best coffee in Ireland you know so if I conform to people just because you know they're happy with the way it is but you know suffering in myself that I'm not happy because you know I'm not doing to the standard which I set out to do then there's no point in this business absolutely no point I'd rather close it than conform so I really resonate with what you're saying I guess it's kind of a universal journey in that sense like you know some details about me trying to push through to the new direction which is actually one of this this podcast is one of the the step stores that but I announced my big change and got a lot of backlash from it as well and I part of it I admitted that you know it was my mistake I maybe pushed it too too much into forcing people like you said what what I like but you said that you can't just force people to be like oh this is like it's going to be that way and that's it my lesson that I learned was pretty much what you said it's you do have to kind of do your own thing but you also need to include others and educate and I really like that what you said and that's kind of what I'm doing or trying to do myself these days as well it's like instead of saying this is how it's going to be period or coming back to just doing things only what other people want that's like okay I'm going to do it my way but let me explain to you why it's awesome and maybe somebody's going to like it maybe somebody is not going to like it but but it's done but but in the end of like if people the people that you know like and respect what you do is the people for me it's the people that you want yeah you know and the people that kind of don't understand you know why would you sacrifice the customers or or or followers uh because you want to go this way and this is the point where you kind of have to you know nobody likes to be educated you know especially you know in front of you know nobody wants to feel like a dumbass like you know and explain to them why you're doing this uh but but you got to you got to give some information to us why this would be better you know and at the end of the day if they don't respect your decision you know but they're not really you know the client tell that you kind of want or or or necessarily you know if they don't if you educate them and they're like well fuck this guy and you're like you know at the end of the day you know thanks for coming the first time and you know apparently this wasn't for you and I respect that you know there's other places that might be and I respect that too please come again and give us another you know shot you know but at the end of the day I'm not gonna chase you like it's not what I set out to do in this world you know I'm not gonna beg you to come here if you want to come here it's entirely up to you you know what I mean same with your followers if they respect let's see you know where you're going and how you're progressing in life if they don't really understand or if they don't get it and they want to move on then you know bye not bye but you know I wish you come back but you know kind of like you know it's your choice I respect you you know there's plenty of things out there that's your interest and your taste you know but this is the kind of approach I will be you know taking at this point so this is for me for me personally it feels like what I want to do and what I want to achieve and I can only do it through this way you know so you either let me be me or you know you're going you know other places you know what I mean yeah one more thing I wanted to ask is initially I thought about asking you imagining that you were working in a specialty coffee shop and you were thinking why am I not doing my own thing but it seems like you had that idea from early on so my question then is how did the questioning process for you happen when you were working in a specialty coffee shop or plural and then when was the moment where you decided that it's time it's like how did the questioning process go should I still work should I still get more experience should I still get more connections or is it time for me to start like how did that happen and when was the moment where you're like fucking I'm ready to be honest man it was a kind of a slow kind of journey to when I want to open if you take kind of a step back to my beginning right owning a specialty coffee shop was never in my you know path it was never what I wanted to do okay it was not it was not since I was you know little I was like you know I'd be cool owning a specialty coffee shop and I'm gonna I'm gonna work towards that no man I was you know I was a I'm a qualified scientist like I did you know pharmaceutical science and funniest story is how I ended up in coffee is I failed my statistics exam you know and I had to repeat the whole year so I was like okay I have like a whole year to kill like I better get my ass a job so I worked in a coffee shop in in Nazar Street called Kilkenny restaurant there is where I learned you know espresso based drinks you know espresso, latte, maggiato, Americano and all these kind of things and then I got really into latte art you know doing a cool picture having people come over and be like oh that's cool take a picture come back to meet me it's like that was beautiful coffee that's when I kind of fell in love with it right so but still you know it was not in my you know career path but I could see when I was working in this kind of coffee shop I could see like other you know my colleagues this was their job and I respected that you know this is this is you know after this like they're not going to be anything this is what this is their career their baristas there's nothing else that they want to do they love it which for me was like this is crazy but I respected that right because I because in my mind like oh after this you know I'm going to go you know finish my degree and then work in pharmaceutical industries this was you know my kind of process but I saw them and they had passion and dedication and they would take care of everything clean everything and I was like this is crazy you know but I worked in this place for about four years and after I finished my bachelor's I went to do the higher diploma so I did drug development in another place in Sligo another university and then you know I was still back and forth working part-time as a barista in this place and then you know everything that we had in module or we had like an exam or kind of a problem-solving thing I would always bring it back to a coffee shop kind of you know problem and how we fix that problem kind of situations and you know after I finished my my yeah after I finished my degree I worked in industry and I was like I hate this I hate everything about this you know I realized I'm a people person you know I love people I like talking you know being in a laboratory like isolated by yourself for 10 hours working on a boring experiment every day same experiment you're like this is I want to run I don't want to do this I don't want to be with people and have chats and so you know where I was working it wasn't really more commercial coffee than speciality so but you know from working this place I've got the experience of you know the basic you know latte art and you know what is espresso and I wanted to progress to more kind of you know independent coffee shop speciality coffee shop with the higher quality of coffee you know different roasting techniques and I really wanted to get into filter you know so you know I was checking out sitting down one day and I saw this guy he was I think was an aeropress champion and I was like that's dope I wanted I want to learn how to do that so I found out where he works and luckily they had opening right so I went in there and then you know talk with the manager and man I was cocky I was a cocky motherfucker you know I went to I went there and I was like would like a job you know and he's like do you have an experience I just you know throw him my phone and my Instagram was open and all the latte art I did he looked at the Instagram he looked at means like can you work tomorrow and you know that guy hated me for like a month because I was really cocky no no no the the guy I want the aeropress Martin like you know he hated me for a month he thought I was just like a cocky asshole you know chatting to the women and you know having a good time you know but he was like you know he I think not jealous but he was kind of like you need to kind of step back a little bit like this is it's a professional environment you need to you know get back but I was always very loud and talkative so and you know I stepped back a little bit took some of his wisdom and then took other people's wisdom and then after that I was like I saw where the kind of you know the booming kind of aspect like you know from the 2007 to 2015 speciality was like wow wow wow you know but then it differed into you know coffee shops becoming more cafes and they're more food oriented you know and they're going more to that because they think they can make more money from food and leave the you know let the coffee side of it slip yeah so I saw a lot of this and I was like you know what I can do it better I'm not going to do food it's just going to be a coffee shop like we're only going to do coffee maybe something that compliments the coffee but never is before the coffee you know some pastries that will compliment the coffee but never we have food in this and maybe you'll get a coffee you know because everywhere I went in especially at the coffee shops you'd see people eating brunch like brunch is a huge thing you see people eating brunch and not have a coffee and I was always confused like why like what's going on here like came here for coffee this is a coffee shop they're like no it's a cafe now so I was like fuck that I would got like pretty pretty you know you know this heart and then I felt like you know sad and you know I saw you know my my ex bosses I was like look man we gotta we gotta buy this coffee or let's try let's try this one and they're like man that's too expensive no way I was like oh you're a coffee shop like just give something different to the people like you know let them experience what real high quality coffee is they were like no you know margins and this and that and you know probably they were right you know now seeing like now being in business I get what they were saying but if I if I can't do that if I can't buy like coffee that's very expensive and you know have the people in Dublin try it and you know what's the point of me having this coffee shop I want you to experience what high quality coffee is what you know 90 plus coffee is what is whoosh whoosh what is you know geisha you know people pay we go to Berlin Germany you know Scandinavian countries they have like on tap filter coffee you know they have geisha that's 30 euro for 1 v 60 and you know you don't see this here because you know people are you know they don't want to spend that money then who's who's gonna come to spend 30 euro on a coffee you know like nobody's gonna okay I will maybe you know once a week I will go I will treat myself to 30 euro for coffee but who else you know but then in my opinion you don't need that many people to have this kind of luxury you need that one person who really likes coffee to be like yeah give me that geisha and you know in my face lights up their face lights up it's a beautiful moment we share together it's an experience when you go to a coffee shop but it's diverged so that's why I thought I could be that change to show the people you know that it's not about money it's about experience and you know having a connection with people which is one of the the main things that my boss used to give out to me is for talking to people too much so but this is you didn't understand like my you gotta do your job and stop talking to people but people were coming because I was talking to them right and you know the moment where I was like yeah I gotta get out of here and open my own coffee shop is I had a chat with this American guy after hours was waiting for his bus it was a little bit late and I you know I closed I said to him you can chill out with me for a little bit we talked and we talked about life we talked about what he wants to do I talked about what I want to do and you know three weeks later you know he was in California and sent me a message he was like look man I had a lovely chat with you never forget learned a lot you gave me something to think about you know that's what I knew I was like you know what man I want to open a coffee shop like this is this is what's this is the real thing of a coffee shop the experience the emotions you know it's it's a central hub for people to come and chat it's not just you sit down can I take your order sit the fuck down you know get the fuck out you know I have to have that connection with people because you know like your job as a barista is to make you know if you had a if you as a barista had a shit day you can never show it to a person because you don't want to you don't want to you know give these vibes to them like you if I had a bad day and the guy in front of me had a bad day like I would say something to cheer him up make his day better so he's gone but I I still had like a bad day but I would never tell show him like you know my day shit you know but I want you to go out in the world and have a positive experience because you had a positive experience here and then you're gonna go out you're gonna have a good time do I mean maybe that person we had a shit day at work and it came for a coffee and then I made him happy so he's going back to work like fuck yeah I can take on this job you know what I mean so that's the whole you know aspect of being a barista is so much more but connectivity with people is is the main thing it has to be I feel it do you feel it there's connection it's like that's why like when I look back I was about to say something else and I'll come back to it but when I first came down here I was uh just started training at his jar and I was kind of not knowing anything around and I didn't want to be at the gym at the moment at that moment and I was like I need to find a nice coffee shop because I do like spending time in a coffee shop and I was walking around I was like well this looks interesting this is good looks like quality and I came in and obviously there's no sign and your forehead I'm the owner but but I I came in and we started talking and and it took me like a few days I had a feeling like you might be the owner but it took me a few days to be like okay that's that he's the owner but not that that that matters but the fact that we spoke and and we're chilling and also too I was doing some computer work and at that moment I was kind of homeless I was still waiting for someone to take me in and so I didn't really have much work to go and the fact that you're like oh well we're we're closed but you can still hang out and we talked and everything the coffee absolutely made me come back but that connection and that everything that's that's one of the reasons why I came back and I still come back even when I visit Dublin like I'm like oh I need to come here you know it's like it's not just because of the coffee well the coffee is amazing but that's the whole you know it's one of the reasons exactly as I said that I wanted to open my own coffee shop because I can do that right you know I mean other people they don't really care man other people they care about money like how can I make my money close at this time and you know bounce and go back and repeat it again and make my money so after a while for them it's like you know they lose the love or if they had passion for coffee they lose the passion you know because they see it now as a job you know for me it's more you know I love what I do man I love I love you know trying different origins different varieties of coffee explaining it to people get them interested in what I'm trying to to to bring back to Ireland and get them hyped up like wow just what what tastes like beer and then have them try it and then come back it's like that's the best fucking thing I've ever tried thank you so much and I'm like man I'm glad you enjoyed have that emotional connection with people even if they're here for a day if they live here permanently one dude came from Australia and he was like you know he came down he walked I think from uh double and one walked down here because somebody told them about unfiltered and uh he came down here and then you know he had to be 60 and then we were closed and afterwards I was like bro what's your plan he's like nothing I was like you want to go get a kebab so we went and got a kebab and picked up one bike and it was for him it was kind of I don't know this dude like where he's taking me in his car but you know we went and chilled out got a kebab and then after he left it's like the best time I've ever had a coffee shop you know what I mean so it's this kind of connection man is really one of the aspects that drives me you know trying you know very funky different coffees is the other aspect of it but you know and kind of sharing that with people but you know having that connection with people is definitely worth it in my opinion I agree like if it was just for the money fuck it I'll be a scientist like right you know what I mean that's it do you get me so we keep coming back to education in terms of educating your your your customers your audience and you remind me of a quick story where years ago a friend of mine from teenage years he was one of the people to start the rap culture in Lithuania or at least in the area where we were and then we met it was pretty cool but then we met a few years after and I was like oh how are you doing how's it going and and she started complaining to me that you know especially in the smaller city that we grew up in he's like ah people don't really people don't really need rap around here and I was much more cocky as well back in the day and I still believe in what what I said back then maybe I'm just more humble about it but my idea was in my response to him I said it's not that they don't need rap but they don't understand that they need it and your job is to explain to them why they need it and now that I have more wisdom and maturity hopefully I understand that as we discussed no it's not for everyone but there still can be a high percentage of people who would appreciate it if they would understand what what what the thing is about and to quickly give an example even myself I was already interested in coffee before specifically coming to unfiltered but I have to admit 100% that after you you were like oh try this try that for that and I tried and I was like oh my god this is awesome and then I went back to Lithuania Vilnius which is a great city there's there's quite a few good coffee shops that I'm already discovering but then I go to the coffee shops I'm like so aside from that usual shit what do you have like where's the real ship what do you have and they're like oh we have the the aero press or this and I'm like oh and then I try it but I understand that a big part of why I'm trying it out and I love I'm loving it is because you introduced me to to that and you showed it to me I'm not like now I just can't get enough but if you wouldn't have showed it to me then it wouldn't happen so I wouldn't understand that I'm missing it did you see the how the barista light up when you told him what else you have so much I have a lot to get so happy when you ask this me as well man I get like very passionate when somebody asks me especially you know what do you got that's weird I'm like oh I like this they ask me what do you have that's you know I know funky out of this world kind of like that nobody's doing and you're like oh I know what you want and I'm like I got you no worries man and you know and let them try natural Honduras with macerated process you know it will taste like an IPA you know so it's very hoppy so they have a taste of that and they were like well this is amazing I've never had you know and I've had coffees as well you know some people would bring me coffees that you know from one one of the customers brought me a coffee from Japan which was a very very kind of slightly darker rose Nicaraguan and then he was like well I don't know what to think about this coffee I don't know if it's the best coffee I've ever had or the worst coffee I've ever had and I was like let me give it a swing at it you know and I made a v60 from me and himself as well you know so we both kind of see if the guy in Japan brewed it the same way obviously different water and you know different things but yeah I gave him my best shot here and he's like you know it was exactly like how he brewed it we tried it and he asked me the question what do you think and I was like man I have no idea if this is the best coffee I've ever had or the worst coffee I've ever had but I was keep drinking it because I've never had this coffee before for me my brain could not comprehend like what's going on here do you like this do you not like this why I can't stop drinking this you know I mean so that's for me it's amazing like to find this kind of you know different varieties of coffee in different origins and see how you know unique roasters roast their coffee as well plays a big role in there you know so the roasters like huge huge huge role in how this coffee is going to come out to you you know what I mean but like that that moment when somebody's passionate about coffee it just sparks me up man you know makes my day but what's interesting too is that you're on a mission to do it and I am 100% sure that you're successful in it and the fact that you are not only like waiting for the people that are passionate to come in and say oh show me the funky stuff but but I like the people more that are not passionate man but that's what I kind of want to point out that you are also kind of discovering that passion and lighting it up and other people that for me it's it's there's been people who've never drank coffee that come here right that never drank coffee that certainly you know you start them off small you go you know a latte have it more diluted with milk but they've they started from not drinking coffee to drinking lattes to drinking you know cappuccinos you're going to bring them down to flat whites and then take that off try and espresso do you know what I mean and then switch them to filter like this kind of process you know to kind of move people they were like I've never drank coffee before too man every time I need your coffee do you know what I mean like that's a beautiful feeling as well man like I'm more I like you know I like hard hard situations so you know if somebody is interested in coffee it's very easy to to kind of like you know you'll be interested as well you're like yeah you know it's pretty easy but if somebody's kind of like yeah I don't like coffee and you're like come here do you know what I mean I like I like the hard challenges that that's also very satisfying man when you kind of convert somebody to specialities it's very satisfying very very satisfying yeah speaking of challenges one more thing I wanted to make sure we we touch upon the record on the record something we started talking right away off the record when we met today it's about that challenge you're looking at right now and questioning about the next step and basically you can talk about that right so describe the situation it's about taking a bit of a step back as the constant face of the coffee shop giving more responsibility to others so you could expand and grow into your other shit so but that's that's the most difficult like you know for me that's as as you know as a business owner and and you know as foremost a barista you know that's the most difficult part for me man to kind of step back and let let you know your staff kind of take lead because I've been you know since like eight eight months now almost nine months in business and I haven't taken a day off you know you're so immersed in it and you have you want everything to go right and then you feel if you're not there something's gonna go wrong you know but then if you're working all the time I didn't realize that even I made mistakes you know like I probably made more mistakes that I could have avoided if I had more time off the business you know instead of like always being there and kind of like you know micromanaging everything like you know you hire you know my staff are highly trained so they know what they're doing you know at sometimes they've even taught me some things I was like god damn I was like I've been doing that wrong for 10 years and they were like what you know so you know that's you know when you hire the right people you feel you know more confident to step back but in saying that I haven't I haven't taken that step back yet so I'm in the process of you know hiring more kind of staff took in that you know that are immersed in this in this culture immersed in this in this industry that they're passionate what they do you know so you know paying them the money that I think from that kind of expertise that they have that they deserve and kind of step back and let them you know be be be the face of the company you know like at the end of the day like I've set up this business and it's of high quality and then the staff of high quality so they can manage by themselves it's not a difficult thing to do you know it's just you know it's not it's not a hype you're gonna lose a lot of money it's just you know they don't need to be micromanaged I don't need to be here they know what they're doing they're highly trained they know what's going on but it's just a fear that I have that if I step back something will go wrong do you know what I mean yeah but in saying that like I've made so many mistakes not stepping back you know I'm so exhausted sometimes I have to throw away shots because I've let it go over so I'm like oh you know I'm out of it today like I need like two minutes and like sit down or you know kind of this situation like I think like it's the best the best thing in a business would be you know you've made it to that point and you've achieved what you can from so but if you don't stick back you can never reach that next level you can never reach that next ambition or goal that you have because if you're here 24-7 you're blinded you know what I mean you're like a horse has these what's this thing so he's only seeing like straight right so so I'm at the point in my kind of career that I need to step back and re-evaluate you know how can we do things and go on other directions or you know improve on certain areas or you know maybe bring in certain equipment or you know what if I want to do roasting or what if I want to open a school you know I want to train people and be the best you know at their careers as they can and in the barista careers so you know so I'm at the point where I am stepping back now I will take three days off you know that's the step three days then four days and then you know just work on busy periods and focus on other aspects and how can I grow and improve and you know bring that next level of quality that nobody's doing in Dublin you know but you have to step back make sure you step back yeah before I lead up to the last question every time we talk you need there's always some next big thing for you yeah which I like a lot that you have that mindset of always growing more so to show that do tell what's what's on your to-do list like what's what's the big thing I want to open in Japan okay that's that's like 2021 man I want to there's this place in no I'm not throwing hate now I'm not throwing hate but there's this place in Kyoto it's called 100% arabica so especially how the coffee scene in Japan is not you know it's not really it's in the infancy stage you know there's this dude that you know very dark roast very I'm not even going to say which shop it is but very dark roast but beautiful latte art right it does the whole thing it's on point the shop is like clean it's very spotless and you know but I waited in line for like 20-25 minutes and then people in front of me you know they got the coffee took a picture and threw that out because it was disgusting you know I tasted it and I was like this is really bad it's very dark roast it's Vietnamese coffee you know it's really bad but it was back so my goal is to open beside that guy just just to piss him off yeah this is the plan open in Japan but also man what I want to do is I want to do a sub zero freezing of very expensive coffee so I want to have a filter menu uh so when you kind of get uh you you you weigh your grams uh uh you vacuum seal it right so if you want to do a very expensive geisha or a whoosh whoosh or or panamanian anything and you know it's you know per kilo it would be like 400 500 reaches up to you know 3000 so buy very expensive kilo of it weigh it out uh vacuum seal it and freeze it so you'd have a filter menu that can run all year round so you don't have to worry about I've got to get rid of this coffee because before it goes bad so you know so in that way you can keep very expensive coffee all year so people that do come once a month that are looking for this type of coffee bro you know where to come you go down to the filter you get some good stuff you know I mean so this is the next kind of a phase uh that we're gonna do uh there's other things man but that's kind of on the down low I don't want to say too much and the other things but but Kyoto sure 2021 I'm coming for you dude you know who you are uh and the the sub zero freezing I think this is this is a big step man in improving your filter game 100% last question yeah uh for anyone who does have that idea of I wish I would have like coffee shop or cafe or whatever what would what would you share with that guy who's that's you know you sit down and he tells you that you're looking at him there's two things but I would say and I think this is one of the reasons why I kind of I could I can focus on on on on the quality of things without worrying about the price okay do it while you're young man do it while you're young and you have no ties you know there's two two things for that if you if you're you know if you're kind of starting it at you know it's fair you can start it at 50 you probably have more money than I did you know at you know 30 28 you know so you could be financially good you know and you can you know run it but when I started at 28 man I had no mortgage you know no babies no marriage nobody you know I didn't have a lot of things on me that I need to pay you know so when I opened my business it was you know all I had to pay was staff and rent electricity suppliers that's about it I didn't have a mortgage I was like oh we gotta make this much money today or else I'm screwed for the mortgage you know I didn't take a loan out so this is one of the things when you start out young you have that also very you know drive and ambition you sleep like five hours you'll be fine you know because I've seen people who open a business out there you know 45s and they can't you know manage with the lifestyle of you have to constantly be awake you know that's where they're like man I need at least 30 hours sleep you know but when you're kind of young you're like sleep what's that man forget about sleep and so that's the first thing I would say man kind of save your money opening at the young age when you know but when you open at the young age man you have to have very good people behind you you have to have mentors to kind of guide you because you don't know shit like to this day I don't know anything I always go as I always go like man I just I don't know anything as much as I could know so I'm always willing to learn from other people and how their experiences like if a roaster how he's roasting how can I you know improve my coffee by knowing how he roasts you know like how can I sometimes you know there's a coffee there's a defect in the bean you know and if you're not if you haven't learned from the roaster how you do you know that defect how can you describe to your roaster oh bro there was you know so such is such a potato defect in this bean it smells like you know really bad so you know throughout my my my kind of career like I've always learned from other people who had more experience in other things you know so it's good to have a good team on your back and you get that information so the third thing I would say before you kind of run into your your dreams is man make a plan like have a solid plan and have solid kind of money backing you one of the things that that I kind of you know got lucky on was I didn't have a plan I kind of like was oh look man here's the keys you opening this next month you know so I kind of like I had no plan I was like okay I have this much money how can I make it work you know it's like in in the beginning on the first day I was you know I had like a thousand euro to my name on opening day you know I had a thousand euro and I was like well I'm screwed if this doesn't go well I have a thousand euro to pay everything at the end of the month so this business is not gonna go well you know I'm one month gone you know that's how long I lasted but unfortunately everybody was digging what I did so you know it went well but if I didn't if I wasn't who I was and all the experience I had and you know it would have been really really really bad you know and the fourth thing is work if you want to be if you want to open a coffee shop work in a coffee shop you know work two three four six months work a year in a coffee shop because at the end of that year I guarantee you you will either know if you want to be if you want to own a coffee shop or you don't you just know you know if you want to run a restaurant work in a restaurant and I guarantee you after a year you're gonna be like yep bad idea this is not what I thought it was you know so it's it's really really really really good that you work in the area that you're trying to open really beneficial that's that's the biggest one for me you got to work in your area because everybody when I open they've seen the success and you know they come here and they're like how did you do it I want to do it I'm like you need to step back work man go to a coffee shop work there for free for one month and then come back to me and tell me if you like it you know there's gonna be people that really like this energy and there's people that's gonna be like hated it I hated it I don't like people you know I mean so how if you don't like people how can you open you know a coffee shop yeah that's the one that's the one thing you have to do is like people so you know that's the biggest thing man work in what area that you want to open huge huge thing huge thing yeah actually that you said it when I look back I thought about YouTube and I guess YouTube is a bit different to a degree it's a lot about trial and error it's kind of part of the game but I guess it is everywhere but but when I look at my Aikido career when I had the Aikido school I lived in an Aikido school for three whole years I mean I trained in a different one for four eight years I was instructing at both of them and and now one after what you said I look back at it when I started middle school obviously there's a lot of things I had to learn as the guy but there was so much backing me up from the fact that I was a student I was an instructor I was helping out with the management I was helping with everything I had a very successful school and now that you said it after you said it probably a big part of it is because 100% because you were in that you were in that in that in that mentality you know you were from the get-go there so you know if you stayed there for two years you were like I like this I want to continue it you know and then you ran your school successfully because this was your passion back then right so if somebody is just has it's a prestige to open to own something you know it's not it's it's hard work man it's it's so much hard I probably make less money than working in a coffee shop like you know you have to have that like I knew that from the beginning like you're not gonna make nothing you know but if this is what your passion is I was like doesn't matter money it's not important to me I want to do what I want to do you know I mean and you know as long as I'm doing that I can never fail even if I'm not making money I was like don't care quality of life this is my passion I want to do it but man if you go in it you know because it's statue or because in your mind this is going to make you big money forget about it don't do it man go for the passion man go for the love of it forget about the money money should be the last thing on your mind for people who are visiting Dublin or are in Dublin not yet Tokyo a couple years but starting with Dublin how can they find you or even if they want to check you out online so what's right we don't really you know at this point we kind of we went through word of mouth you know we don't really advertise as much as we should or you know spread the word there but we kind of we only have uh we have a facebook page I'm never on it people like Instagram Instagram is a big thing that we use uh just Instagram you know usually people find us through that way or our friend has told them about this place so it's mainly word of mouth or Instagram if you guys on Instagram unfiltered coffee co look us up and we funny we funny on that we funny on Instagram very funny sometimes naked what about in Dublin itself in Dublin man we are in in Chiquor so Dublin 8 so uh 205 B Emmett Road in Chiquor Dublin 8 there's a little bit outside of the city but there's a lot of history in Chiquor that people are not aware there's the war memorial gardens just across and there's a lot of uh kind of you know the the the core you know the the real people are in Chiquor like they're they're most harmless people you'll ever met is one of the reasons because I grew up here and I was like you know this place was available I thought it was destiny that I opened here in so far it's been amazing it was a pleasure boom thank you we're so white haha we're not cool


So it's pretty exciting to be here. This is kind of a fun topic for me to talk about. It's one that I feel pretty passionate about because I've seen it done badly so often. And I've done it badly as well. So some of you may know me, some of you may not. But I do have a blog. I podcast as part of the Ship Show podcast, which is release engineering and all kinds of other stuff, as well. And you can find me on Twitter at slasher underscore D. I also recently started working for Ops Code, which I like to at least disclose at these conferences because I was not accepted to speak when I was an Ops Code employee. So good to know. I mean, I've always been pretty suspicious of vendors. And this microphone is a little weird. And so I like to always let people know that I am now actually a vendor, which I find hilarious a lot. Can everybody hear me? OK, this keeps kind of sliding funny. OK, so credentials. What qualifies me to tell you anything at all? I have been doing configuration management since 2010 when I met it. I had no idea that tools like this existed at all when I met Chef. And I was like, where have you been all my life? And it's pretty much been configuration management ever since. I also have a long background in systems, mostly retail web ops and things. And I have seen a lot of inconsistent environments and things like that over the years. And so I have a lot of good stories and anecdotes to tell you about. And I think that that will be fun today. I want to kind of talk about why we want to have consistent environments and some of the things that can go wrong when you don't have them. And then talk about a few ways to focus on making them homogeneous across all of them. So we have a problem. Our environments are run in a muck. A lot of times what happens is we build an app. And as it promotes through environments, we refine those environments. But a lot of times the refinements don't make it backwards. So production, everybody cares about production. But people care less about other environments. And as we make things better and better in production, a lot of times things don't get migrated back. Or different teams are responsible for different environments. And they don't communicate. And you don't really know what's happening in production or how things happen up there. And so you just kind of figure it out back in depth on Steph. And this doesn't really fly if you want to do continuous integration, or continuous delivery, or continuous anything. Because nothing is going to be continuous if your stuff is different everywhere. Because you are constantly going to be stopping and trying to figure out why things are different. Because things are just not going to work reliably ever. So environments run a muck. One of the ways that we can cope with this is actually working with configuration management tools. And what that can help us do is homogenize across environments, and help us make our process more consistent. Because what we end up with is a thing where the larger your organization, a lot of times, the less communication you do, the less you know about other environments, and then things become more and more out of sync over time. So the next thing really just shortly, you may actually think that I'm up here to talk to you about Chef, and I'm not really. Everybody knew, even before I started working for Apps Code, that Chef was my preferred tool, and that I like it a lot. But I have a real passion for just configuration management in general. And I think you should be using it. And I think that it doesn't matter to me as much what tool you're using, as the fact that I really think you need to be using one. So I'm not here to talk about Chef. A couple of the examples I have are going to be in Chef, but as far as what's going on here, you should be able to do anything I'm talking about with any of the major configuration management tools. So don't worry too much about that. So it's really just what you do with the tool. Don't worry about which one you have, or which one I'm talking about in here, or what the examples look like. So environments, who cares? So environments are where your app lives.  The environment is really what you put your app in. And if it's not the same everywhere, you're going to have problems. So you have to care about other things too.  And so it's important to realize that when you have a dev environment that is different from tests, that is different from production, you can run into problems. So what kind of problems can you have? Does anybody here not believe me, by the way, that you need the same environment? Because I have had these discussions with devs, who are like, well, we don't care what the deployment looks like in production. Or we're just going to build our Jboss JVMs like this. We don't care that they have nothing to do with what they're doing. I have had these discussions for real. OK, so good. I'm glad that everybody believes me, at least, and that we all agree. But in the meantime, let's talk about a few fun things that can happen when you have out of sync environments and process. Eventually you get something right to, say, stage your production and somebody else is doing your deployment. And all of a sudden they're like, well, I tried to do this thing, but it didn't work. And it turns out that there were actually three other steps that never got into the enormous spreadsheet that is being passed around. So there's something like that. Or you might have deployed something by hand and forgotten about it, and then done it in the next environment and forgotten about it. And now it's prod, and it's 3 AM, and we've forgotten to do it again. So that happens, or this actually I have seen several times, where the wrong database connection information is deployed to production or a bad password, like the dev password for something ends up in production instead. 3 AM outage is no fun for anybody. Somebody fixed an SMTP server setting on the server by hand and then forgot about it. And so then when we build another server in a couple of weeks, that server also has the wrong SMTP server setting until somebody comes in and fixes it by hand again. And we keep forgetting to do this over and over, and it just becomes one of those things that never gets done. This is one of my favorites that we had. It was really obscure. We were migrating a giant website, and one of the apps in the website actually had really special, weird SSL configs where the Apache server actually had to talk to the Java app with special configs, and nobody knew how it was set up. Nobody knew how to make it work. It just kind of worked in tests for the app, and it had for years. And it took them three weeks with five people getting this app working when we re-built it. It was just amazing. I mean, something like this, you could just, once you have these configs in your configuration management, it's so much easier. Nobody has to guess at stuff. So and then the last one that you see a lot is people pushing SSH keys by hand, whether that's for batch jobs or just general administration or who even knows what, but this happens all the time. And then you end up with a giant list of SSH keys somewhere that you have to maintain that are never maintained anywhere, but in some directory on an admin server somewhere, right? Not even someplace safe. So I mean, what happened? Applications are complex and organic, and things just grow, and they can get out of control. And people don't give as much love to non-production environments. And this is really a problem, and you really do need to care about this. So I really feel like we have two failures here. One, I'm not going to talk about too much, and the other one I'm going to talk about a lot, right? The first one, we have a failure of communication, because people don't talk. The devs don't talk to people running production, especially, I mean, sometimes it's possible that you are also managing your own production. But it's pretty rare. The bigger your company, the less likely that's going to be. And the more layers you have between you and the people managing your production infrastructure. People don't talk. They don't know what's happening. They don't want to know. They don't want to talk, right? Half the time, they don't like each other. That's a problem. I actually think that was going to be funny, because it's just life, right, in a big company. So ownership is the other thing. Nobody wants to own devs, because it's gross. It's a mess, right? So these are the things, right? Communication ownership. So first of all, people got to talk, or none of this is going to work. You got to talk to the people you don't like, and you may actually find that they're not that bad. So, and I'm not even talking about dev ops, right? This is not dev ops. This is basic human communication. You got to talk to people, or nothing is ever going to work. And that's really all I have to say about that, because I think that there's probably already been a lot of really great sessions today on organizational stuff and communication. So I'm going to move on to what I feel is more important, which is technical ownership of non-production environments. So Dev and Test are really kind of like the Cinderella's, right? Nobody wants to own them. They get out of control really fast. They become a mess, nobody really understands them, and they kind of work, and they kind of don't, and they might work, and people don't want to touch them, and they get more brittle over time, because it's just a mess, right? So, because they're complex. As you get more apps, and you get more people working on stuff, you just never know. Manual deployments happen in Dev all the time, and people just kind of figure stuff out. When they don't work, they just kind of copy files around, and kind of figure stuff out. They're unloved, nobody loves Dev. Oftentimes, Dev is like somebody's responsibility among 20 other things that they're working on, right? Or it's, the Dev team is responsible for Dev, and they just want to write code. So if it kind of works, that's cool, right? And over time, they become unlovable, because that brittle thing, right? They just, they get worse and worse over time. The ownership becomes less and less, we have nothing at the end of the day, right? Cause nobody wants to own this stuff, and so hopefully it just kind of works. But unfortunately, they're really necessary for us. Lower environments, who in here is ops? I'm just curious now. Dev's business? All right, so this is the thing, especially for ops. We have to remember this. Dev is production for Dev. These people, they're using it all day long. Sometimes they're using it all night long. So I mean, lower environments are production for QA, for developers, for user acceptance, and a lot of other people. And so letting them languish in a state of barely usable existence is really not gonna work out long term, because what this does is it causes development teams to lose hours and days of time, trying to figure out why things don't work. And so it's a memo to Dev too, right? I mean, somebody has to own this stuff and you have to care about it. You can't just keep making the release engineer try to figure out why stuff isn't working, right? Cause eventually, the release engineer's gonna quit. So and what happens is that everybody is miserable using these environments. They have to use them all day long. And eventually what you have are miserable people because all the people who wanna be happy are gonna leave. So there are so many things in your environments that are not code. A lot of times what we find is that people are putting integration information and other stuff right into the code base, right? And we wanna avoid that. None of these things belong in your build. Packages and versions, obviously not. Once in a while though, some things get packaged up, right? Also, mail server information, multiple data center information, database users and passwords, connection strings, integration URLs, climates, batch jobs, all of these things, right? I mean, these are things that should never be in your code base. They should never be in your app. You shouldn't be having to rebuild your app because somebody changed the database password or somebody is changing the connection string for whatever reason. Or you added in a third data center with HA stuff, right? You should never have to do this. You should get your stuff outside of your app. So this information is always changing. So and it's really hard to know what the definitive information is, especially if you're in Devs because Devs are always the last to know what they are supposed to be connected to. So what are you gonna do?  Like there's probably a config set on the admin server, right, with like all the integration information or there's a wiki page, right? That hasn't been updated in six months. Or there's a spreadsheet from the last project last year that has integration lists and ports. So configuration management offers you a way that is reliable, consistent and visible to people. So one of the really great things about configuration management tools is that we now all agree that this information should be in version control. Everybody can see it. I can see how the servers are built. I can see how things are configured. I can see how people are constructing database strings. They're not in the code base. And if I need to make something new, I can actually go over and look and see how things are configured. And that's really empowering for you to regardless of what group you're in. So some of the things that you can do, some of the things that we can talk about with configuration management, right? We can eliminate mistakes with it. A lot of the things that I was just talking about were mistakes, things that are done. People do them by hand, they make typos, they make a mistake doing the build and all of a sudden you have the dev password in the production war file, right? Something like that. We can ensure consistency across environments, across servers, across data centers. We can automate the complexity. The complexity is not ever gonna go away entirely, but we can get our fingers out of it, right? I mean, the fewer things that we do by hand, the more likely it is that life is gonna be good for all of us and things are gonna make it to production in one piece and work. And we can talk about separation of duties. I don't know if, is the compliance signed? We can talk about separation of duties and why configuration management makes this easier, right? So we can eliminate mistakes with configuration management because the more manual steps you have in your environment is an invitation to human error. It's not malicious. People don't make mistakes on purpose. People don't make trouble because they're angry. They, you just make mistakes. My life is typos. I spend more time fixing typos in my code than I do in writing code. I mean, really, that's just the way it is on a lot of days. And so these things, you know, tools exist for a reason. Get things out of your pipeline that's manual. Don't have people running things by hand. Jenkins exists for a reason. Configuration management tools. Puppet, Chef, Convict, CF Engine, whatever. These all things exist for a reason. And the fewer things we touch with our hands, the less likely it is that we're gonna break something. So it's really important to think about stuff like this. What about Apache config that I'm talking about? If somebody had written that ahead of time into Chef or into Puppet, people would know about it. And at least, even if it wasn't still accurate, it would at least give people a clue as to how things were configured, as opposed to the 30 backup files that were in the config directory that nobody knew which one was the actual write one. So, get your people out of the pipeline. It's the fewer fingers you have in there, the better things are gonna be. And honestly, it's more fun to write pipeline code than it is to run pipeline scripts, really. So the other thing that we can do is ensure consistency, because a lot of times what happens is the apps team is building production with one way and then the dev team is who knows how they're getting their kick starts or their app builds, right, their server builds. They're getting it however they get it. They might get a server delivered one day with one version of the OS on it and three weeks later, they might get another server with something else. So you just never know. So one of the things that we can do with configuration management is stop building our servers by hand. I don't actually think most of us are doing this anymore. I think we actually have mostly evolved beyond that, but I mean, stop making people guess how the things should be configured. If you are making things in production one way, migrate that back, make that information freely available, make it easy to use. So don't just provide people with a server where there's a kick start server. Actually that won't actually happen a lot of times because it's production and people can't actually touch production kick start servers, right? So there should be a way for people to bootstrap servers and dev that allows for a mimic of production without actually giving them access to that. So I mean, there are ways to do that. A lot of times what I find is that there is a dev kick start server that has no love, right? Nothing, no love. It was configured two years ago and somebody might be reconfiguring it or they might be just re-kick starting that things with the two-year-old kick start and then fixing stuff by hand. Cause that never happens. So we can ensure consistency. We can make things easy, fast, reliable, right? One OS to rule them all. Make sure that your OS is the same everywhere. It's such a simple thing to do, but it's so powerful if you know that everything is the same from one environment to the next with your operating system. Otherwise packages run amok all the time. Versions of things run amok. Even kernel versions sometimes are different. And trying to troubleshoot that stuff in production is the worst because it's so low level trying to figure things out that a lot of times you're just not gonna notice until you get to production and you have a performance problem and you don't know why. And then all of a sudden someone's writing scripts to compare package versions across 30 servers. Has anybody ever done that? I've worked with people who've done it. I haven't actually had to do it myself, but somebody on my team has had to do it for stuff. Right, so ensure consistency. Packages, I have a real passion for package management because I hate it, right?  I hate that I'm still stuck at this level of the operating system and management because I would much rather be doing stuff at the application level and making beautiful deployments and things. So yum and sell Tomcat with your keyboard, don't do that. Don't download some random Tomcat and then put it in a download server somewhere locally, right? Don't, don't do this. Just don't do it anymore. Find your package that you want. Use your version control and use your configuration management to ensure that you have the one that you want and that it's the same everywhere, right? This is a chef snippet. I think Puppet looks almost identical to this for the most part. They use insure, but it's pretty similar. Or you can even do this, which is basically define your version outside of your code and then you can have production versions and then you can also have the version that you're bumping in dev while you're testing things and you can ensure that you are testing things separate from what's happening in production. But you still know what environment is on what version at all times because that information is defined in your tool and it's in version control, right? This is a big thing for me. Locate your environment-specific configurations outside the code base. I've said this a few times now, but I keep wanting to say it because people keep wanting to put this stuff in the code base and then we have to rebuild the war file every time something changes and I'm like, why do you wanna do that? Why? Why do you wanna do that? Why do you wanna manage any of those things? Get all of it out of your code base. This is one of the biggest things that you can do to make everybody's life happier. Abstract all of your data out of the code base, right? Put it in Chef, put it in Puppet, put it in a jar file in Artifactory, right? I mean, I've seen all of these done. You don't even have to have a config management tool for this, but I prefer it myself. But I have seen people do all of these things but get it out of the code. Do not have to rebuild your war file when connection data changes, when an integration URL is changed, when you add a new one. I mean, just, you don't wanna do that. Anybody wanna do that? Just checking. All right, we can automate our complexity. We can use configuration management to automate our deployments to a certain extent. We are never gonna get rid of complexity. Deployments are complex creatures. We hear there are a lot of moving parts. There are app servers, there are web servers, there are load balancers, there are cache servers, there's database schemas, and all of them are versioned and often they require some orchestration to go together. But you can do quite a bit to make your app and your infrastructure self-aware with deployments, especially with the simpler side of things. And instead of running around to servers like you do, right? You've got this big long list of checklist things that you have to be, you gotta turn off the app servers. You gotta make sure that nobody's taking traffic, right? You gotta make sure that everything is ready to go, that the database is the right schema number, right? And that everything is pointing to the right thing before you actually go in and run the deployment scripts that may or may not work, depending on the moon phase or whatever it is this week. I mean, really, what you can do is, what this will do is make it so that your app can be just a little bit more stable with a little less hands-on kinds of working with it. So app versions, one of the ways that you can do this in tools is to have an app version attribute outside of your tool or inside in an attribute in your tool, right? So you have app whatever, 1.2.2, and everybody knows that that's the version that's running in production. And then if you bump that version, then that should trigger several things to actually start looking for a new instance of your app version, as well as understand how to stop taking traffic so that it can actually like do the deploy of the app and restart itself and things like that, depending on the complexity and the amount that you can communicate between teams, right? Because this does require a fair amount of trust, but for continuous delivery and things like that, this kind of stuff is key. The ability to make your app infrastructure self-aware and fault tolerant is key to a lot of the stuff that you wanna do with continuous delivery. You can't do it without trust and without some automation. You really cannot. So finally, anybody here in compliance or security? No, well, this isn't nearly as funny then. But what are the big things today, right? PCI, devs can't have access to production database passwords. It's a big deal, right? It makes so that there's all kinds of like voodoo dancing around the fact that we have to be really careful with database passwords for production stuff that touches credit card data and things. Also, whatever, Sarbanes, Oxley, there's all kinds of healthcare stuff. Everybody has stuff that nobody can touch, right? Except for like the chosen few. And so a lot of this is what this allows you to do, again, when you abstract all of your configuration data out of the app, it allows devs to manage the dev password and it allows whoever's managing production, whether that's ops or somebody else to manage the production password in a way that keeps people from knowing what it is unless you're the person who has to know what it is. Right, but it doesn't stop the rest of you from managing your passwords in other environments. So I mean, that's kind of a really cool thing, especially in this day and age of having to have so many different compliance initiatives that we have to adhere to. The more stuff that we can automate with things like this, the less it's gonna slow us down, right? I mean, the less you have to call somebody at 3 a.m. to get a password out of the password vault, a literal safe, right? So I've done this with like Oracle databases and PCI stuff where you have to get somebody at 3 a.m. to go to the actual safe with somebody else and get passwords out of the safe to reset them when they don't work and stuff.  So being able to actually separate these out is a big win for a lot of people as far as speeding things up and not slowing people down in dev, right? And it's something else. So anyway, what you really get with a lot of this is confidence and the ability to move a lot faster knowing that everything that you're working with is solid beneath you instead of having to wonder all the time if you're gonna have to stop and figure things out and try to figure out if maybe you have a package version mismatch or if IP tables is all of a sudden somehow on on some of these servers because somebody didn't turn them off or somebody didn't disable it during kickstart or whatever, right? I mean, so the big thing you get out of a lot of this is stability, confidence, and the ability to focus on what's fun because really none of this is fun. Confidence? For a few questions. I would love to take questions. Any questions? Hi, I got a question. So agree with taking configuration out of the code, right? You shouldn't have to rebuild your code in order when you change your configuration. But I guess how far are you taking it out, right? So you mentioned, you know, yum install Tomcat's no good because hey, you've no idea where it's coming from or what version you're getting. Be very explicit, but that explicit where you had, you were defining the package version and release number, where is that? Do you support the central storage of that or does that live as a kind of manifest or, you know, recipe with the code base in the branch or the stream that you're working in? Well, I believe that Devs shouldn't have to care about versions of Tomcat. I think that they should be able to just like provision themselves a VM with a sanctioned version, the current sanctioned version of Tomcat in the organization, really. And so I think that's cheap. But you might have to think about it if you happen to make changes in your code as a result of a newer version. Then the branch that you're working in, you're gonna need to make sure that when you deploy, you're deploying with the newer version of Tomcat versus the old one, which is still in production. So to that extent, you do need to be aware. So that's, I guess, my question is, where do you support storing that level of infrastructure configuration that's required for the app that may be specific to application code changes? In your tool, absolutely. So what happens is that when you set these attributes in all of the major tools that we use, you set a basic attribute, there are ways to override those at application-specific levels. So you can actually have your, whatever, your organizationally sanctioned version and you may have a later version or an earlier version even, for something that is specific to a specific app. And you can override that at different levels in the tools. And you can make that at an application level, you can do it at an environment level. So really, what we're talking about here now is that we have OS level configs and package level configs that you want to deploy to your servers. And then you have, say, like an application config that you would also deploy as well. So that would maybe have the information about that application's integrations and that application's database information. And the same thing, like you can override a Tomcat version in those areas as well. So you do really wanna separate operating system level, infrastructure level configs from applications because most of the time your OS stuff and your infrastructure is going to span dev teams and organizations. But you will still want to be able to have app teams have specific information, configuration data for their servers and their information, for sure. Thanks. I think we have time for one more. Hi, so our problem of our dev tier is not that, it's not in Puppet. It's in Puppet, it's all configured, it's all nice. But it's a dev tier, so dev have access to it and they're doing experiments on it. So it's completely unstable, not because it's not in configuration, but because engineers are constantly thinking of it for their project and breaking other people's projects. How do you deal with that? I have done this, right? So the first year that I worked with Chef, I was constantly getting a, Sasha, I tried to create a VM, but it doesn't work. And I would be like, oh yeah, because I was updating the network cookbook and obviously it worked for me, but I missed a config something somewhere, right? I'm constantly breaking that. So it's important to remember that when we're doing the stuff for sure, that again, dev is production for somebody. And so what you need to do, and I don't actually know how Puppet is dealing with this specifically, but Chef is versioning aggressively. So we version our cookbooks and everything. So what we do is we pin step to versions and we make sure that we pin environments to versions. So environments and Chef have cookbooks listed and basically, and then we pin versions of cookbooks to those environments. And so what that allows us to do is mindfully promote, especially in production, which is good, but it also allows us to play around in dev, but it also allows you to pin your own little bit of dev to a known working configuration set and allow whoever's doing the infrastructure experimenting to not break you on a daily basis. And then when they're ready, they can say, look, I have a stable version of this. You can use it or you can not, it's up to you. And honestly, I don't know how Puppet is doing that, but I would be surprised if it were not possible.