दीलिंग मुल्टिपल करन्सीज आन रिस्क्स के में पता आजी कुन्से रिस्स से मैंनों को कैसे मैंनिज करनें दिस विल हैल्प मैं यु भी स्पार्ट रेट कनवर्जन तु मीट कैश शोडजिज आन ताईप लाईं अगर में को ताईम लाईं में इशुज भी आते हैं तो हमें कब क्या करनें ताईट शुड़ भी वैल डोक्युमेंटेड एन आर सिस्टम्स आन प्रुसीज़र्स प्रुवाईट बिलिंग आन इनवोएसिंग चोएसीज़ अएड्यली तो ये भी एक अप्ष्यन है के हम जिस्टा चाइना कर रहा है, वो चाइना कैता आब, what they are doing is they are saying that you have to trade with us in our currency. अगर देखे ये तो अच्छी वाड आगर उनकी सारी ट्रेड इस्ड़ा हो जै, historically they have been doing it with dollars, but now they are shifting towards this part. तो अगर दो if you have it in your domestic currency, nothing like it, because आप फिर सेवगाड हो जाते है, currency risk factor say. अगर दो if you have it in your domestic currency, nothing like it, because आप फिर सेवगाड हो जाते है, currency risk factor say. और अभी भी उनका एक वलुम ही फ्रक्षन इस पे मुव कर सके है, they have not completely shifted. अगर देखे लोकल आपके जो कुस्टमर सेवगाड है, या जो आपके सपलार सेवगाड है, उनके साथ क्या टर्म सेवगाड है, so you should have options for what are the currencies that are being used. अगर दो if you have it in your domestic currency, that has to be taken care and documented in the strategy. अदर अफर different payment methods, similarly present your customers with most convenient way to pay based on where they live. क्या बाभ जब international trade, हम लोग देखे हम ने मक्सध हमारा, currency risk को कम करना है, ये नहीं करना गया हम ने इतना सेवगाड करने के वो transaction इना हो, you should facilitate customers to maximum of their self, के वो जब जब जब नहीं करना चाए, they can facilitate and use their own currency tools to pay. तो वो पेमेंड मैठर्ट उनका जो अपना प्रफ़ मैठर्ट है, लाएग मैठर्ट है, वो उसके तुब पेमेंड करने ने, so the payment method of their preferred method is, तो सब से ज़ादा युज मैठर्ट है, for all imports and exports, that is through bank wire transfer. दे प्रट्टरबड मैठर्ट पेमेंड, एक मैठर्ट ग़ादा से देखा, the process time is also the process of documentation, दो वो अपना आपने नहींग, ग़ादा से लिए प्रफ़ मैठर्ट है, लोगा तीट आपने भी ठादे है, चोद तर मैठर्ट होते है, you can pay through credit cards or through some other payfalls and other digital tools, digital tools, but they are normally used for small purchasing as you purchase online from amazon, you buy books or you have short orders in that case you may go for other digital payments, but otherwise why transfer is the more used method ensuring regulatory requirements are met this is a critical point that when you go to international trades for funds then these are highly monitored, these are highly regulated so make sure you have to comply with the rules and procedure of your own country as well as the country you are dealing with so it is possible that in some country, specifically in a currency, the remittance is not allowed so you have to take care of how the transaction structure will be there so all rules and regulations which are applicable both in your country and in the other parts of the countries different countries in which you are trading so we have to abide with rules with all which are applicable on us manage cash flow efficiently this is a very important point because we don't want excessively holding money just to safeguard ourselves supposedly if I want to do a foreign currency payment then I have to take an ample amount of excess amount of currency if I have to do a payment then I should have it this is not efficient efficient is you need to have required levels and you need to take care with rates and stuff so that you can manage it well imagine a situation where your fund transfer takes several days and you are unable to trace it until it treats its final destination because transit time is also there and its channels are also there plus if there are errors then that can be tricky so the documentation of your process should be very efficient because of inefficiencies it is not attached so you are making it applicable on you on your own such uncertainty is not good for any business because you don't know about transit money if you are travelling where is it falling so you should be well familiar with the process and you need to manage the flow well because most of the time organizations are using borrowed money so if you don't receive or timely payment then you may be paying extra in trust you may be losing some discounts so all these things are impacting you ensure that you have a proper invicing and payment system that guarantees that inclusion of correct fees and identifies in your transaction it should be also locked for x-ray and tracked fund transfer so the rates of foreign currency you need to build in your effective fund management and you need to think about the timing what method, what channel should be used and what circumstances implement centralized multi foreign currency trading management this is very important for example you have imported from a European country that means you have to pay 10,000 euros you are sitting in Pakistan and you have to pay 10,000 euros so this means you have a foreign currency risk similarly you have done an order of selling to another European country from where you are going to get 8,000 dollars and luckily time frame is similar that too you have to pay in 3 months and you have to pay 8,000 in 3 months now selling department is different purchasing department is different so if we say that if it is a lack of the coordination what we will do is those who want to pay they will hedge 10,000 and those who want to receive they will hedge 8,000 which will be stupid we have to pay 10,000 we have to pay 8,000 our open risk is 2,000 so we should only hedge 2,000 so you can see that payment is expected 10,000 and the receipt expected is 8,000 in this particular currency so our net exploit is 2,000 euros so we just have to hedge it so these reduce mistakes such as funds being dominated in wrong currency we are hedging extra 10,000 we are hedging extra 8,000 which is not required so this centralized work can be done in centralized transaction in branches in my Lahore branch 10,000 is receivable and in my Faisalabad branch 15,000 is payable so I should have a bigger picture so I should know how much my net receipt is I should hedge it then our last strategy is adopt an automated payment solution we talked about this briefly that we should move towards sophisticated and dynamic software so this structure of payments should be well structured your team might find it difficult to handle large volumes of currencies complicated payment processes and tight schedules so what we should have is we should have an automated solution which is timing wise according to the best rate when we are getting the best possible discounts when we are getting the best possible rates and the point of view of our relationship the best point of view those systems should advise and tell us when to make payment so our memory should not be linked it should be structured in such a way and the tools should help you do that an automated mass payment solution can help manage all this and can elevate your financial operation so your overall structure can be improved and you will have timely reflection partnering with solution that can automate those back of task, eliminate risk and streamline the entire global payment process so your last part is that if you have multiple payments and long structure all these things are built in the software and tools will help you pay manage risk and structure it so that it can safeguard you thank you


Right, okay there you go. So yes, so we, at our book club we read Team Mastery from Jeff Watts and we've invited Jeff along today to discuss with us the book. Hi Jeff, how are you doing?  Oh there we go, there we go. It wouldn't be an online meeting if we didn't do that right? No, exactly. Yeah I'm good, how are you? Yep, we're all good, a little bit cold today. I got my fire on so it's nice and warm in here now. So yeah, so thanks for coming along and taking time out to discuss the book with us. I think as we said in our sort of emails to set up this, really the idea of today was, you know, the people who read the book and also we've sort of viewed one of your YouTube videos which actually is about Team Mastery in the remote sense so quite a few people have watched that as well. So it was, it was Ez that bought the club, bought the club your book so Ez maybe if you'd like to start off. Yeah Jeff, I won't ask for any royalties but yeah I was the one who brought this on up. I actually started reading this book before I joined SMART so I've been here for three months and I commented on Nickton actually at the speedy delivery so yeah good service man and it really resonated with me, a lot of it did resonate with me and I read your Scrum Master book as well in the day and what I really liked about it was the fact that, you know, you state what you think good looks like and that's okay, good is good but then you also make the point that actually there can be a step higher than that, you can go great as well but that's fine if you just stay at good and I like the fact you translated that idea into Teams and a lot of the themes in this book actually resonated with me which is why I brought it forward to the other guys and said maybe we should have a look through this because you know we've got a lot of work to do at SMART, a lot of good stuff's already happening but we can definitely use the themes in this book to help improve our teams. Cool, was there anything in particular that made you, made you go for the book in the first place? Well I quite like the idea, I know you weren't going for this but I quite like the idea of having a one place where I can look at some good examples of what good looks like and I know the messaging in this book is not, hey you know if you follow all of this stuff in squad you're going to have the perfect team and if none of it's, if for example you don't have a bit of it then your team's not going to be great, that's not what you were going for but I quite like the idea of having a book that I can go back to time and time again and look at sort of areas where I can help my teams. Yeah and you do have a lot of good stuff in there. Yeah and the idea being that you can, if you're struggling to know where to start then it could give you a few prompts and you know people read through it and they can't help but look through their own lands right and they'll let you say it resonated with you I'm sure every other person that read it a different part would resonate with them because their team's in a different place and their challenges are slightly different and it's kind of almost my philosophy of writing a book is write a bad book quickly and then find out where it's bad rather than try and write the perfect book so it's rated a number of different times but it's the same with a model I think is write something, bad idea prioritize your backlog badly whatever it is and then get feedback on where it is bad rather than try and get it perfect straight away so you've got something that you can go to but equally I really wanted to make the point that it's not the way of doing it it's not an exhaustive list or a complete list.  Yeah that's true but I wouldn't have had as much cringing going on either when I look back do we want to go into questions then David? Well just really just sort of like an actual flow sort of discussion really I mean what I'd like to know Jeff is like how did you come to the you know the way that you've you know the way that you wrote the book and the way of using scenarios like you did quite a lot. Why did you sort of decide on on on that path? It's mainly it's my own biases really so when I when I learn I over the years I found that like I really I tend to resonate with people who can tell stories because I put myself in those positions a little bit like Ezra was saying there. It kind of normalizes it for people realizing that they're not the only ones experiencing that kind of thing and I find it a little bit easier to get messages across in that way I suppose than just writing very theoretically. I was never really one for the theoretical textbooks at school I was more for like case studies and things so that's partly where it came from I think and the good to great thing was I'm not really a big fan of anti-patterns I mean they're useful because I'd rather learn from somebody else's mistakes than mine sure but telling you all the ways that things go wrong doesn't necessarily lead to how you can do things right it might do if you eliminated all the bad stuff and avoided all the traps maybe but it's not necessarily elimination of the negative that leads to positive so now I find it's easier to pick holes in things than it is to give you know good things to focus on and I think we all need a lot more good stuff in general. And it also makes people be a bit more receptive I often think if you're telling them a good story as opposed to saying this is why you shouldn't do something or this not anti-pattern in terms of it being a little bit softer for them and a bit more a little less resistant. Absolutely we incorporated that we're writing a load of training material for the moment and we realized that we'd fall into this hole of just like bunging down loads of anti-patterns rather than actually speaking to people from a more positive place and saying this is something you could do rather than don't do this so we really we really took that on board actually that was a really helpful piece for us. We're now just translating all of our anti-patterns into what does that actually mean you do do. Don't throw them away because they're fun right they're good stories they're fun and it's nice have a few things and think oh yeah I won't do that I won't do that I won't do that but one of the things that I can't even remember when it was but it's such a long time ago but it's sort of stuck in my mind I don't even know whether it's true I'm not a neuroscientist but I was told that the brain doesn't think in terms of modality so if you if you think into yourself don't do this don't do this you know don't mess up don't mess up all your brain is thinking is mess up and that's what I can't verify it but it's stuck in my head for so long I thought well rather than if I'm going to repeat something let's have something a little bit more positive. So Nicole yeah sorry before we maybe get into the detail of the book to stick on what Es was saying about your experience of writing it and then your experience of sharing it with people I think we were talking about what are the bits that have most confounded people um where they just you know they've got stuck on it and they ask you questions about it over and over again if if any and then contrast like vice versa what are the bits that people just can't get enough of have you seen patterns like that in speaking to people? Yeah I've probably got more if I thought about it more but the one thing that really pops into my head is people reading it and thinking okay it makes sense Jeff but how do I get my team to do it? Yeah people in the team reading it think yeah it makes sense Jeff but how do I get my managers to let us do it okay so it's all it's the sort of the external focus you know this was to come back to Es's point this yeah there are books written about teamwork but there aren't necessarily that many written for the teams themselves it's usually for a leader to do to a team or to create a team and the idea here is yeah you could you could pick this up as a scrum master, a product owner, a coach, a manager whatever but equally you could pick it up as a member of a team that didn't have any kind of leadership and wanted to create that sense of self-leadership and you could you could do that that was kind of one of the things that I want to get out of this but it's easy yeah it's a common human trait of yeah this this would all work if so and so did something or so and so changed and you know another of my sort of operating principles is that the only person you have a hope of changing is you so focus on yourself rather than focusing on trying to get anybody else to do anything differently and try and work out well why wouldn't they naturally do something different better more helpful and how can I affect that rather than I need to make this person do something different that's that's the number one common thread I would say it's not necessarily around one of the topics although bravery is usually a difficult thing for people in organizations because they have to sort of it's the phrase learned helplessness again I had no idea whether this is true but it's one of those scientific studies that I was told once and it stuck in my head around the tip where the term learned helplessness came from it's something to do with sticking a bunch of monkeys in a cage and giving them some bananas and when they went to pick up the bananas they shot them with a water cannon and I presumably this was done a long time ago when animal rights wasn't a thing right and and so every time they put a banana and they went to get it they hit them with the water cannons so eventually they stopped ignoring they started ignoring the bananas and over time they took monkeys out of the cage and put new ones in so they're replacing them and the new ones obviously didn't know about the water cannon so when they went over to pick up the bananas the rest of the monkeys hit them so don't do that and however they speak and then but it kept going until a point where no monkey had ever seen the water cannon but none of them would pick up a banana and it's that that sense of learned helplessness they just didn't know why they weren't doing it they just weren't doing it and and in our organizations I see a lot of that there's no point that's making this change because change doesn't happen well can you give me any example of well no that's just the way things are around here I can't give you an example of it but I've been told so often that things don't change that there's no point hmm that's a really interesting one because I was going to ask you about audacity that was kind of the one that sticks out to me it's a difficult one audacity to me I know yeah I've since I've read your book I'm always blabbering on about audacity which I would imagine would be quite annoying but yeah I know you've mentioned before that you'd maybe phrasing it as bravery might have been a bit more useful because uh but it wouldn't have fitted into the squad acronym so maybe not but um but actually um well especially during these remote times right or trying to I guess we can't control how people behave right but we can guide them and we can we can at least show them what good looks like but I'm finding audacity really difficult like getting people to challenge the status quo uh based on what you know what you were just saying um to challenge one another even in a in a team full of people that barely know each other and they're all remote uh it's it's a real challenge and um yeah I just wanted to know if you'd had any feedback on audacity um in general from other people yeah the I suppose that there are two two techniques that people have picked out this book more than anything else one of them might spelled out in a lot more detail than the other but one of them was the user manual which which people come out with quite a lot and I only really sort of paid not lip service to but I didn't really go into detail of it in the book uh but the one where I did was the where people had really latched on to his fear setting and that idea of just challenging your limiting beliefs challenging your assumptions about well what what do I think's going to happen here if I'm from being brutal with myself here what am I actually worried about and let's put a plan together to reduce those worries to to recover the situation if it does happen and actually to be a little bit more um have a different perspective on it as well so going through that process for a lot of people whether it's you know speaking up in a in a retrospective or challenging management or you know whatever just starting a new personal habit whatever it is that having that process for okay I'm going to assess my fears here rather than just have this undiagnosed unlabeled unanalyzed just massive which is has been really helpful for people I think yeah it's definitely a part of the book that really struck with me the most because it's that kind of element uh the people I mean loads of people talk about self-improvement and delivery and unity but audacity is often one that sort of overlooked in a way as a as a really you know big part of what makes uh you know a strong team so uh yeah it resonated with me I really I really enjoyed that part of the book and it does take a lot of courage to to do something different to go against you know however many years of culture and you know what you're assumed to be the case um and to step up and take more responsibility so there's got the first one I mean it's got to be benefit to you to do that there's got to be an incentive a personal incentive to do that uh I mean the good news is that most people if if everything was equal and they knew nothing bad would really happen they would rather have greater control over themselves and be controlled um so the philosophical fundamental is is compelling but there are too many butts associated with it um for a lot of people so yeah that being aware that that's that's there and the other thing for me around audacity and courage and bravery whatever you want to call it is a lot of people unconsciously assume it to be binary either I am brave or I am not brave you know I am audacious or I am not when actually there's different degrees of it and you don't have to start big you can start small and it's a habit it's it's all these analogies if it's a muscle the more you exercise it but also the more you see other people do something and nothing really bad happens if you're looking for it and you're actually you know it's not filtering it out with your biases you think okay yeah all right so that happened and all right well didn't end maybe I could do something you know what's the smallest step you could take without those fears being paralyzing and then reflecting on it and thinking okay maybe I could do a little bit more next time so working out where you are on that almost like continuum and taking as as big a step as you're comfortable taking cool I love book club and we were talking to Jeffrey Squirrel sorry Jeffrey Frederick and Douglas Squirrel I just amalgamated their names there and they were talking about sort of related it's not the same technique by any means but they were talking about like mining opportunities for conflict so they're kind of making the sense of conflict and spinning it on its head and saying actually we should be inviting it and we should be building a culture where conflict in this particular way is seen as a positive because we are wanting the kind of dynamic that comes from really digging into our differences of opinion because that's where that spark of creativity is and that's where we can get to greatness rather than avoiding it which I thought was really interesting and also like you say psychologically maybe quite terrifying for people at first but it's how do we build towards that place so that we have a culture where people are more likely to do that yeah and I yeah I think you're right it's for it's gradual right it's going to test the boundaries be a little brave do a little more encourage other people but I think that's such an interesting and tricky mindset actually to mine to purposefully mine for conflict and to kind of have that almost as a working agreement yeah that yeah yeah we'll get one of the most one of the most powerful exercises or techniques I've seen in this area but it is a risky one is is ritual descent I'm not sure whether you're familiar with with ritual descent so it's it's a cognitive edge technique so Dave Snowden's company and and the idea and you can use it for all sorts of different things but basically for idea critique so if you look into generate multiple ideas and get some really quick feedback on them if you've got a number of people and ideally an idea to get a number of people to go break them up into groups and everybody come up with an idea in their teams once you've got a draft of that idea whatever it be for a user story or a solution or a impediment removal or whatever one person from that each table goes to another table and presents their idea to that group and the rest of the group who there who are there listening they don't do anything they don't talk they don't interrupt they just listen when the presentation has been finished and people have heard it the the idea presenter turns their back or if you're in a virtual world turns their camera off now you know you're remembering it and then the job of those people who've listened is to rip the idea to shreds is to say nothing positive at all about it okay they've got to come up with why it's the worst idea ever and why it will absolutely fail and backfire and the person whose idea it has just has to stand there and listen they can't rebut they can't argue they can't defend they just have to listen and then they can take that feedback back to their group and decide which of that feedback if any they want to incorporate into the next iteration of their idea and then they go to a different group and do the same process and the reason I bring that up is because if you know it's a game if you know it's I can't say anything positive then it's not personal or it's less likely to be personal we may need and I say it's risky because you may need some ground rules around respect or a little bit of trust in the room but yeah actually ripping it apart is actually something I encourage people to do with their own ideas you know adopt the the psychopath approach put on the black hat of doom how could this all go wrong um and but if you're doing that with other people then that's it's it builds trust so that once you actually are out of that mindset of I'm going to rip it apart you can be a lot more in the middle and actually real criticism if you like is a lot more tolerable right I actually was thinking about this the other day because I was thinking about how do we proactively build psychological safety um within teams and I think yeah one of the kind of key things was frame it as an experiment whether it's this ritual descent or whether it's something else if it's within the bounds of an experiment then it feels much safer for people to to push and to to give constructive criticism because and to fail because we know that experiments are based on hypotheses and that they fail even in science which is meant to be the subjective thing you know um so that I think that was my favorite my favorite answer when I was thinking about this recently in terms of how do we build psychological safety which is such a tough one in itself like we asked people that in interviews which is what's the best answer you've got so far well that was one of them I really liked that that was actually my answer I think she also said encouraging a culture of of feedback in general continue with feedback uh and making that the norm um which obviously is a great one for building trust um yeah cool you're on mute David yeah actually on feedback I would say um I've been uh re-looking at your book over this weekend and I quite like the uh the the what you pulled out about feedback about actually feedback about being you know when feedback is uh not requested but but but but is given and that those sort of things or those mistakes around around feedback the kind of the feedback sandwich yeah and things like that so yeah I thought that was uh yeah it's it's it's that idea of like when we give feedback it has to be people requesting it and then and they're open to it mm-hmm yeah it's um that was what that was one of the things I was taught on my probably my first week in the job um my first job about the feedback sandwich as it was that was that was a good way of giving feedback people people told me yeah yeah um feel right at the time so in terms of like the the the like the milestone cars that you have at the back of the book it's so it's for any the you know this there's got there's quite a lot of stuff there and is there any that you kind of would would would pull out as these are the the sort of precursors to to making the sort of stepping stone from from good to great or did they not work like that in your in your mind I mean I suppose they I think they probably could um but I don't think they would work like that if I picked them if that makes sense um so for me I like teams to start from where they are um so a really nice exercise is to go through those milestone cards and pick out a couple that they're really comfortable with already and say do you know what we're not starting from scratch I reckon we've already got some of these in place um and whether they they start on building those because they're already confident in that area um or using that what's the next one that we think could uh we could build on and and almost you know move to next I think that is a really nice way of starting we'll often assume that and and I mentioned this um in the book itself actually when she start as a team there are so many ways that you can get better it can almost be a bit overwhelming thing well where do we start and actually jeez we've got so far away from where we want to be but then choose and to view that as well if you've got all these opportunities to improve then brilliant just pick any of them and and that get into a habit of getting better but the cards are like to the ones that I've seen working with teams where you know they've got these ones yeah we like these we're doing okay with these we'll check in now again to make sure we haven't slapped on some of them but where which one would we like to focus on next which one do we think would be really useful to us where we are as a team and then consciously proactively working towards it it's kind of um well partly inspired like I said in the book by new baby but also from um I've just lost it I lose my thread thread down again but yeah this idea of working towards something can be a little bit intimidating if you think it's you know it's normal so the cards we've got um for a baby for example you know they're not time bound on when they should be crawling or anything like because every team develops at a different rate just like a baby develops at a different rate so it's not about putting a judgment on the teams and you know you should be here um by this time you know you are where you are and you're developing when you're developing and you're meeting different challenges well that's that's interesting what you say there in terms of like engage it how do you find that you you sort of engage teams to start making this this journey from from from good to great you know because there is a lot there and what what what's sort of the the natural approach because that that's sort of been my sort of question when I've been reading the book there's so much stuff here how do how do and also about the change about the amount of change you know that am I trying to push change or in trying to do this how how do I do this where I'm not pushing change to people part part of the idea behind these milestones is is it's kind of to try and make the journey to greatness a little bit more enjoyable rather than just enjoying it when you get there because it is going to be a long journey and you know rewarding yourself along the way and having a bit of fun and you know satisfying things as we're going I think it's quite important for a motivation point of view but also you're going to spend most of your time getting there so why not enjoy the process of getting there rather than just being there at the end um so where do I start well I start really with trying to get some permission and trying to tap into their intrinsic motivation you know so it would be a I know some of the teams that I've seen but every team's different so what what are some of the great teams that you've been part of tell me what what was great about them and try and create get that team to create their own sense of greatness that they can you know come to agreement on because there will be edge cases that aren't relevant for some members of the team maybe someone's talking about a sports team and someone's talking about a software team someone's talking about being part of the um working in a hotel or something and it's there are there are commonalities and there are overlaps but there are things that aren't that they're domain specific so try and find the core threads that they can agree on and then then ask them what would what would it be to them to get to that state here in this team would how beneficial would that be for you what would it give you um almost you know effectively doing a bit of a cost benefit analysis is that it's the process of change worth it is it something you want to go towards and if the answer is yes because you don't they don't have to as you can stay as a good team that's fine it's probably better than most of the teams you go with but if you had the opportunity would you want that okay and if the answer is yes then would you like my help and then we agree on how what that helps going to be it's not going to be telling them it's not going to be holding you know i'm not holding them to account they're going to be holding themselves to account all that kind of stuff and so also where do you want to start next this is a list that may be something jumps out but equally you might already have something that's forefront of your mind this would be a great place to start so i i try and meet the team where they are i want to get their permission for themselves but also my permission um and then give them something that they can work with if they want to but in the knowledge that it's their agenda not mine so i have to be open to the fact that they'll say no so that's the tricky thing when you're a coach right um if people are struggling with items that you feel from your experience could be super helpful like in the i was just looking at the cards again and you've got reference on there to things the process and the system and some of these words can feel quite dry and i know you're trying to alleviate that through the cards right but um that's where i've come unstuck sometimes and is that there's uh this um resistance or this difficulty getting rid of that hurdle of how to think about the system and how to think about the process particularly in terms of how to measure it and how to really understand some of those metrics that can look really scary and can seem really dry um and that's where often i found myself having to do lots and lots of work and then you know sometimes you may not succeed um so i yeah i guess in a in a roundabout way i'm saying how do you deal with some of that where it can seem it can seem a bit dry and overwhelming when you're talking about system process and metrics so by the sense of it the fact that you said sometimes it doesn't work would imply to me that sometimes it does sure sure so in your experience when has it worked and what have you done to help it what you're doing there jeff i see you you're coaching me now yeah i i think you're right it's about putting the empowerment back in their hands so i've asked them how they would want to how can they tell if we're going to run this experiment so i've framed it as an experiment how will they know that it's been successful and how will they know that potentially it hasn't and they want to drop it and try something else um so that's helped um but i still have found them kind of looking at some of those metrics you know with our scatter plot diagram and things that's still we can have had a really positive start and to yes we do want to improve and we've agreed these are the things that we think are important to track and now i'm looking at the scatter plot diagram and all i've kind of lost the thread of why we were here you know so what is what was the thread sorry yeah you had it at some point when you had it what was it um so i think the general gist here for me is why why are they doing it not not how are they doing it but why are they doing it and what does the system what does process what does metrics mean and do for them so why would they want to improve the system tell me tell me a story of when the system screwed you over all right you did your bit but it didn't work out all right so why is it in your interest to change the system that these kinds of questions tell me a story of how the system can work for you rather than against you okay what would you need to do to get closer to that and who do you need to bring into the conversation to make it more likely to happen but bringing it back to the personal bringing it back to what's in it for them why should they you know um and again it's kind of story based because they can tell me a story of when the system screwed them over yeah yeah yeah and then by doing that we're tapping into their emotions we're tapping into their drivers we're tapping into things like frustration and then we can start rewriting that story so how would that have felt or what how would that look like if the system had been working for you where's the gap so what's how much value is there to you in changing this actually David was has been talking about this quite a lot recently say sometimes we get lost in our own sort of coachy world where we're so used to some of these concepts and some of the jargon and we forget to kind of put ourselves in the shoes of that individual within the team in so many different ways in terms of their like personality type and their role so I know David had a lot of success kind of putting himself in the developer's shoes in that way recently and it can be easy to forget to do even though it seems obvious in a way it's almost like taking it down to forgetting who we are and thinking more about what the people want that you're you're working with I think Jeff in your you know the video that we made reference to before the remote one I think you're you're when you were talking I'm not a big cricket fan but when you were talking about a cricket coaching and you were talking about that bit and I think it's towards the end of the video I found that really powerful actually you know it really actually really put a lot what you talked about in the book in context for me which is like like what do you want to get good at people know like you say that that question that it's a really great question it's like what do you see as being great let's let's let's start from there and maybe not bringing you your own preconceptions which I've now done in the past it's like well I see a great team is doing x x y and z but but and people within the team may have a completely different different view of that and they they come from a different perspective they're they're not thinking it as an agile coach they're thinking it as a as a developer how they want to take forward their career or how what stresses they have on them yeah I thought I thought that was really good that what you thought it's about about cricket coaching is it in many ways it's a scary concept if you talk to a parent about it they said but what how does my tenure know what to get better you're the coach you're the expert you've played the game you know what so to teach them teach them so I could but to be taught you need to be receptive and what's better way of being receptive than to ask to be taught something rather than to be told what they're going to be taught now the the rhetorical question that I would generally ask back is can you imagine anything that your your your child would ask for to get better at a cricket training session that would be bad for their development no so it doesn't matter what they want to get better at they're going to get better and and the multiplication factor there of not only the fact that they're more receptive to the thing they want to get better at but they're now also think proactively in the habit of thinking what do I want to get better at and how can I get someone to help me get better at it it's much more powerful than turning up thinking what's someone going to tell me to do this week Mimi you've got to give your examples you got to hi hi no I'm nodding like profusely like I'm nodding like crazy because I totally agree I used to be a nanny so I totally understand the whole idea of um encouraging children to take command of their own learning and allowing them to kind of learn on their own or be in charge of that because they the bite you don't need you don't need to figure out how to get buying they're already bought in if they're trying to figure it out if they ask questions about certain things or if you can give them tools to kind of figure things out on their own then when they come to you then you know okay they definitely need my help but the more you kind of give them tools to figure things out on their own the more that they eventually learn I have a brother and I've I've given him free reign of YouTube and he's messed up my settings because now all I see is everything to do with space craft and space and this rocket and there's this game that he he doesn't play it he just watches how people play it but it's building spaceships he tells me about the different parts I have no idea what he's talking about but he's taking his own learning to the a completely different level he's only nine but some of the things he can talk about he can probably have a conversation with some kind of rocket science you can tell I don't know anything because the terminology I'm using but he can have a conversation it's so I'm just I was just resonating with what you were saying about you know giving a child letting allowing a child to ask for what it is that they want to learn and the fact that they will you know the whole idea of buying and things like that but exactly push versus pull um but then it's I think again like I sometimes forget that adults I don't know how to explain it's like sometimes you wonder if adults will figure it out on their own and I'm just like well they're adults so of course they'll figure it out on their own but it's like it's finding that balance of giving them tools that they can figure the things out and not allow I don't know how to explain do you get what I'm trying to say I think so I mean we we it's amazing how ingrained this this culture of being waiting to be told what needs to be done is yeah seeing that between the difference between a parent and a child is is quite illuminating because children haven't got that drilled into them yet and so the natural curiosity and the natural they just want to be independent yeah it's sort of part of our natural development as a human being we need to develop independence and so we're facilitating it but we can have this learned dependence as well as learned helplessness within the organization and so what helps I think is is it's nothing that's particularly groundbreaking here but as well as having the tools to be able to do something having a purpose you know something's a reason to self-organize a reason to take autonomy and a valid assessment of the safety to do so because we do assume that something if something goes wrong there's going to be and I was guilty of this when I first started doing scrum I was with the opinion that we were going to be back this is like 20 years ago that we were just waiting to the end of the sprint and vote off the weakest link genuinely thought that you know we were going to hold up cards yes Steph the person we voted off this week um I was really worried that was going to happen right because there was such transparency so that sense of that's just the culture that was sort of bred into us if you like so yeah it takes quite a while but for me what I'm seeing is certainly over the last 20 years I'm seeing a shift leadership level wise in terms of appreciation that culturally the the domain that we're in as an organization we can't we can't rely on expert leadership we need collaborative leadership we need engaged leadership um so it's been slow a lot slower than I would have liked but changing human habits is slow and when they're reinforced by money and culture and family and all these different things it's very difficult to do any quicker than than you can so you just got to be patient and keep working on the kids the kids of the future so don't don't get them to a point where we have to get them to unlearn what they've been taught because unlearning stuff as human beings is quite painful I'm gonna ask actually about the whole idea of culture um we're quite lucky that we have a very very open culture in in this organization but there are some places where you go and it comes from the top that kind of closeness or the command and control style of of working so sometimes you find that even if a team a specific team is doing great things they're deemed to be lazy or not doing enough work or having too much fun or something like that what advice would you give to those teams to kind of encourage them to keep doing what they're doing and hopefully kind of let their what they're doing shine shine as a beacon for others to follow rather than maybe thinking it's actually easy to just conform to the current culture and stop what they're doing yeah all right before I answer your actual question I've just got a little bit of advice for you if you want it yes please you said you were lucky to have a culture of openness it's not luck that's been created definitely it's been nurtured it's been maintained and using the word lucky although it was in no way intended this way is a little bit disrespectful to the people that have put the effort in to create that openness so it's kind of leading me on to my answer to your actual question which is that those teams that are doing that stuff the the good stuff but are being met with the responses that aren't necessarily reinforcing that commitment is to have a little bit of empathy because it's very easy to say you know we're doing all this stuff and they're not doing their bit and they're keep asking us to do it it's natural and it's actually on the face of it exactly what's happening however those people that are responding that way still I would be willing to bet want that organization to be successful I'm pretty sure those people don't want to demotivate or disempower or sabotage those teams but there's something going on that's making the behavior that they're demonstrating a no brainer for them so they are facing some difficult challenges whatever they are we don't know what they are because we're focusing on our own stuff but they've got other challenges other conflicts other demands pressures and if the team can empathize with that and say all right I you must be in a really difficult position here how can what we're doing help you so you use the phrase you know be it shine a light for others to follow but I think there's something else there around shining a light for others to share that light if you like because you know one of my favorite quotes is it's amazing what you can accomplish if you don't care who gets the credit and if you can let other people take the glory take the credit then you're more likely to get the result you want and if it means a senior leader getting the credit for doing something agile or getting the credit for you know your your team's success and change so be it it's a bigger game of play here thanks for that I appreciate it Aurelia did you want to say something you used the little symbol Hi um yes uh I I had this it was a bit of an impulsive question because you you know when you asked the question I called Jeff sort of turned around and started asking all these coaching questions but like but really good ones really good ones and uh not like how to say I kind of it's a bit of a big question like on learning to ask the right questions where the person doesn't you know feel interrogated or you don't yeah essentially how did you learn I assume it's some of the experience but maybe yeah just how to learn to ask to ask the right questions um and and on the spot uh well through doing right not just doing right mindful and reflection okay so I have a I have a hypothesis I have a I have a feeling I have a intuition call it what you like that this question would be useful many different ways that I could phrase it I've got to try something and then not only do I ask the question but I'm also asking for feedback on how useful that question was how did it make you feel did it make you feel defensive did it make you feel curious how did it make you feel so I can do that actually as I'm going or not just or and I could do that as an actual practice so one of the things that that I do myself and do with other coaches is is hot seat questioning um and it's it's a really simple technique it feels a bit weird but it is basically to practice one particular skill which is asking good questions what was the technique sorry I didn't catch that hot seat questioning so practically how it works is in a small group say six people one person just talks for about 60 seconds on something that you know they're struggling with they want some progress and make some progress on and the other five people are listening and then when that person's finished they get to ask one question each just one question and they ask it knowing that they won't get an answer to it so I would ask you my question and it wouldn't be I don't want you to answer my question I want you to tell me how useful the question was to you okay now knowing that I'm not going to get an answer frees me up a little bit because I'm not indulging my curiosity anymore it has to be useful to you not me and that's the first step is try and take your curiosity out of the equation um and and it's not that any there it's not that some questions are amazing and some questions aren't it's um it's that some are useful in certain circumstances with certain people and it's about building up that that almost database if you like because that's all an intuition is it's just a set of data points collected over a number of years that is and you can't really draw that the a precise pattern from but you you kind of know through experience so that's just practice and get feedback is the short answer thanks but uh yeah the specific technique and yeah thanks it was really helpful okay um we're we're out of time unfortunately um but Jeff thank you so much I really appreciate you taking time to come along and talk to us today it's been for my perspective I think for everybody else it's been uh really useful and uh really enlightening thank you ever so much thank you thanks so much people thank you so much


Hello and welcome to this episode of the Road to Intelligent Data Apps. I'm Shelly Kramer, Managing Director and Principal Analyst here at theCUBE Research and today I'm joined by my colleague and fellow analyst George Gilbert and we are joined by Jason Neppol, the Senior Director of Data Management for Databricks. So this is part of our continuing conversation about what we call the sixth data platform. This is an emerging data platform where leading vendors in this space are Databricks, Snowflake, AWS, Azure, and Google. And what we're seeing in this evolution is that the data platform is actually becoming an application platform, thus the name of our series, the Road to Intelligent Data Apps. So this is happening as all applications are becoming driven by data, analytics, and AI. So for our conversation today we're going to take a closer look at Databricks, both the company and the customer activity and we're thrilled to have Jason join us. Jason, hello and thank you. All right, thank you so much Shelly, it's a pleasure to be here today. Absolutely, we are very much looking forward to this conversation. So before we dive in with Jason, I want to give you some historical context. Snowflake kind of defined the modern data stack with the separation of compute from storage, which essentially allowed the management of tons of tons of infinite amounts of data in a data lake house architecture. The data lake house gave customers a simple solution for managing their end-to-end data and their analytic data estate. Then with partners DBT Labs and 5Tran, they provided an end-to-end solution that also reimagined and pretty much radically simplified ETL to ELT. And this is the next generation of ETL. I could, could I throw a few more acronyms in here please. But this is significant because in ELT transformations took place before the data was loaded. And running transformations before that load phase results in a more complex data replication process. So in contrast, this is a significant switch. In ELT, the process is reversed. And so it's extract, load, transform. And here's where it gets interesting that we see Databricks as redefining Snowflake's core innovation of separating storage from compute. In Databricks version of the data lake house, storage and compute can come from different vendors. This is actually, I think, very attractive to customers. End-to-end data management becomes end-to-end data governance in the Databricks Unity catalog. And we have different compute engines, including Snowflake, which can run on workloads on the data, which can run different workloads on the data. And this gives customers a wider choice of price and performance or capabilities. So there's lots to unpack here. Jason, we have so many questions for you. So let's start with the basics. Why is talk with us about what a data lake house is and how it's different from a more traditional data platform, you know, multi-vendor compute versus single-vendor governance separate from DBMS. Just jump in there. Yeah. Well, so the data lake house, I mean, fundamentally, you can't have a data lake house unless you have your data stored in open data format. And what does that mean? It means, you know, more and more data warehouse is being built in the cloud, not on-prem. So you're going to put your data in the place that's the cheapest place and most durable, which would be cloud storage. So in AWS, that'd be S3, Azure, ADLS, Google, GCS. And this is the cheapest place to put your data. And then to process the data, you're going to want to spin up compute. And then usually, you only need that compute for a finite amount of time. And then once you're done processing it, you can shut down the compute or the virtual machines. And because you separate these things, now you're basically optimizing for cost. So in the cloud, you pay for storage based on how much data you have and how long you want to retain it. And you pay for the compute or the virtual machines based on the size of the machines and how long you want those machines up and running. So contrast that with like the old days with like, let's say you've got a Oracle database on-prem. That Oracle database has a finite amount of compute. And that means you have to basically schedule things. So if you want to run your ETL, you're not going to want to run it at the same time during the day that your business users are running their BI reports. You're going to want to run that ETL at night time, which then precludes you from doing things like real-time data analysis because now you can't really be using that same compute during the day as you're doing those BI queries. And so what's that? You got to wait. You got to wait. And so with the separation of compute and storage, which I don't think it was snowflakes innovation. I think BigQuery might have been the first commercial vendor to do it. But everybody's doing it now. With the separation of compute and storage, now you could have your ETL running on one cluster and then your BI queries being served on a different cluster. And that really opened up a lot of possibilities. Now with the Lakehouse, because all your data is in an open format. And so for Databricks, we invented this open source format called Delta Lake. It's the most popular open source format for Lakehouses. And it basically is supported by almost every open source tool out there and commercial providers. So what you can do is you can write your data to the Delta Lake really with either using Databricks or using regular old Apache Spark. And then you can read that data using other engines. So you can read with the Databricks photon engine, or you could read it with a Presto or Trino engine or even the snowflake engine if you want. So because of that, because what happens is over time, all these different data processing engines go in and out of favor. So 10 or 15 years ago, Hadoop was coming out of the scene. It was replacing Terra data. Then Spark became popular. Then Trino. Snowflake has grown in popularity. And so I can't tell you what the most popular data processing engine is going to be in another 10 years. I mean, I hope it's still going to be photon, but we don't know. And so if you've got a large amount of data, some customers have petabytes or even exabytes of data, you can't just like move that around. You can't just like export it out and then import it into the next data platform. You need to basically keep the data where it is and then bring the right engine from the right tool for whatever you're trying to do with your use case. Makes sense. So what about an intelligent data lake house? Well, so the lake house is something that we kind of pioneered, I think around 2020. We wrote a great blog article. And now recently, like last November, we came out with what we think is the next evolution. I think it's similar to what you guys are calling the six data platform, but we call the data intelligence platform. And the whole idea is that historically, you've had different silos or different tech stacks for your data stack and your AI stack. And now these are kind of coming together with the lake house, because whenever you want to train your models, you want to be able to train them in a distributed fashion over open data sets. And the lake house has facilitated that. But really, what you need is you want to build these intelligent applications on top of it. And to do that, you need some sort of intelligence engine, which is going to be able to take knowledge of all of your metadata about your data, and then combine that in a large language model that you train on top of your own data, and then leverage that to answer questions. So if you think about it, historically, if you were a user on Databricks, you had to be technical in some way. You had to be able to write at least a sequel. Most people write Python, but then there's also like ScholarNor. But if you couldn't leverage any of those languages, you would have a hard time with Databricks. But now, we're enabling the rest of the users of an organization enterprise. So if you can speak English and type English, you can ask a question. And then the data intelligence platform will be able to take that English question, translate it into queries that run against your data using the context of your data, and then give you an answer in a visualized manner. So we're calling these like data rooms or the product name is called Genie. And we're already trialing this with customers where they just ask questions, and then Databricks generates answers for them in real time. And those users don't have to know even how to write sequel. You know, this is just another step in the democratization. Sometimes I get tired of that word, but that's really what it is, right? You know, the democratization of the access to and the ability to use this data that doesn't just rest in the hands of people, you know, of coders and developers. That's true. Yeah. So I think that we want to talk now a little bit about some some items on your list, George. Take it away. Okay. So Jason, let's talk about some discrete workloads and how over time those might transform into the building blocks of these intelligent applications that build on the data and the metadata. But let's go discrete workload by discrete workload, that how you might have different capabilities, performance characteristics, price points to run the data engineering, business intelligence, supervisor or unsupervised like old fashioned or more gen AI. And then and then applications and how Databricks defines those today, help us understand those as, as discrete capabilities, but then how they become essentially workloads within an application. Yeah. So I mean, I think Databricks, you know, we've been, we've been at it for now for, for almost 11 years, I think, but we've, we kind of started as just basically a notebook service where you could have notebooks, you can interactively interact your data. Then we added the ability to run those notebooks on an automated basis. So basically kind of like, you know, job scheduling service. And we've since kind of like expanded into more different product lines. So we've got a separate product line for streaming. We've got a complete orchestration tool, we call Databricks workflows, which you can use to orchestrate anything across data in AI. We also have Databricks SQL, which is our data warehousing engine, so that you can basically any sort of like BIOC style query you can run, you can run directly on your Lakehouse without having to replicate it out to another data warehouse. And so what we can do is because we've got this very flexible engine, photo engine, we can spin up different types of clusters that are optimized for these different types of workloads. And then likewise, we can, we can price these workloads at different rates. One of the, one of the things we learned early on when we were doing Databricks is we were always competing against other open source Spark product products out there that's hosted by different cloud vendors.  So we are able to, you know, one of our first price points was like we offered our automated jobs at half the price point as the interactive notebooks. And then that really allowed us to capture more of the market that otherwise we would have been locked out. So when we look at, you know, every product that we release, we, we actively look at, you know, what are the, what are the price points that customers are willing to pay? Because, you know, if you just have one price point for all of your use cases, that means you're, you're probably overpaying for, for some of those use cases. Sharing though. Yeah. Let me, let me just get you to clarify something on that. So data engineering is typically more throughput oriented, where business intelligence is more latency oriented. Might you be using different engines? Or might you just configure the engine, one engine differently for the different workloads and, and then help us give a sense, help give us a sense for the different, the different price for say data engineering versus the interactive business intelligence. Yeah. So you can, I mean, for, for Databricks, we have one engine, the, the photon engine, which is our native vectorized engine that we built. And we use that for, for all the different use cases, but the type of cluster that we spit up for you, it might be configured a little bit differently. So even in streaming, if you're, if you're basically just doing, like ingestion, where you're basically just reading data from a source and then writing it to another target, and maybe you're doing a little bit of map reservation, you know, you can get away with a cluster that's more CPU focused, but as soon as you start to do, you know, online aggregation within that stream, then you're probably going to want a more memory optimized cluster. So we can kind of like take those types of things to consideration as well as like the amount of memory we allocate for certain things. But you could also use other open source engines as well. So we have some customers that are using flink. So they'll, they'll read and write data to Delta tables using flink, because they've already got a lot of expertise in house and how to use that. And that's perfectly fine. Okay. So, um, and then talk about how machine learning has transformed into gen AI and the difference in that workload, like with machine learning, there was a lot of feature engineering because you've had a sort of handcraft each model. Um, and you had to say what features to train on and then there's, you know, essentially you're creating a mini application for each, for each use case. And how does that change with gen AI? Yeah, it's, uh, it's interesting. So like, you know, gen AI came on the scene a little bit over a year ago when chat GBT came out. And before that, you know, you know, people didn't really talk about LMS too much. It was like, you know, about natural language processing and it just kind of exploded overnight. And I was, um, I was talking to one of our large financial services customers a few weeks ago and they've got a, um, an AI governance team that basically reviews every sort of like model that gets proposed or developed. And I think about 30 to 40% of the models that are getting proposed are now these, these gen AI time models. So, um, I was really surprised by that because they're, you know, they're a very, um, older enterprise, large enterprise, but, you know, overnight, essentially 34% of what they're looking at this gen AI. And I think that's really important because, um, it's just going to get continued to grow. And if you think about it, I mean, all, um, this is kind of like simplification, all a generative AI model is, is a transformation. You're just like, you're getting inputs in, you're transforming it, and you're getting an output. Now it's, I'm oversimplifying it, but that's what it is. And I think, um, this is going to like cause a whole bunch of more, uh, scrutiny around AI governance and regulations. So there's already a whole bunch of different regulations that are being proposed in the US and Europe and the UK. And if you look at those, there's a few similarities. One is that, you know, you're going to have to have data and model security and privacy protections at every stage. You're going to have to have, um, lots of, you have to be able to document a lot of what's going on, um, across these models and then how you're generating them, because you're going to get audited. Um, you're going to get to produce those results and you have to be able to address, um, you know, bias and inaccuracy and you have to be put in guardrails as well. So there is, we, um, there's one example where if you went to, I think it was the, uh, the, the Chevrolet website and you typed into the chat box, what's the best truck out there? They would say the Ford F-150, right? Like, and you know, you shouldn't allow that. There should be some guardrail. You put in things like that. And all that type of, um, you know, governance is, um, it's, it's not, uh, it's not being implemented as fast as like the models themselves. So, um, there's a lot of catching up to do there. And I think Databricks is well positioned because we have a single unified platform and we have a single governance layer with unique catalog to be able to do these types of things. I do a, uh, webcast on cybersecurity weekly and we just actually covered this this week talking about governance and guardrails and the danger of jailbreaking and all that sort of thing. But to touch on your comment about your financial services customer, you know, I, I see this, this was not a surprise to hear you say that you're seeing this traction from them because really, I think the financial services sector, sector has kind of long led some of these early transformation initiatives. Um, and so I think it's a good thing actually because you know, these are financial services, healthcare. Those are, those are areas where governance, data privacy and security, all of those things are critically important. So I think that as a use case, it's much better to step out with a customer like that. And, you know, I think that will perhaps speed some things along as it relates to some of these other things, the policies, the guardrails, the governance, that sort of thing that we need. And if, you know, 30, 40% is GNI, that means, you know, 60 to 70% is like the, the older traditional ML and that older traditional ML, you know, that's really good at math, you know, whereas the LMS are not good at math, they're good at more creative things. So you're so probably gonna need both of these things. And I think that's why there's been an interest in what, you know, what's being called compound AI systems because you're gonna have to leverage an ensemble of all these different models to be able to come up with an entire application. It just gets more complicated. Jason, maybe, maybe help, help educate our, our viewers and listeners to what some of those compound systems are doing. Some of the use cases, like when you mix prediction, you know, with something that's generated as part of something larger. And, you know, what does that application look like? Also, you know, the, in the supervised case in the traditional machine learning, you know, you built pipelines for feature engineering. And then, you know, you built, you built something where the model was repetitively, you know, doing an inference on those features. Help explain how the gen AI pipeline, what, what characteristics it shares with that, and, and then how it's different and then how they work together. Yeah. So, I mean, did you say something, Shelley or? No, it was a very big question. Well, there's a, there's a, you know, it's a very big space. So it is. So, so feature stores and feature engineering.   So that could be like an interesting metric. Now, if you're, let's just say you're Amazon and you're, and you're, you need that feature, you know, if left to your own device, you got all these two, two pizza teams out there, they're basically be recreating the same feature over and over again. And so if you know that this feature is going to be useful for lots of different teams and models, you want to centralize that feature, calculate it once, and then, you know, keep it up to date in a streaming fashion, and then, then use it for all your different models. So that's, that's why people create feature stores and Databricks has a feature store. The nice thing is that every, every inference of those features gets basically tracked with lineage, and then we can basically make those features available online in a very low latency way. And so by doing that, you're governing the individual functions that create these features that they can be used. Now, these are probably going to be used more in traditional machine learning, like for statistical analysis, but there's no reason why if there is context and knowledge of these, these functions or these features, they could be called by the LLMs themselves. So, so that's another thing. And there's a, there's a really good research paper put out by Berkeley and their artificial intelligence research center. I think it's called, it's the, it's B A I R Berkeley artificial intelligence research, but it's all about compound AI systems. And the idea is that, you know, you're going to have, if you think about an application, you're trying to achieve something, then, you know, it's probably not just going to be one giant model that you call, you're probably going to like start by calling them, you know, one model. And then every time you call a model, it's usually precipitated by doing some sort of retrieval augmentation where your question is searching for documents. And then even the, the searching and the ranking of documents, that's a whole other set of expertise that people are focused on. And then those documents are then fed into an LLM to give it some context. And then, and then that asks a question, and then it gives it the output. And you're probably going to have like, you know, a workflow or a chain of these models where instead of having one giant model, which is expensive to train, expensive to host, you could have much smaller models that are tuned for specific domains, specific use cases.  And the smaller the model, the cheaper it is to host it for the inference and the cheaper it is to train it.  And it's very nascent. Like this is very huge stuff that everybody's trying to figure out. Yeah. So let me ask you then, how does Databricks define an application today? And, and how might those different, like the old traditional machine learning and the new gen AI become workloads within a broader application? Yeah, I would say the way that I'll give you a couple of examples of how customers use Databricks. So like one example is like, you know, you're just using it like any old platform, you're doing your analysis on it. Another is like, you might, we have customers that use Databricks as the back end for the heavy data processing and, and training their models and then actually serving them up. So they're kind of like a back end for the, the customers like website or app. And then we have some customers that actually white label Databricks. So like they make Databricks available to their customers and they might put their logo on there. And then we also have instances where, let's see, customers are basically creating, you know, applications that then talk to Databricks. And in those applications, I think, are going to evolve a lot of our time. So we announced a data and I sounded last year, this concept of Lakehouse applications. So we're going to have the ability to basically have customers create an application and whatever language they want, put it in a container, and then Databricks will host that container for them. And then that can, that container will be, you know, within our customers environment. So we have over 12,000 customers now around the world. And so if you were to create one of these Lakehouse apps, any one of those 12,000 customers could conceivably discover it in the marketplace, download it and then make it available within their environment.  We already have all the certifications you would expect, like, you know, ISO 27,000, one, two compliance, bed ramp, et cetera. So you don't have to worry about, you know, those security compliances and you could basically just be running in someone's container environment. Now, another thing we released last year was SDKs. So we created SDKs for Python, Java and Go. And these basically map to the REST APIs that we've had in the platform since day one. And so these, the combination of these SDKs will allow customers to build pretty powerful applications. And I think these next generation applications, they're going to require AI. If they're not using AI, it's just a regular application. So, so you're going to have to have the SDKs that map to the APIs to do things like serve up the features or create new features to be able to serve models and then basically infer the results from that and then serve up the large language models as well. And it's pretty early days, but I think we'll see some really good examples by the end of this year. Shelley, this is, this is maybe a chance for you to start exploring how that application platform evolves. I think you just did. Okay. Well, then Jason, maybe, maybe help us, help us by describing how a definition of an application might change over time. And I think maybe some of this too, Jason, based on what you're saying is, you know, we're early days, right? So some of this is that we don't yet know what we don't know. Right. So anyway, yeah, let's so let's know that we're hypothesizing in some ways, but tell us how you think the definition of apps will change over time. Yeah, I think, you know, these people will be able to host essentially like chat bots. And then those that those chat bots will, you know, be focused on an LLM that, you know, the person, the company that creates the lake house application, they'll have some LLM that they've trained themselves on a set of data that's for specific purposes. And then as part of that interaction, you know, that chat bot will need to go out for more information within the customer environment. And I see that that chat bot then being able to, you know, have some sort of like query against the context that we have a unique catalog to figure out, what's the next set of questions that could be asked on that customer's data set. So that way, you know, the data is not leaving the customer's environment, but the LLM has to be aware enough of the context to be able to ask questions about the environment. So I see that is going to be something that I could see like one or two of these types of things being becoming the de facto standard for every industry, you know, because if those industry, if the companies that create these applications are, you know, clever enough about how they secure and, and then make data exclusive to them, then they'll be able to have the best, the best environment and the best application for, for that particular vertical. Well, this will no doubt change how developers build applications, right? In some ways. Oh, yeah, for sure. You're going to have to be much more, in some ways, it's easier, right? If you can basically just make a, make a question call to an LLM and it gives you a result. That's, you know, that's, that's pretty simply put an output. But I think the, the hard part will be, you know, really optimizing and tuning that. And at the end of the day, you're going to want to train these LLMs on your own data. So, and if you can basically train that model faster than somebody else, then you can operate these models much cheaper. So with the technology we have with Mosaic AI, we can train the same model as, as, you know, PyTorch, but we can do it like seven or eight times faster. And, you know, that equates a lot of money in the cloud, especially if we get the GPUs sometimes. But Jason, let me, let me just ask when you're talking about training on your data, there's, there's, you could have different objectives. One is you're using an LLM, perhaps the customer or, or Databricks to infer from all the data. So where you can essentially materialize the metadata that, that that's what populates the Unity catalog, you build a representation of their business beyond, it's actually one level beyond Unity, where you're learning, you're learning the physics of their business from their data. And so Unity, it doesn't just tell you what tables there are and what pipelines you have and what the lineage is, but you start building a knowledge graph that says who are the people, places and things, and the activities that connect them in their business. So that would be one direction. But then another might be, that becomes the application platform. And then within that, you might have models that say, okay, we use our Uber example, you know, a lot just to make a concrete. Now I want to match a driver with a rider. I want to calculate a fare. I want to calculate a route. So one is sort of the model of your business. And then the other is at runtime, you know, help me orchestrate my business. Do you see a, do you see an emerging pathway among Databricks technologies to help customers get in those both those directions? Yeah, I feel like it's kind of like, you know, metadata versus record level. So like, so today, like when I talk about context, it's a lot of it is metadata. So the, so like an example we gave at data summit last year was that, you know, if you were to ask the question, you know, how well is our serverless product line doing in an Europe last quarter, you'd have to have context that serverless, you know, we have a code name called nephos that waste of serverless. And then last quarter would be offset by a month because of how we do our fiscal calendar. And Europe would actually be Europe, the Middle East and Africa, because, you know, that's how we call Europe. So, you know, you'd have to have all that context, just to be able to run a query on the data. And we have enough of that information within the catalog that you can kind of like form that query and then, and run it and get the response. Now knowledge graphs would be more like, at a record level, like if you, if you have a graph of like this customer is connected to this address, and they've had this many claims associated with them, if you're insurance company, for example, then, you know, you'd have to have some sort of representation of that and more importantly, API is to be able to call it. So that's something that I can foresee that, you know, coming into the mix at some point, I don't think it's going to be this year. But, you know, that's something further out where I think it'll definitely just improve the accuracy of these applications. But we've got a, we've got a lot of, you know, low hanging fruit that we can get to before then. I mean, just as, as one example, we've got one of our customers is a Chevron and they've got all these people out in the field that are, you know, fixing oil rigs and everything else. And, you know, there's a there's a mountain of like, in the instruction manuals and parts manuals and what to do. And so we've helped them create LLMs where they just load all that unstructured data. And now the people in the field can basically just ask a question about what they're, what they need. And then it'll be intelligent enough to give them more or less the right answer without them having to flip through all these manuals. So, I mean, that's like an example of low hanging fruit today that we don't need an algebra for. But I think it's also an exciting glimpse in the future for those of us who hate instruction manuals and who hate the process of having to, oh, my dishwasher is, you know what I'm saying? I mean, it is a glimpse into the future because I think in a, in a relatively short period of time as these things go, we'll see that kind of a universal capability. Yep. So let's talk about how maybe keying off that, the keying off that example, where you start to integrate all the data, all the data types into this one end to end data estate. And it's not sort of what's in your DBMS, but it's giving coherence and definition to data that could be managed by other systems. But you want to, you want one view of it and one common definition for how to navigate it. Explain how unity starts to, you know, change the focus from what data am I managing to how do I govern all this data wherever it is. Yeah. So just kind of like using that, using that Chevron example, I don't, I don't know the implementation details, but if I were to build it, you know, I'd be, I would use like volumes within your catalog, which is basically a way to store and govern unstructured data sets. So these are a bunch of PDF files, you could have those sitting in a volume, which should really just map to like an S3 bucket somewhere.  And so that would be something that also gets governed within unique catalog. And then when you go to access that vector index for retrieval automated generation, you can, you can basically, you know, do so going through unity, every request that you're, that you're making unity is going to be audited so that you can look at later. And then everything you do for transforming or creating that vector, that's going to be logged as lineage as well. So you can go back after the fact, if you need to audit, you know, how is this model being done? And then you can also start to add security as well. So maybe, maybe the whole Chevron field shouldn't have access to all the mandates for some reason. So you can kind of like specify, you know, which, which groups have access to which things. So then when you go to do that look up, you know, if you don't have access to it, it's not going to be part of the context. So yeah. Okay. So now help us explain or help explain to us like how unity can create sort of like an abstraction layer across some like data that's managed in different systems that makes it look coherent as though it's in one system. Yeah.  So it doesn't matter if your data's in S3 or ADLS or GCS, you know, Delta Lake works on all of those. And then whenever you're interacting with Delta Lake, it's just one common API. Unity catalog is very similar in that on all those different clouds, you've got different services that they provide for identity access control. So with AWS, you've got IAM roles with Azure, you've got like either managed identity, service principles, and Google Cloud, you've got service accounts. And so once you hook those things into Unity catalog, then from that point above, it's all one common API. And we've made the API pretty simple. So we've, we've focused on the ANSI SQL standard of the domain control language DSTL. So you like grant access to this object, and that object could be a table or it could be machine learning model. Then to a group, and that group is synchronized with your identity provider. So you've got one common layer. So if you need to, you know, move applications from one cloud to another, you're, you don't have to change your code because it's all the same. It's all the same API underneath.  So that's, that's one advantage of having like this common governance layer. It really allows you to leverage these multiple clouds, which, you know, we're finding like, you know, just being able to find GPUs, like some clouds are more able to give you GPUs than others. So you're transferring workloads because of this already. So would it be fair to say that you've sort of abstracted a permissions model, and then it maps to the different permissions models? And that's one simplification. That's one simplifying abstraction. Would, would lineage be another one? And tell us some more simplifying abstractions that would work across systems that, that Unity provides. Yeah.  So you can register different databases. So those could be Postgres, MySQL, Oracle, or even Snowflake, and register those as what we call foreign catalogs within Unity. And then once you've done that, you've registered their metadata. And now we, Unity has context about these datasets. And when it goes to auto-generate queries for the data intelligence, it can do so against those. And that's one. But then two, you can, you can federate queries out to them. And then all of those queries would be tracked from an auditing in a lineage point of view. Now with lineage, I think there's like, anytime you talk about lineage in general, there's like three components. There's like capturing the lineage. There's like persisting that lineage in a, in a common format, and then indexing it. And then there's the visualization of that lineage. So everything you do in Databricks, we automatically capture that lineage for you. Now, if you want to, if you want to do use some other system as an engine, we're not going to capture that lineage. We don't have any knowledge of what, you know, Oracle or Teradata is doing. But if you, if somebody were to write a collector that would capture that lineage, then you can push that back into Unity and then leverage all the visualizations we have. So, you know, we can accept a third-party lineage and we can basically automatically collect what's done in Databricks. But we're not going to try to, you know, create the collectors for 200 different systems. So so the part of the platform is to let the, let other providers write collectors so that you can have a global view of lineage. Yep. And when you, when you map these external databases, so lineage might be one thing you could collect from them if they write a provider. But what other context are you capturing that allows the Intelligent Lakehouse to query their, this external data as if it were native? What external context do you need to map that, again, just the way you're mapping permissions? I mean, really, you just kind of need the metadata as well as, you know, the lineage. And then in that metadata, you've usually got some sort of descriptions. And one of the things that we did, one of the first applications of LLMs we did in the product was if you open up Unity catalog, you can automatically create descriptions of tables and columns based on what the metadata is. So most customers, most customers, they don't actually ever fill in the description fields on their metadata, but we can auto generate that for them. And then by doing that, it just makes searching for things in your catalog that much more accurate. So I think it's awesome that you can do that for them. That's a, you know, I mean, that's a step that is important. And we trained our own small LLM to do that. Love it. And love it. Well, I think, George, I'm going to give you one opportunity before we wrap this show to ask another question, and then I'm going to move on to our last question. So, okay, think of, or should we think of Unity as the application platform? And if so, how does that grow over time to make the developers experience more intelligent, you know, so that they're less down at the physical technical metadata level, strings, and they're more thinking about the people, places, and things. I know that gives you a lot of room, you know, but just help us think, help us reframe how that abstraction is going to maybe evolve over time. Yeah, I mean, I don't think of Unity as like the application platform per se. I think it's more of like a governance layer. And as part of that governance, it's capturing and persisting a whole bunch of metadata. You know, you can think about like, hey, if you've got two tables in your data lake, and one of them gets queried like a thousand times a day, and one of them gets queried once a day, you know, which one is more important, you know, if you thought it was this one that gets queried a thousand times a day, you might be incorrect because unless you look at the lineage, you might know that this one is actually feeding the other tables. So you need lineage to have the complete context of what is the most important, you know, versus other tables. So I think, you know, that's just one example of how the metadata can be interesting. Another thing is, you know, which columns get joined on the most, you know, and that could kind of like give you some sort of insight about how people are using it. Yeah. I mean, in terms of like application and hosting, I think like our model serving layer is a really good example of this where if you want to do model serving where you've trained a model and you want to make it available behind a rest endpoint for an application to hit, you know, you need a way to host this thing so that it's up and running all the time. You need a way to automatically scale it. So if you have more requests coming in, you know, all of a sudden you can scale out the compute behind that and then also scale it back. You also need to be able to monitor this thing. So for every request that goes in, you know, what was the inputs and outputs? We say that to a Delta table and then you be able to monitor that. So if the result of that model starts to drift over time, you want to be able to have a way to query that and then, you know, set up alerts so that if it starts to drift too much, you can trigger actions to either retrain the model or have somebody go investigate. And so that's what I kind of think of as a really small application. Now, that one is just like a rest endpoints. There's no UI, but you can kind of like build it out from there where you're eventually going to add a UI and you're going to add, you know, calls to multiple model serving layers as well to try to build a more complete application. Okay, that's helpful. Yeah. So Jason, as we kind of close in on the last part of our conversation here, I want to talk for just a minute about, you know, we did touch on this earlier, the Intelligent Lakehouse, but can you just share maybe broadly what you see Databricks vision for that Intelligent Lakehouse is? Yeah, I mean, I feel like, you know, being able to address the rest of the users of an enterprise, you know, the tens of thousands, if not hundreds of thousands of users that are not really like democratizing, you know, data and AI, they're basically leveraging other teams that do that for them. I mean, think about all the questions that never get asked because, you know, if you have to ask your data science team to go figure out, you know, what's, how was this category of product doing at a particular location, you know, you're going to be, you're going to be thinking of like, do I want them to spend time answering that question or these other 10 questions? So when do they have time to talk to me? I got to wait in line. I mean, that's an issue too. Yeah. So if you just had an easy way to ask these questions and get good enough results to guide you to your next decision, you're going to ask a lot more questions. And I feel like, you know, that's, that's a vision that's not going to take a year to develop fully. It's going to take multiple years to really optimize this. And so I see that as kind of like the evolution of where things are going is being able to democratize or, you know, make available the insights of your data to a broader set of users without having to, you know, upskill them in programming languages or data science. So that makes perfect sense. So then I'm guessing that is a natural part of this process. We'll see more and more semantics that are captured in cattle. Is that correct? I think so. Yeah. I mean, I think George is onto something with this, this knowledge graph. Like, you know, we've, we've got a, we've got a lot of stuff to build this year, but I could see, I mean, not just Databricks in general, but like just the industry moving towards these knowledge gas, if you can get you a more fine grained, optimized answer or so. Yeah, I love it. You know, George is always onto something. That's just who he is. So who do you think will author data semantics and how, you know, will it be a combination of a data engineer and an LLM mining that metadata? What do you think? I mean, I kind of feel like the answer is yes. Well, I think, you know, if you look at a lot of these semantic layers that have been generated over the years for every VI tool has got a semantic layer. And now, you know, there's been more interest in semantic layers and metric layers in recent years, but for the most part, all of them are still like manually curated. So someone is manually going in and specify what, you know, what type of data this is and what currency it is and description of it and et cetera, et cetera. And I think the evolution will be like, how do we basically, you know, get to get to the same, if not more accurate response by doing less manual curation. So how can we basically automatically kind of infer a semantic layer rather than, you know, having somebody manually curate it for you? Because that's going to be what the real efficiency is. And once you kind of like, once you make things super easy, like one click, people don't want to go back to manual stuff. Absolutely. We all live that, right? All right. Well, Jason whole, and thank you so much for joining us today. We have so appreciated you sharing some of that, some of the work that you're doing at Databricks and some of the forward looking vision.  And, you know, as we're seeing this emergence of the six data platform and this change within the industry, it's, it's, it's really exciting times. Thank you so much, Shelly. It's, thank you, George. It was, it was a pleasure to have a conversation today and looking forward to doing more of these. Well, I have a feeling that this is the first of, well, I know this isn't our first conversation with you, but we'll be talking about this for some time to come. So we'll be, we'll be tapping you to rejoin us again in no time. And we look forward to that. So George, Jason, that's a wrap. Thank you so much. Thanks to our audience and our listening audience. And we'll see you here next time.