Well, welcome to the show Dan. We're so excited to have you here. Yeah, well, like I was saying earlier, it's great to be here. I started listening to you guys around 2015 when I first started getting into podcasts and listening to podcasts. And so to be on your show is a real pleasure for me. So thank you. Yeah, well you came highly recommended by a good friend of the show and Johnny and I have been so focused on not only communication, but leadership both on our own team, helping young leaders who come through our training programs. And as a Navy SEAL, you've led in multiple capacities inside and outside of the military. And now it's really one of your passions and pursuits in your career. So I'd love to just hear a little bit of your background, what sort of got you so jazzed up about leadership and what you're working on currently. So grew up in LA. I'll start out where you guys are located. So I actually grew up in Southern California, joined the Navy 20. So I traveled real far down to San Diego after boot camp and went to Buds down there. So while I was in boot camp in Great Lakes, my stepdad passed away and I actually wanted to go to the East Coast just to look for something different. And because my mom and everyone I knew, most of my family was on the West Coast. So I ended up staying on the West Coast. I went to SEAL Team One. I did 10 years at SEAL Team One. So most of those deployments, Iraq, Afghanistan, the time that I had joined, I was at SEAL Team One and essentially I was there for maybe six months and 9-11 happened. So from the beginning of my career, almost all the way through was essentially 15 years of combat. So every other year deploying overseas. Like I said, 10 years at SEAL Team One and then I went to the East Coast. I was served at a East Coast team for about five years there. Same thing, deployments to Afghanistan, Africa, Somalia. So places that you won't visit on TripAdvisor. And after some, just a long road kind of burn out to be honest with you, I noticed everything was just in a downward spiral. Attempt to get out of that spiral. I took orders at Naval Academy. It was probably the best move that I did both personally, professionally and just started to rebuild the foundation with myself and with my family and started to prep for what was next. And that brings me to today, I'm a leadership coach and right in my dissertation this year, hopefully graduating in nine months with a completed and past dissertation. I'm writing a dissertation about trauma resilience specific to the SEAL teams. So I think that kind of hit the big wave tops. Yeah, and I would definitely love to hear a little bit more with obviously from that background in the military to then moving into the Naval Academy and now looking at coaching and leadership. What are the lessons along the way? Obviously drawing from your time abroad, but then also in the Academy where it's a little bit different of a lifestyle than I'm sure you were used to going to some places on Trip Advisor. Johnny and I wouldn't necessarily visit. Yeah, and I wouldn't revisit either and don't recommend. What I saw working for so many different enlisted and in various officers was such a difference in style. Everything in the military is very by the book and with leadership it wasn't. There was some leaders that were amazing on and off the battlefield. There were some leaders that were amazing off the battlefield, not so great on the battlefield and vice versa. There were some leaders that were just terrible that we questioned how they got there in the first place. And then I started noticing even myself how I performed as a leader depending on the environment, how the environment influenced my own behaviors, how I felt inside. So I just noticed all these things and then I started to formulate like everyone else does their own opinion about it. And then I also realized everyone else has their opinion on leadership. And then I was curious, what does academia have to say about it? So that's really, so asking that question of what is research found, what makes leaders so different on this spectrum of amazing and terrible within the same rank structure too, whether it's middle management, upper management.  And then obviously going to the Naval Academy towards the end of my career really supported and supported me in pursuing the academics. I think that's where the biggest myth lies when it comes to leadership is that different leaders are going to lead differently and have different ranges of success due to the environment that they're in. And being a podcast fan myself, one of the things I had taken from the great Dan Carlin and listening to his podcast is there are great leaders during times of peace that are able to lead very well, they do very well in that realm and people look up to them but completely fall apart when it comes to leading during times of war and battle and having to replace those leaders with leaders who are going to excel in that arena. And I think the myth is that a good leader is going to be a good leader no matter what, but there's going to be different skill sets that are going to be of value in those different arenas. I agree with you 100%. I think for many of us, we're not thankfully going abroad and deploying or putting ourselves in a life or death situation but you've had those different contexts now to compare leadership styles, leadership methods, mindsets and I think leadership is one of those loaded terms that we all kind of come with our preconceived notions around. So I'd love to draw on that experience that you've had across these various arenas, as Johnny said and really define what leadership means to you. How I approach leadership is through frameworks. And so as Johnny said, everyone approaches it differently. We're all unique in our own different ways. So even if we have create different boxes and we take different personality tests, we have different values, we have different backgrounds, we have unique geographic region. All this stuff are upbringing. Leadership is really how you lead AJ or how you lead Johnny is not my question to answer. It's your question to answer. And so for me, what I like to do as a coach is just show up with framework to help the individual really how they mentally look at leadership. So if you look at all the definitions and there's millions of definitions on Google for leadership, they all fall into three categories. It's a category of self-improvement, self-care. It's a category of helping others out. And it's a category of accomplishing a mission task or goal. So often if I'm working with an individual or a group of leaders, I'll ask them their definitions. And often I'll listen, because as you listen to somebody, they're essentially communicating goals and values and culture in the communication. So as they tell me what their definition is, I often look for the deficiency in it. So it's like, oh, it's taking care of the, okay, got it, they're tracking the team. It's a team in the mission. Okay, they got those. Now, where do they fit in it? And so that's where I start working is really expanding their lens to see a bigger picture. And then another thing that I'll add in with leaders is what do leaders do? So there's, once again, out of all the definitions, all the academic nerdery that I can dive into, what I like to try and do is really take complexity and simplify it. Leaders direct, organize and influence. Very simple. And so now we'll go and we'll talk, how do you show up both professionally and in your personal life? Because a lot of the leaders, and once again, this framework, we'll start with their place that they value the most and that where they value the most may be at home. So to start to get things going, if they value something the most at home, the idea, what's the hardest thing in any goal is just to get going. So really that's just what I'm trying to get the individual to do is just get in this habit of practicing. And that is probably the biggest thing that I've seen in the military and the civilian world is not only the lack of training, but the lack of taking knowledge and implementing it. So moving from knowing to doing. So for folks that listen to podcasts, they're like, Dan, I do all the leadership, I read all the books, I listen to all the podcasts. That's awesome, I love it. Keep doing all that stuff. And now tell me, what are you doing with that information? I listened to this podcast on Art of Charm with this amazing leadership coach, Dan Luna. Awesome, love it. What did you extract from the podcast? What were some key points that you can now implement and what is implementing look like in your day to day? Or how does that change how you show up as a leader? So that's really what my approach and style of coaching individuals starts out with. What I love about that perspective and lens is how the self is infused in it. I feel that many of our ideas around leadership are all around the other people, the people we are leading, building the culture, and there's not enough self reflection built in on how you yourself are leading from the inside out. So with this framework now, and as we see the pieces starting to come together, do you see that there are a lot of leaders who have a stronger side in their personal lives, their personal lives, their professional lives? Do they lean on one or the other? And when there are deficiencies, how do we start to improve those deficiencies? So take that lesson, that framework for you, and actually implement in our lives because I know some in our audience are already in a leadership role. Many in our audience are looking to get into those leadership positions. And in order to do so, it's gonna be really important for them to take this advice and as you say, put it into action in their own life. I think what Johnny was saying earlier about how there are certain individuals that can fit in certain spots at certain times. And what I've noticed with leaders and it almost goes into this idea of work-life balance, which I'm not necessarily a fan of. I get the concept and the idea and I definitely spent a lot of time working with individuals on how they can do more self-care and how self-care shows up more in their life. And there are kind of a time and place. So there's moments in time where somebody may have to spend more time at work and then things will plateau out. And so with leadership, I've seen that same thing. There may be an individual that is more focused at the workplace and as far as leadership, being a leader, showing up and then they go home and they don't carry over those same frameworks or that same value or that same idea.  Like, hey, a check-in. So if I'm taking out my team members once a month, how often are you taking out your kids? If you have, you know, I have two kids, so I try and make it a point to do something individually with the kids. So there's framework that's very easily transferable. And so that is, so really just helping leaders, if there's a mission statement at the organization, what's your mission statement at home? So if there's a value statement at work, what are your values outside of work? And we can get into ethics and ethical practices and all that stuff. But it's really like, really, how are you deciding to show up to really our human experience? And with that, if there's differences or if there's misalignment and it's not for me to judge, it's really just to help the individual see what that word is and where the work that they wanna put into. And once again, it's also their decision on where they wanna put the work in. So that's kinda how I lay things out as well. And it sounds like leadership is not a switch that you just flip in the moments where you need it. It sounds like it's a practice and a muscle that you need to strengthen in all areas of your life to really be that great leader.  But instead, looking at it more holistically of how am I showing up with my friends, how am I showing up with my family and how am I showing up with my team? So there's a framework by Bandura and it's called social cognitive theory and I love this framework and it's self-behavior and environment. What happened, we have this saying, what happens on the inside shows up on the outside. So typically my first few months with the client, and this may sound like a lot, but I just, it sounds like a lot because I have the time with the clients. We'll spend three months just with them. Three months just on themselves. Not with the team, not with the organization, not with the kids, just on them. They're journaling, they're doing soft goals so they're noticing. So they're really becoming a scientist, if you will, in a new environment and that environment is themselves.  So really reconnecting with the body. Western culture, we tend to be very, you know, cognitive from the neck up. So we'll start to drop down. I do a lot of somatic work with clients, dropping the breath down. I'll do some nervous system work. So really look to regulate or reregulate their nervous system. So a lot of Vegas nerve work, work on the vagal break. And so there's just with the breath, just dropping your breath.  So if their feet are flat on the floor, so we'll do grounding techniques. So if there are certain meetings or people that we know that may cause a heighten arousal with them or anger or, hey, this person at work. So we'll work on grounding techniques.  Okay, well, what's that look like? Oh, you need to ground. Okay, what's that mean? So we'll really spend a lot of time in this area just on the individual. And so as they learn these techniques and they explore, they become aware. So they become aware of their thoughts, everyone journals. Even if you don't journal, you journal. Yeah, well, I think it's a great moment again of self-reflection, especially for those in the audience who might not be in that leadership role that they're striving for yet. This idea that raising your own self-awareness, being self-guided in the way you show up, in the way that you lead yourself, in the way that you understand the emotions, the thoughts, the patterns in your behaviors, that might be impeding you in your communication, your ability to influence and your ability to lead in the future. Sounds like it starts internal first, so that three months is really building inside out. And then how do we incorporate that now into the way that we show up with the teams that we're leading? I made this journal and I printed it and it's a guided journal. And so I sign all the journals and what I sign inside the journal is leadership starts with you. And you said it, it's this inside out. What happens on the inside shows up on the outside.  I understand the idea. Leave your personal life at home and show up. It's almost as silly as asking somebody to leave an arm because your body is your hard drive. Like it's your container. Where's your personality stored? It's not stored, like it's in your body. Where's your character? Where's your values? Where's your past history? Where are all these things? Where's your thoughts and emotions? It's in your body. It's in this container. And we move this container around with us. So it's very important to understand and to spend that time, what's going on in this container? And how do I move? And in this movement, what is that movement telling me? And so self-guided, so many people will, fancy people out there in the leadership realm will say leadership is influence. And I love when they say that because influence is what? External, it's outside. Everything for our eyes are outside. It's outward facing and forward facing. They're not looking internal or behind us. So now how do we dive into ourself? And so for what you just said was self-guided. And I think that's so important, self-guided.  Whatever age you are, it is self-guided. So as you transition through different phases of your life, you have choice. And you make choices internal that show up in your behavior. I just wanted to touch on what you said because I think it is so important, AJ, that self-guided. The position, the title, you can still influence and model and do all those sayings with or without the title, how you speak, how you behave, how your body language, everything that you guys are experts on, it starts internal and it's that guided, it's that decision. And if you want to change, that can be a very conscious decision as well. I grew up as an only child. And when I went into the SEAL teams, I noticed that, oh wow, I was doing things for me because I didn't have to think about others. And then I realized that is not the type of person that's not the behavior and that's not the type of person I want to be. And so it took very conscious effort for an only child to really be team focused and it took time and energy to then switch and adjust that behavior. Well, it's incredibly important because if you want that leadership position, if that is something of a goal that you're working towards, but yet you're unable to lead even the person in the seat next to you, well, then how do you expect others to see you as a leader as somebody who they're gonna put in that role? That starts with yourself. When you are self-guided, that others start to look at you in that office and they're thinking, well, he seems to know what's going on, I'm just gonna follow suit. Well, now everyone in the higher ups are looking at, well, that we have a natural leader downstairs, everyone's already following him. That's the guy that we're promoting. He doesn't need any work. He already understands that in and himself. For there are many people that we have encountered in this business who are like to say, well, I will do all the things I need to do when I get that position. Those things start now. I agree with you, Johnny, 100% and it's often because sometimes I'll hear leaders talk about control and controlling others. Oh, I'm trying to do this and they won't listen. And we'll go back to the self and I'll remind them, like, well, can you even control yourself? And so many people often think they're like, oh, I can control myself. And then we'll start diving into that some more.  So I think that influence and really understanding influence and persuasion, in my opinion, as you learn to influence and another term, just really work with your own energy and recognize how the combinations of your thoughts and emotions and your values show up and how it shows up in your behavior. It'll help you work with others and just being patient, meeting people where they are, just like I remind clients to meet themselves where they are, where themselves are and to there's this other saying, there's no bad parts.  And it's not, it doesn't serve the individual in really growth, it's self-deprecating.  So once again, going back to the individual and just having them pay attention to their words. Behaviors and actions don't lie. Exactly. And the words we choose in our communication about ourselves and about those around us is a very clear window into what's going on subconsciously for you. The stories you have, the limiting beliefs that may be a part of your narrative. The piece that I think for a lot of us we don't often realize is just how strong mirroring is between humans. So this in-group, out-group phenomenon, as much as we wanna control other people, that control that you're looking for actually starts within. The more you can control your reactive state, the more you can control your body language, the words that you use, the more influence you actually have over other people. So as humans, we look to mirror our leaders. We see someone with that strong frame that we look up to, that we admire, that has the leadership role and we do look to fall in line. We are driven towards hierarchy as humans. So as a good leader, it's not about what you're putting out there consciously, but it's about what you're bringing to the table subconsciously and how that's being communicated in the words that you're using, your body language, the way you're showing up in those situations. So this internal frame that we're talking about here, it does feel a lot of what we think about military leadership and the warrior mindset that you have to be rough and tough and grumble to be a good leader, right? You have to be hard nose. You have to be tough as nails to be seen as a leader. And again, going back to our earlier conversation around the different contexts that you've witnessed and experienced and studied leadership in, is that a misnomer? Is that really what leadership is inside of the military and this warrior mindset? I would say it depends on the culture. I'm gonna give you a political answer there. The military is so, as you dive into the military, even just some branches are so different. As you dive into the branches and you separate the branches, somebody can be in the Navy, but they're a pilot or they're a SWO, they're a surface officer or they're on a ship or they're a SEAL or they do explosive ordnance or they do master at arms, whatever it is, each little subgroup has their own culture within it. Going back to the individual or the environment that they're in to look at it and to understand it and then to really decide how they wanna participate in the environment. So roughly, rough numbers, 70% of people learn from the environment and will mirror, model that behavior that's in it. I like actually, if it was up to me, I would up the 30% numbers. Like I don't necessarily want the environment to drive how I act or behave. I want to influence the environment. I decide my choices and how I participate, my behavior in that environment. So I actually, so a lot of this goes back into reverse. So I would say most people and statistically, most people are heavily influenced from the environment.  But once you get to a certain age and become an adult, that's kind of where you start, where I call it like peeling off the layers. Start peeling off the layers of your parents, of your economic status, of whatever those limiting beliefs are and really going down to yourself and the philosophical idea of like, who am I and what am I here for? And you don't necessarily need to go that deep but just really understanding how does the environment that I am in, how does it influence me and does it influence me in a way that I actually don't like or that I like? And there's some same thing with people, right? So it could be a work environment, it could be certain people, there's certain people that I notice will shift my behavior and my way of thinking. And I'm like, ooh, I need to be careful. But noticing it and having the awareness around it is really the first step. What we're talking about sounds simple. So AJ, how you framed it, it sounds really simple, but it's not easy. It is so much work. And for some individuals, maybe it's a lot more work. For me, it has been a ton of work and it's work that I'm still in, my own personal development and like unwrapping those layers and trying to be a better dad and noticing a shift when my teenager comes home and I can hear the music blasting out of the car before he turns down the street and I'm just like, that's my kid or you can meet, right? And I want to go out there and be like, what are you doing? And then I pause and I put myself in teenager shoes. I remember I was that same kid. So it just, and meeting him. So all these things that I say at the workplace, I can't do with my teenage kid, like somebody who I love like more than almost anyone else on earth and I can't go out there and meet him where he is and like, hey, pal, like let's talk about this and ask him questions around it and partner with him.  When we think of the word being clutch when it comes to sports, that means that you are trusted to be consistent no matter what pressure is put on you, whether it's the playoffs, whether it's a regular season game, whether it's the Super Bowl to be clutch is you're gonna play it how you always played it and the environment melts away in those moments. Now, you may be leading an office and you're not on the field trying to score on the fourth quarter with 30 seconds left in the game, but you have quotas to meet. Your team needs to be performing and reaching those quotas. And there's a lot of people who are dependent on you leading that team to those quotas. So, the pressure of playing in front of 50,000 screaming fans in a stadium is still the same pressure of a lot of people. And if you're depending on where you're leading, stockholders and board directors, wondering why are things not working the way they are supposed to be working? Why is this leader not leading? Why is he not consistent? This is what he was paid to do. That pressure is there. For anyone to think that the idea of being clutch as a leader is going to be easy, has not gotten off the couch and put themselves in a position to actually lead. I view leadership in the lens of a muscle versus a destination. And I feel unfortunately, a lot of what school teaches us and a lot of what we view as leaders, as it is a rung in the ladder, we've reached that now we can rest, relax. Okay, I have the title, I have the respect. And there is a degree of earning it to that point, but all of the leaders that we talk to look at it as a lifelong process.   And that self-awareness piece as you talk about, that's not just a light bulb that goes off and goes, oh, hey, I know that in this environment, I as Dan show up in this way. Okay, that's good. Now I got that checked. Let's figure out something else and let's influence others. There are always gonna be these variables, these environments, these external factors that we are gonna have to deal with. It doesn't matter if it's the combat or the boardroom, your ability as a leader is gonna be constantly tested. When you're in that role, it's tested by the mission, as Johnny said, the quotas, the numbers you need to hit or the conflict, the communication on the team, the culture are these people who are joining that team that you're leading the right fit. They're also going to be testing you.  And we talk a lot about journaling on the show, but one thing that we hear time and time again is I don't know where to start. I really don't know where to start. And I think you do a great job in your leadership journal of just helping people with those prompts start to understand where to start from a journaling practice. So some clear examples for those in our audience that are again, trying to take this valuable lesson from the podcast and really implement it in a self-directed way. What can we do to start journaling from a leadership standpoint to raise that self-awareness? Most people don't understand the value. And so they don't do it. So like, yeah, journaling, I heard it's good, but I don't really, whatever I've heard these sayings, journal, journal, journal, what about it? So the utility value in journaling and what journaling does is it helps you practice extracting your own inner wisdom. So as a leadership coach, when I'm working with individuals, you know, AJ, I just noticed this.  The topic is emotional. What did you notice? Did you notice that you swallowed? Was there something going on with that? So I may ask a body question. I may ask, hey, as we've been talking this conversation, there's this theme of this. It's come up a few times. You know, what's your thoughts around that? Like, oh, I didn't notice that theme or I didn't notice I did that. So journaling gets you in this practice of asking questions. So moving from statements to questions. So so many times in life, I'll go back to my 17-year-old driving around the corner with the music blasting. I can make a statement. I can't believe he's doing this again. I've told him several times. And now because of that, I'm gonna go down there and yell at him. Or I can say, like, ooh, I wonder what's going on. How often does he do it? You know, where did he just leave the gym? You know, is somebody else?  Let's go explore. So we're actually moving from judgment to curiosity. And so what helps with moving from judgment to curiosity is moving from statements, making statements, whether it's an individual showing up late, whether it's somebody not meeting an expectation. And that's where conflict essentially comes in on expectations, making statements to being curious. And so curiosity shows up in asking questions. And so in journaling, you start practicing the art of asking questions. And you start with yourself. Once again, everything starts with you. So the art of asking questions, know why questions.  How I look at that is it is a great start. So if you're like, Hey, I'm really, I'm thinking about doing jiu-jitsu, awesome.  That's where why questions fall in. Like I want to start asking questions. Hey, why questions? Sure, that's a great like white belt place to start. But now if you want to like continue and develop this art, we actually move away from why questions. There's research on actually diving into and constantly asking why question creates a negative feedback loop. So we move from why questions to what questions. Once again, which takes time and practice. So even for me, it's simple in language where I'm like, oh, why just in a conversation? And now I've done it so many times why have to reframe it to a what? And it becomes difficult because now you're moving sentences around to form it. So it takes a lot of practice. And then you notice because what is a knowledge question? What has motivated both of you to continue this podcast for this long? Vice, why have you continued this podcast for so long? And so the other thing that why questions do is it tends to put people on the defensive. AJ, why'd you ask me to come on your podcast? AJ, what about me led you to asking me about your podcast, asking me on your podcast? So it's a different, so just that first word reframes how the individual receives it. And so same thing for yourself. So if I'm sitting there journaling, you know, why did I do this?  Well, why did you do this? Because you're, so you like you can really create a self spin up even in your own journaling. So know why questions. Stream of consciousness is the easiest way to start. So just sit down, pick a place, just like any other goal I want to work out, you know, probably you're probably not going to start out working for working out for an hour. With journaling, I'll ask clients, you know, what can you commit to? Oh, I can commit to 20 minutes, you know, every morning or every evening. I'll go, awesome. How about five minutes? And I'll just, I'll, I'll slash it because they'll, because with goals you have to calibrate new goals. There's a calibration in there. And so I'll slash it because all I want them to do is start doing it and see the value in it and get their, get their repetitions in the practice. And then I'll create a window. How about three to five minutes? And they're like, yeah, I'm like, is that doable? They're like, oh yeah, it's totally doable. And even with that, that still takes there's a week they'll come back, not in journal. And then I hold them accountable and then they start journaling a couple of times and then it starts growing and expanding. And then they start seeing the utility value and they start going through and like, oh wow. And now it prompts our conversations as well. So we'll start talking about the things that comes up in their journaling. And they're like, hey, I noticed I keep going back to this one event every time I sit down in journal or every time I go to journal, I avoid it. I'll even ask questions about that because they're now thinking about certain things when they go to journal, which is once again, we're just extracting information and helping them dive into themselves. So the key to journaling is it's, you start the art of asking questions and you learn how to really excavate your own inner wisdom. There's so much to unpack there. I want to first start with the why what because I feel ultimately in our default we have this search for why because we want to search for meaning, we want to grasp onto something and we create this unnecessary certainty behind our answers. The why, as you said, has this context of judgment versus the what, which is the impartial observer, right? So even in your self-inquisition, we don't want to place judgment on the thoughts, the feelings, the emotions that we're having because that creates the tangled web then of us turning those into beliefs, turning those into behaviors about ourselves or a lack of action in some areas if that why becomes so strong? Well, why don't you do that? I'm an introvert. We hear this all the time. The label, self-label, I'm an introvert. That's why I didn't do that. What about that was challenging for you? Well, I was lower energy because I have introversion, right? It's a totally different shift when you go from the what to the why and oftentimes we'll do it in conversation with others but it's even more powerful when we turn that why back on ourselves because we start to form beliefs about ourselves, beliefs about how we show up in the world that impede us from the growth that we're looking for. The second angle that I wanted to touch on in the journaling practice is we all have goals. Like everyone listed in the show is goal-minded. The biggest problem that we find with our clients that we work with is a lot of times when they haven't taken any action towards a goal, they feel like the greatest leap is actually gonna get them closer when starting but it's actually the inverse. It's the smallest step done repeatedly that'll create the greatest impact on the goal that you have. And we hear it time and time again in X Factor Accelerator like, okay, you've joined us because you're feeling a little socially anxious, you haven't had as much socialization as you'd like coming out of the pandemic, you've been working remote and some of your friends have moved away or you haven't been on a date in a little while. Okay, what goals do you have for the program? Oh, I wanna go on five dates. I wanna talk to 10 people. I wanna have three friends by the end of the month. Okay, all great goals but if you haven't taken any step in that direction, there's no way that you can create that expectation, fulfill it in a way that's gonna empower you to keep working towards that goal. And the last thing you wanna see when coaching someone or working with someone is they set the goal too ambitious, they fail to act on the goal and then they just stop trying entirely. And as you said, the practice of journaling, it doesn't matter if it's 20 minutes or even three minutes, it's getting used to this idea that I'm gonna document what's going on for me internally, what I'm thinking and feeling, how I'm processing these thoughts and getting it out on paper as we've talked about on the show before, it uses a different part of your brain and it allows you to actually see and reflect in a way that's different than just listening to that inner critic, that dialogue inside of your head. That's why I've added calibration to it. So every time, as soon as I talk about goals, I'll tell them, hey, there's a calibration period. And so right off the bat. So what I've also noticed with folks is as a set of goal, I'm gonna get up at 4 a.m. from now on, just all the time, because I listen to this podcast and they get up at 4 a.m. So I'm gonna get up at 4 a.m., like, okay, cool, awesome. A week later, how's that 4 a.m. working out for you? Like, oh man, I'm actually sleeping in more, it's gotten worse, or whatever. So we'll dive into it. So, hey, what time do you wake up now? I wake up at 6 a.m. Okay, well, how could you wake up at 5.55? And they're like, pfft, that's nothing. All right, well, let's just start there. So once again, just getting that movement and then they get up at 5.55 and what's that do? Oh, that was super easy. Okay, well, what do you think makes sense for the next one? 5.30. Like, let's ease into the goal. And so often, what you just said is there's this big, massive jump and it's really hard to continue. Some people can do, like, there are folks that are, you know, maybe they're probably listening, they're like, yeah, that's no problem, I already do that. Awesome. For the majority of people out there, they're not doing that.  With those folks, it's very environment dependent as well. So even if you have examples in the past of making that big leap of setting that goal and hitting it immediately, it's very hard to maintain that in all aspects of your life. They're typically in that situation was a massive identity shift that you were now living into that made that lofty goal for yourself really easy to attain. But oftentimes we don't have the identity yet for the goals that we're reaching for and we don't then have that willpower necessary, that motivation to stick with it. The flip side of this, of course, that we experience as leaders is outside of our self motivation, how to influence the motivation level of our team members to achieve the collaborative goal that we all have. And as Johnny and I have talked about a lot on the show, this top-down approach to leadership, being authoritarian and just barking orders at other people, it might motivate a small segment of that team, but it's not gonna be a large enough segment to consistently reach the goals motivations that you have as an entire group. So what is your approach when it comes to coaching leaders on how to motivate outside of themselves and how to communicate and lead the team when there's conflict or people aren't reaching their own goals that you have set out as a leader? So I really like defining things. So whether it's my definition or your definition, hey, let's give this thing shape. So if we're gonna use this word motivation, let's give it shape. And then as we give it shape, then we can measure it. So if we define it, we can measure it. Right now, resilience is a great one. So there's over a hundred different, there's more, but particularly, you know, for my dissertation work and what I'm writing about within the military, there's over a hundred different definitions of resilience. So if there's that many definitions to define something, how do we measure it? And then what are we measuring? Are we measuring process? Are we measuring outcomes? Are we measuring attributes? So how do we get our mind around this?  So, oh, I'm really motivated to work out every day. Well, what motivates you? Well, I choose to live a certain way or whatever that choice is. Well, even though I choose to go work out, what's the mental effort in which I do and I put into that? Like I could go downstairs and ride the bike for 30 minutes and not break a sweat and somebody else could go downstairs, I have a gym in my basement, or I could go downstairs and work out for 15 minutes and crush myself. So how much effort am I putting into that thing that I'm choosing and then persistence? How long am I doing this thing for? And so for motivation, that's a great way to break it out. What does somebody choose, persistence, mental effort? And then when it comes to goals, there could be a motivational gap. So in what they choose, it could be lack of confidence. Oh, I don't think I can do it. So now we're talking about self-efficacy. It could be, I don't find value in that. So now we're talking like utility value. It could be an emotional like, ah, it doesn't feel good to me or I don't feel good, right? So now there's an emotion behind it. So there could be, so all of those kind of fall in the category of motivation. Another thing could be a knowledge gap. They simply don't know. And then the final thing is classified. So the framework that I'm sharing with you guys right now is called KMNO. So Knowledge Motivation Organization. It's another framework. It's actually a performance framework for gap analysis. And I'm being a little nerdy here. I feel like I'm looking at you guys and you guys are like two really cool dudes. And I feel like the nerd on this podcast right now. So if we set a goal, I want to lose 10 pounds. We're essentially saying that there's a deficit that I'm missing something or I want to make more money or I want to do this. There is a space between where I currently am and where I want to be. So if there's a space, there's a gap. So now how do, what are the pair of glasses that I put on to look at that gap? Is there something that I don't know?  Awesome. Do you know anything about real estate? No, I don't know anything about real estate, but I'm super motivated. Okay, that's great. So now how do we close the gap? I need to learn more about commercial real estate. Okay, do you have the resources to get into commercial real estate? So kind of the, oh, the organization. Nope, my company has no money. I have no, I have nothing. Like, okay, maybe we need to work on some resources to bring in so then you can then close that gap. So once again, there's many frameworks out there. This happens to be a framework that over the past few years have done my dissertation. I picked because my audience is the SEAL teams who are deeply concerned with performance. So I chose a performance framework. Well, what I love about that is it allows you to start to see the pieces in that gap that are missing. So we laughed about this earlier, but that common refrain of I've read all the books, I've listened to all the podcasts. We often overindex on the knowledge piece because it's just freely available. It's everywhere you look, every platform you on, there's knowledge, there's little tidbits of knowledge, there's science studies as we've gone into some of these deep mindsets and frameworks here. They're all based in great experiments in science. We've shared a lot of knowledge, but oftentimes what we find with the clients that we work with, they're failing in the other two areas. They don't have the resources available to them in the organization piece and they haven't really clearly aligned the motivation behind it. They're not quite sure. And sometimes that motivation waxes and wanes, and that's really where mentorship and accountability and coaching can come into play to help close that gap faster, right? But the last thing you wanna be doing is just sitting there spinning your wheels in the knowledge department, constantly accumulating more and more knowledge, but that gap never changes. It'll certainly allow you to argue on Twitter for hours, but it certainly ain't getting you any closer to where you wanna be. It goes back to the beginning of this. Talking to folks like, oh, I do all this stuff and I know, awesome, what are you doing with it? Or what's holding you back? What is your leadership? The other things, what's your leadership workout look like? So if leadership is a muscle, like you said earlier, how are you working out that muscle? What are you training on? What are you focusing on?  Or are you focusing on the mission and the goals for it? So which element are you really focusing on? And I even, I say all these things in a shorter way when somebody asks me about book recommendations. Same thing, Dan, what book do you recommend? It's almost like what workout, what exercise do you recommend? Like, well, what are you trying to achieve? What are you trying to do? And how I look at books, it's the same way like nutrition. I'm trying to lose weight. Well, then I'm gonna eat a certain way. I'm trying to put on weight. Well, then I'm gonna eat a very different way. You know, I'm trying to do this or that. That's gonna change what, so I'll ask individuals, like, oh, what book do you recommend? What are you trying to do? What direction are you moving in? And specifically, and then what books will help support you with potentially that knowledge gap? And then going back to like, hey, we're closing that knowledge gap and now how do you implement that in your doing? And if there's something missing there through journaling now, so now that we have framework that we can go through and framework helps ask us, we can ask ourselves questions. And as we put ourselves through the framework, as we learn to ask ourselves questions, we can now show up as a better, what coach, mentor, teacher to our team, which then supports, and now we're bringing everyone up, which then what? As I support myself, I'm now, as I get stronger, I'm now better able to support my teammates and as they get stronger, we're now what? More ready to accomplish a mission task or goal. So upon self-reflection, having these experiences in combat and then the academy and now in your graduate school work, is there something you wish you knew around leadership, wish had been part of the military training at the very start looking back and speaking to Dan, the young leader, ready and able to take action, committed that you feel right now that you've learned through all of this work that, man, if I had it in my 20s would have been really impactful.  It's almost like a performance paradox. Like being hard on yourself, get you to certain places and get you in certain rooms. And it also can be crushing at the same time. And so I would look to still work hard and maybe that internal dialogue of things that I said to myself, I'd be a little gentler with myself around mistakes or certain expectations. I think I still could have met all those performance markers being a little bit kinder to myself. Yeah, I feel that's always the challenge looking back is this need to feel like it's a sprint when in actuality it's a marathon. For me it was pedal to the metal at all times and it only drove me right into a wall. So it's important to let up and know that you have that time. And we see it with a lot of our younger clients who are like, I want everything right now. Okay, well, we got a lot of work to do. But I'm glad you have that one. That one will be the motivation that we need. And now later in life, right? So running that formula all the way through life and now later in life when I do slow down what pops up in my head, oh, you're being lazy.  So it's just like, oh, but just noticing those thoughts and not shutting those thoughts out because those thoughts are communicating something. So just like if I have discomfort in my body, my body's communicating something with me or if my intuitive state or my intuition, whatever, it's all communicating. So those thoughts are communicating something with me and to take note of it. And then almost like a self-negotiation. So going back to goals, I want to get up at 4 a.m. but I'm getting up at 6 a.m. right now. Well, let's split the difference and let's try 5 a.m. Let's start there and see how that goes. So it's the same thing with, hey, I'm gonna take a break. Okay, well, maybe a break is 20 minutes. Maybe it's an hour. Maybe it's taking 30 minutes in the morning. Maybe it's taking something later in the afternoon. So almost like finding that balance point and that goes back to calibration. So how it shows up for me will be different than how it shows up for you guys.   The calibration piece to me feels essential in this conversation as we talk around leadership, but also very difficult to do on your own. I feel that calibration is often deeply influenced in a more productive manner when there's a third party perspective or multiple perspectives to help you check yourself to get to that level of calibration. Often left to our own devices, we can miscalibrate completely and get off course. That leads me to the last piece that I'd love to touch on with leadership, which is feedback. Both the receiving of feedback as a leader, being open to the feedback, and then if you can touch on giving feedback because in a leadership role, feedback is such an important piece. And I find that oftentimes we don't have good frameworks for them. We have a lot of myths and misconceptions around feedback. I think in short, I'll say two things about feedback is how I like to give feedback is appreciations and recommendations. And I'll ask permission. So that's kind of a different thing too that I've also, a lot of these techniques have come from leadership coaching, have come from books and have come from different things. As a leadership coach, one thing that I got from the program that I implement as a coach too, a lot that I used to never do is ask permission. And I'll ask somebody, like, hey, are you okay if I share my thoughts? Cause they may not want to hear my thoughts and it's okay. And I'll tell them, like, hey, it's okay if you say no. Obviously I'll read it. Sometimes it's just somebody venting. And I'll just like, oh, I can tell that, that they just, they don't want to hear my opinion. They just need support. They don't need me to make a statement. They actually need me to ask a question just to like, if they're venting, they're trying to get rid of something. So I'm gonna ask them a question so they can to help like pull the rest of it out for them.  There's a lot of holding space is something I hear all the time. So like holding space. So it's hard on this camera. So if I'm holding space for somebody, if I'm holding somebody, what people don't realize is if, if I go in my backyard, I have woods in the backyard. And if I put logs and I carry logs and I hold logs and then I set the logs down, I'm probably gonna have what on my suit. I'm gonna have some residue. I'm gonna have some dirt.  And so now how do you, how do you move that residue and how do you clean yourself or brush yourself off? So going back to self, like learning those self care techniques to, hey, I just held space for somebody. And in that conversation, it went to a very dark place. I work with clients that really have deep struggles with certain things in life. And so I'll hold that space. And then as I set it down and learning even how to set it down and not carry it with you, because there's a lot of folks who are very empathetic and they'll think and feel and carry those thoughts and emotions of that individual after the conversation. It's not yours for you to think or carry about, at least in my opinion. So I set it down. It's their stuff. I helped them in that moment or in those moments. So I set it down. And then how do you clean yourself and move yourself from it? And so using different techniques. If it's a real heavy conversation, like I'll go take a shower, I'll change my clothes, like I'll physically like do something physical, even for like law enforcement or military, if they're transitioning in certain environments, changing their clothes is a big one.   Like nobody wants that at home. So change that.  And so just doing something physical or having a behavior to help shift in there. Sorry, I know I digress from feedback, but in feedback, I'll ask permission first. Like, hey, if you're okay with, hey, I have some thoughts, are you okay if I share those thoughts? And they'll be like, yeah, I'd love to hear them. Okay, I actually really appreciated how you guys engaged me in this conversation. I appreciated how you guys allowed me to finish my thoughts and you guys didn't interrupt. I appreciate it. So I'll share appreciations and I'll share like, hey, next time, something to think about, a recommendation is maybe this. And so it's very soft, very gentle. And typically it's not for me and some people are like, be direct, be this and this is my culture. And for some cultures, that's, hey, if that's your culture and if that's what you choose to do, I would only ask, is it for you or is it for the individual? So if I'm making recommendations, it's not for me, it's for the individual. And so now reading that relationship and making the best decision, what is best for them to now receive? And it may be direct or it may be indirect. I typically go, my style is more indirect. That's softer style. And with people with either sensitive egos or big egos, whatever their ego status is, I've noticed this style works for me. I love that. And I think it's important that you do have those behaviors and actions to change the context, whether it's heavy, whether it's a role you have at work, the shower that do something physical to take that hat off, that ritual to put yourself in a different place, mentally, physically, spiritually, for the conversations you'd like to have to not bring the work home, to not bring the negative feedback home is so important. Thank you so much for coming on the show and sharing all these great leadership perspectives and frameworks with us. Johnny and I love asking every guest what their X factor is. What do you believe makes you unique and extraordinary, Dan? My movement in the in-between space. So connecting, knowing to doing, that's a space that I like to hang out in. And so this doctrine of education that I'm pursuing right now, both looking at trauma and looking at resilience, moving in that in-between space, connecting those, just trying to make things better. Same thing, journaling has always been around. So now taking something unique to journaling, adding the prompts, helping, really helping people on their journey and then connecting things.  Well, thank you. I feel you did a great job of that here with us today. We really appreciate it. Where can our audience find out more about the work you do, Dan? I have a website. So my company is team3LX.com and it's 3LX is Roman numeral, 3LX is 60. So 360, I do like to take a holistic approach for starting out. There's a lot of positivity and specificity. So even in my research right now, there's things that fall through the holistic gaps and there's certain things. If you wanna run a marathon, you should probably do marathon-specific work. So resilience to what? So if we know certain things within an organization that we need to be resilient to, a holistic approach is a great start and then we can probably make it specific to the organization. So a lot of the work that I'm doing is there. So team3LX.com and then I'm on Instagram. I'm still trying to figure out my identity on social media and what to post and how to share a lot of this stuff. But it's, I think it's dan.p.luna on Instagram. Great, we'll link it in the show notes. Thank you, Dan. Right on. AJ, Johnny, thank you. It was a real, real pleasure, especially being like one of the first podcasts I ever listened to. So it was really cool being on here. So thank you. Thank you for stopping by.


Let's talk about this Tyree kill thing. Okay. He was really chirpy yesterday. This is what he had to say on 810 sports Take a listen She's keen on when the Miami Dolphins come to arrow here stadium this year. Guess what we gonna do? Guess what we gonna do? I hate to talk the peace sign against y'all.  I'm gonna be y'all worse enemy that day. I'm gonna be y'all worse enemy that day The signals I know every signal y'all got It's April It's too early for it, but I love it, you know, this is what he does ever since he came down here to Miami He's just been stirring the pot every time it seemed like he picks up a mic and it's early in Tyree Hey, it's been a lot of talk about trash talk lately with Andrew Reese and Caitlyn Clark And if you're gonna talk trash back it up Tyree has backed it up since he stepped foot on that on that football field And it's your former teammate. It means your former team. So it is gonna mean a little more for Tyree I'm looking forward to it now. It is Patrick Mahomes and the defending Super Bowl champs on the other side So I'm somewhat worried about that but we didn't train for Jayla Ramsey to show up that deep as a backfield It's a big heart. So I like it. I like the trash talk. I love it Actually, it's true, you know, it's funny that peace sign is talking about the Usain Bolt And somehow that isn't on Twitter for it trending and nobody has opinions on that It's just looking looked at as good competitive Confidence in oneself. It's so interesting how that happens quickly before we let you go What is the talk to me about the Chiefs DB chain right now with that? Chiefs DB. I didn't see that. No, I'm saying like you're Give me Are they texting each other treating each other or are they just enjoying their off-season? They don't care Yes, it's April. Who knows. I'm probably in Cancun and Costa Rica somewhere right now They don't care what Tyree Hill is saying on 810 sports right now in April 6th It's probably turning to the Masters right now Actually, but they're not saying anything, but when Tyree Hill shows up, you better believe it's gonna be a full game plan It's gonna be a little different than every other week when the cheetahs out there


Can you let me know when it's okay, June? Yes, it has already started. Go ahead. Perfect. All right, so welcome again. My name is Karim. I'm part of the executive committee of the R-Validation Hub. And in particular, I lead the regulatory R repository work stream. Today we'll be wrapping up 2023 with updates of what happened during this year in the R-Validation Hub. Then we'll do a deep dive. I can show you the agenda. Then we'll do a deep dive and focus on the regulatory R repository work stream. And then we'll open up for discussion. And there we go. So while we were waiting for people to join, I asked you to share in the chat who you are, where you're from, who are you joining us today, and what your cat mood is. I cannot see your comments. So it's just for you guys to read between yourselves and for everybody to know who's here. Does anybody want to share your cat mood today before we kick in? This is a quick icebreaker. If not, it's fine. We can continue, but for those who are excited for seconds. Okay. I think a bunch of people are like in between five and six cat mood wise. Perfect. Thanks Julian. Yeah, I'm rather five. It's six p.m. here in Switzerland. This is my last meeting of the day and then I'll go and have dinner. So finishing up the day. Hope everybody's okay. So looking back on 2023, I tried to summarize a couple of events that happened in chronologically. We welcomed Doug as our new chair. Took you, taking over from Andy. So congrats again, Doug. And immediately Doug took a couple of actions. For example, creating official communication work stream. I know there was a communication activities already before, but this was some work done around consolidating the communication activity in a dedicated communications work stream led by Julian with, for example, Aaron, there's Jackson, Anuja, and some more people. So that is something that Doug kicked off when he started. There is also a new executive committee composed of the following people here. So Doug is chairing us from the art consortium since the art validation hub is supported by the art consortium. Joe on the call, we heard from you. We heard you Joe before, Prietham. I don't know if I can see, I cannot see who was on the call, but Prietham also is part of the committee. We have Julian leading the communication work stream myself, leading the repository work stream. Eric and the risk metric and Aaron with the risk assessments work stream. So some changes in the executive committee since this year. A couple of events we participated in because it's good to remind ourselves also our presence because part of what we do is about, is to make an impact. What we do is to make an impact for our community. So part of our job is also to reach out and communicate and speak about what we do. So I here have the use R, it was renamed R in R Basel this year that was on the Rosh campus. That was in July and there I was there to represent the regulatory R repository work stream. I know we also had members of the risk metric team representing the R validation hub. So that was one of the events. We had the Posit R and Pharma Roundtable in Chicago organized by Posit just before the R and Pharma conference where Doug was there to lead a workshop. So it was not a conference this time, it was a workshop. So I remember Doug organizing workshop is different than simply giving a talk, there's more logistics to it involving sticky notes, which is honestly sometimes quite a nightmare, but it was a very interesting, so well led workshop and with interesting output from it. So that was another one. And finally we were present at the R and Pharma conference in October where Julianne and I represented the R validation hub and the different work streams. So I think we can say we've been present at the main events around R this year again, I guess. So this is I think a sign, a healthy sign of us having content to share with our community. Otherwise we would not be invited because we're not invited just for saying we're not invited by defaults to those events and conferences, right? So we propose topics and usually there's a committee of each conference selecting who goes and who doesn't. So based on the interest of the community. So that was something else in 2023. And now I'll do a little work stream updates very, not very briefly, but I will do a quick, still quick because then we'll have a really deep dive into one of the work streams that I would like to give you an overview of all the work streams. So as a work stream, I put here still the R consortium. Even though it's not a work stream per se, but R consortium has a strong presence in the R validation hub. So the first news that happened this year is that in August, the R consortium announced that Joe had been appointed to the position of executive director reporting to the board of directors. That was a new position that was created in the R consortium to ensure that there was going to be long-term and consistent oversight of day-to-day activity. And Joe had demonstrated this in the past years. So again, Joe, congratulations for this and it's worth reiterating this news that happened this year. Also the R validation hub worked in bringing its communication activities under the umbrella of the R consortium. So for this, we thank, of course, our communication work streams led by Julian, but also on the R consortium side, Jesse, Aelina, Agedda, Maya, and others from the R consortium marketing committee who creates great material for our comms. So that's another update of this year. And worth mentioning also that the R validation hub and the R consortium started, still early days, it started a collaboration between two of its work streams. So the regulatory R repository work stream in the R validation hub and the R consortium repository working group that are, even though they have repository in their title, both of them, they do focus on different types of repositories. But for the moment, basically our R validation of work stream is attending the R consortium repository's working group to make sure that there is a link between those two topics. So that's an example of a collaboration that started this year. An overview of the different work streams. So on the communication side, as we said, so there was an effort to consolidate this work stream and to put several people together to lead this. Examples of the topics that were tackled this year are how to structure the R adoption series seminar, how to structure the advertisement efforts, how to make the Canvas project more visible on our websites and revisiting our websites as an ongoing theme as there are always improvements to do. So we thank the communication work streams, work stream members. I skipped the repositories work stream here because I will deep dive into it in a couple of slides. On the risk metric side that by Eric, so for those, the few I hope who don't know risk metric, it's a framework to quantify in our package risk of use, right? And developments in the risk metrics include, for example, creating prettier outputs, ease of use, have more complete metrics, for example, for as many package sources as possible. Modular additions, for example, to allow custom assessments based on community package, for example, as Oyster, SRR package test, for example, and also focusing on the metrics and scoring. So making custom waiting more robust and convenient. So thanks Eric for this work this year and the team on the risk assessment side, led by Aaron. So risk assessment and risk metric work together as risk assessment is actually an R package that contains a shiny front end to improve the use of risk metric. And actually it was mentioned in the RN Pharma conference, but for those who didn't attend this app, risk assessment received an award, the title of best app at the SHINee conference 2023. So that's an achievement worth celebrating. For example, things that have been going on in this work stream are improved report builder, database view, better dependency inspection, automatic risk assessment, and many more. So congrats to Aaron for this work this year. And finally, a new one. It's one of the kicked off recently, risk score led by Aaron as well. So it's a data package for cataloging risk metric results across public repositories. Little disclaimer right now, the risk score is in the POC stage. So it's not fully operational, but since it was kicked off this year, I wanted to give also an update on this one. Before I have this little gif here to thank all of the work streams, I'm aware that I gave a very high level updates of all the work streams. So this is why before I close this section and go to the deep dive of the regulatory, our repository work stream, I would like to open the floor to my colleagues who are on the call and the executive committee or those of you who lead those work streams if you'd like to add anything that I didn't mention that you'd like to share with the community. You can take a couple of minutes to do this now if you would like. Does anybody want to add anything? Now's the moment. Sure, this is Aaron. I help lead the risk assessment and risk score projects. So I'll just say a few words. Risk score. To say that I lead this is, I use the word lead gently, but I did push some initial commits to sort of get the repo started. But yeah, I think we're currently soliciting interested parties who want to contribute on this project. But yeah, essentially like Colleen said, just a way of capturing risk metric, scores and outputs for all packages on Cran. So that's really exciting work. But yeah, just it's really in a tentative experimental stage right now. So if anyone wants to contribute, I'll share a link in the chat to the repo if you want to check it out. And yeah, for the risk assessment application, I would just say a couple of the big things is our friends over at GSK contributed some code recently that I think has really launched the application forward a ton in terms of the way we can manually review package risks. So if you haven't seen those, I think we pushed a release, I think it was just a month ago. It wasn't very long. It was around the time our environment was happening. And so you can now actually explore files for any package. So we actually are downloading like the source file so that you can explore like the description file or the test files, whatever you want to view. It's really helpful. And then we also have a function explorer that GSK also contributed. That's really awesome. For any exported function, you can view the source code, you can view the rendered help docs and you can even view the test for those functions. So it's just a really standardized way to sort of evaluate your package or the package that you're reviewing in just a consistent way for the whole organization. So that's my two minute plug for some of the recent features. So go check them out. I'll share some links in the chat so that you can follow up with if you have any interest. Thanks. Aaron. Anybody else? Julian, Doug, Eric, can I continue? Maybe I'll just add a small note about the communications work stream. So sorry I don't have video today. I'm still kind of recovering from about with COVID.  But one thing about the communications work stream is that they're still kind of finding their footing and starting to think about revamping some of our traditional communication streams like the website and how we post content. But they're also thinking about how we want the community to look in the future. And part of that is building kind of more like grassroots communities at organizations and being more connected with those. So how do we give materials to like pharmaceutical partners so that they can build smaller like local communities within their own organizations and how do we kind of like feed that information back to a broader community so that there's more kind of knowledge sharing happening across companies. So I think that's a really cool idea. Another way of looking at how we like broaden our communities and build more inroads across our organizations. So just be on the lookout for that kind of material if that's something that you're looking to embed more within your organizations. Thank you, Doug. Wish you a good recovery. Yeah. Yeah, also, if you want the other validation hub to have like within like if you have an internal user group and you want to have a conversation internally in your company, like just feel free to reach out to us. We are definitely looking for opportunities like this in particular next year. So and if you want to help out, there's a ton of work to do. So that's also always an opportunity to help out with this initiative. So Eric, any word if not we'll continue. All right. Thank you guys for complimenting this summary of the 2023. So we'll now do a deep dive of the particular work stream called the regulatory R-REPEL simply because we, it's a new work stream this year and we didn't have the opportunity to share further information in the community meeting before. So we decided to dedicate a bit of time in this one. And then I'll open up for discussion. So the regulatory R-REPEL, hopefully most of you have heard about it. So the idea is that we build a repository of R packages that are considered validated. So it's a centralized place. As it says here, open dynamic cross industry repository of R packages are validated, validated R packages. So to distinguish ourselves from CRAN, it's not another CRAN that we're trying to build. This is what the R Consortium's working group called Repositories is looking at partially. But here it's really another need that we're trying to meet. It's right now the fact that every pharmaceutical company is setting up their own validation process and team to meet the needs of our industry, which is regulated. And when we submit results and outputs of our internal analysis based on our clinical trial data to a health authority, we need to show that this code was validated. So this is a new challenge, new since a couple of years now, but new challenge since the pharmaceutical industry started to use open source programming languages like R for its clinical trial data, clinical trial data analysis, compared to SAS in the past, which was a licensed software. So we didn't have this question of validation. At least it was not a concern for sponsored companies, pharmaceutical companies. It was for the company SAS. Anyway, so this is not a new topic. But since we are hearing more and more about the need for validation, each company is developing its own solution. And we thought that it would be a good solution to have a public centralized place where to avoid duplicate work, basically, in each individual company. So we have taken time to reach out to different stakeholders in the industry. I write here, validation leads, health authorities also, developers, of course, people who maintain also infrastructure and companies, and so on. And we basically concluded that this would be a solution that is of interest to all the stakeholders. So this is why we created this work stream at the beginning of the year. Basically, how we got here is that our validation have had its white paper widely adopted. But implementing it was a bit difficult, as each company has its slight variation. Sharing software, in this case, with health authorities, is sometimes a challenge. They are overwhelmed by technical inconsistencies. If each sponsor company shares results with a different version of R, a different environment, which was developed then, the health authorities, when they need to replicate our work, our results, they then face some challenges. And as is written here in this last bullet point, so we felt that the most productive path forward was a shared ecosystem, a shared and central place, hence the spreadsheet. So this year, we've been working on a couple of things. So it is, of course, important to build some type of consensus around what is a package, evaluated package, what does it mean to evaluate a package, how do we distribute those packages. So we've split the work in different parts. So we started with stakeholder engagement. So we interviewed a lot of people, stakeholders, as I mentioned before, roles like validation leads, health authorities, developers, system admins, and so on, to make sure we were speaking about the same thing and actually grasping, sensing a need that was actually the case. Then we split the work into several proofs of concepts. And then we started developing the proofs of concepts. So it was all of this is voluntary work. I have at the end a list of all the members. You will see how well we are represented across the industry. But it's really incredible we were able to achieve with voluntary work. And just it was great to see from the start the confirmation of our initial assessment that this was needed. So what a solution looks like is, first of all, to have some standard and public assessment of packages. This is probably the most difficult piece to agree on. But because each company, as I said, has a different way of validating, we've seen it with the case studies at the R-Validation Hub together last year, asking Roche, how do you guys do validation? And the artists and so on. So each have a different and slightly different approach. And agreeing on the one that fits all, at least to start with, is a difficulty, but is needed. Thinking about how we communicate, how we contribute to the repository, how we then consume from the repository, how bugs can be solved, how this would be maintained, and so on are pieces of work that are key to solving that we've been tackling or come to this unit. So the proposal so far is not concrete, but we know that we need a central place where there is some type of assessment of packages. That's the source of those packages are probably going to come from a source like CRAN or our universe where we've been exploring also other avenues, that there will be a validation pipeline, that there will be also user interface, and some type of forum where people can discuss the assessment of packages and so on. Another way of representing the future solution is like this. It's very less elegant than this one, but it still speaks, I think, where we have in the middle the repository setup, what we call, I wrote here in the yellow sticky notes, as one of our pilots, the possible repo setups. So what are the pieces of the puzzle that we need to put together so that this so-called repository works, and it includes a repository, a reference image against which we validate a UI, a user interface, and some type of database of validation metadata that we did have here, with contributors on the left contributing to the repository, consumers on the other end, obviously, consuming the validated packages, maintainers, or we hope this would be as automated as possible, so it could be a robot's maintainer, or a robot at least, an automated way of maintaining the validated state of those packages, and something that would look like a governance board, making sure that the repository still meets the need of its users. That's all the different stakeholders and people engaging with the repository can do so, and that we still meet the needs that it's meant to do. A couple of items that we have thought about is, so I mentioned this reference image. This is something that we ended up concluding that we needed, otherwise we could have probably several reference images, but at least we need to define, at the beginning, a golden image or a reference image that we validated against, because a package is only considered validating in a specific environment. We cannot say package X and version XYZ is validated. It has to be validated in a specific environment. So this would be defined here with the reference image. Obviously, the part of the defining requirements and testing and matching those automatically is part of the validation. So this is also another point of discussion. Expectations of public communication, so how do we, communication channels for the different stakeholders of this repository, what do we use, industry standards, communication channels is also another topic of discussion, and we, so here edge cases are better handled by transferring community engagement, as this would be a central place with validated code. We want to have, as I mentioned before, a place where the community can contribute and discuss, this cannot be a black box, so it needs to be transparent, otherwise it will not work. And I'll finish with this one. So there are two last updates I wanted to share. So first of all, the one, the second one, I'll start with the second bullet point. We have put together a benefits of joining us PDF. It's a one pager where we summarize the reasons why we think it's important to join, and this is a document that we ended up creating because in our members meeting all hands, you were asked, well, I have my day-to-day job, so if I were to ask my manager to join this working group, I need to explain why it's important, and so on. So we decided to put this one pager together so that anybody can pick it up, share with your colleagues in your company, or share it with other people you think are talented and should join the working group. Hopefully that's useful. We'll share the slides after the call, so you'll have this afterwards. And the last piece of update I wanted to share is that we have last month submitted a grant proposal to get a bit of funding to my idea as lead, and I think that the rest of the team also was interested to see if we could get some small amount of money to hire a couple of people full-time dedicated to building the repository. Since, as some of you know, when you have work done purely on volunteer work, it can be sometimes not necessarily very constant. People give the time they can, so when you meet on a weekly basis, you don't necessarily have everybody there all the time, so sometimes it's a challenge. It's also, I think, important to say the truth, to bring people together and to focus on one vision. But this time we were, so the funding was not granted, but it was a good opportunity to get some feedback on what was expected so that we are successful in the next round. And also something I will always remember, the discussion we had a couple of weeks ago with the executive committee. And Joe told me, well, there's a lot of you can do with volunteer work, and that's an opportunity to make it grow, make the momentum, surf on the momentum, create the momentum, and see what other working groups are doing based on also only volunteer work, and a lot can be done. So this kind of reassured me, because at the same time, the infrastructure steering committee was actually in support of our work stream, but needs a little bit more concrete results. So this will be working on for the next round of grant proposal in March. So yeah, this is what I have for the, I just want to thank all the repository members. And as you can see here, I'm really proud of the variety of companies and the various stakeholders we have on this work stream. We have sponsor companies, as you can see here, a variety of them. We have CROs also represented. We have health authorities with the FDA, and we're working on getting somebody from the European medical authority. And we have also service providers who are key to collaborate with since they own, they own sometimes part of the infrastructure we that are part of the R ecosystem that we would like to base ourselves on. And some of them also have some expertise that we want to leverage and that we count on to make this working group successful. So really another little fun gif here for Christmas Eve, Christmas Eve gif. But really to say to all of you, I hope some of you, I think, I know most of you were going to attend this call today. So thank you for your time. We couldn't be where we are today without all of you. So it's an opportunity for me to thank you in front of this audience. Now it's the time to open for discussion. We have about 20 minutes. Thanks for listening. And yeah, I'll open up for questions, discussions. And Julian, if there are some questions in the chat, then I cannot see. Not at this point, but it doesn't mean that people cannot come up with questions right now. Great presentation, Julian. Thank you. Quick question. Are you going to provide these slides to us so that we can maybe share internally with our, in our institutions, that's one question. Yes, we will. Julian, you know better than me, how and where we share those slides, but of course we will. Yeah, I can totally, like if you send them all, send them out to everyone who has been included and it's called, we can send it out to the list. Can you go to your, I think it's the MVP, like the, it's the slide 15, I think? That's what you have, some proposals so far. Question, documentation is critical for any implementation in anywhere, right? So if we were to use this documentation of everything that is being done, yes, it's part of it, but it's, I just don't see it pointed out there as part of the MVP or the proposal. So I, in my opinion, it should be part of it, right? Because that's how our IT colleagues and everybody else is going to see and consume. Granted, this can be done through a, through a communication site, you know, external for the, but even if it's that, there has to be a part of it that is, that is documentation. If it's not, I just don't see it there. So that's what I'm saying. It's a good point. Yes, we, it's not represented here, but it's obviously part of the outputs of this. I take notes to add it in the, in this diagram. I'll just maybe touch on that a little bit. So I know that like the documentation has been, I know it's part of our early discussion. And I think part of the challenge is that the way that people document and the context that they want to add specifically to that documentation is very company specific sometimes. But what we can definitely provide and what we've focused on so far is like the data that would make its way into the documentation. So how do we, how can we support it? So you have to do as little footwork as possible to collect all the, all the underlying qualities of a package that you would then want to document. And from there, maybe there's this intermediate where things still end up being a little redundant across companies, but that gives us the, the foothold to then start establishing this like common template of what that documentation could look like. I know that we've had discussions about having a, template like that from the start. So it's just a matter of like, where do we put that effort as we're driving things forward? And I think right now the primary questions are around like the technical components. There's not really that many open questions about how do we pull it into a report when it's all there. So if we're thinking about what things we need to, what questions we need to answer to know that this is viable, most of those end up being, I think more the technicalities of hosting the software and pulling those in a more standard way, the metadata that we'd want to pull into a report. So yeah, unfortunately you can't foil the whole ocean all at once, but that's definitely in the backs of our minds is something that eventually we need to cross that bridge. Thank you, Dr. Thur. Any other comments? I mean, this isn't a perfect common opportunity to bring up vicious opportunities, ideas without having to, having to join another working group, right? So use that opportunity. Maybe I will just toss up a question while we're on this slide. So I see the governance board called out here and I'm wondering what kinds of decisions you see them playing a prominent role trying to address. I think in any project like the one we're trying to build like I imagine there's one for CRAN and one for our universe and one for GitHub, GitHub, whatever tool and system you put together by experience, I think it's important to have a group of people who are not with their head in the wheel, that makes sense, I don't know if that's a French expression, but some people who have a more helicopter view of what's happening with the system in question or in this case the repository to make sure that things are happening as planned, that the purpose in this case, the repository is still up to date, that if there's a new need arising that this governance board brings it back to the people maintaining the repository or I think it's important to have always a group of people who are independent from the day-to-day activities and who are able to give feedback and improve the tool during its life span. So that would be my mission for us. Let me add to that. So it's particularly important to jump off based on what Kamin just said. The repository, it also has to have governance that's transparent and open. So any endeavor like this has to be in compliance with the antitrust laws. We have to make sure that there are, let's say democratic and open mechanisms for the governance of which packages are included in the repository, how people interact. So this is important concern for the art consortium and I'm very pleased to see that people are thinking in this direction. Yeah, thanks, Carol. I think it's not a, it would not be an alien to have this, it's actually quite a good practice. Okay. So how often do you lead with the, just if anyone is in the background who wants to join, how often do you mean and do you have like a smaller circle, bigger circle, like how does the organization work? Yeah, good question, Julian. So we have a core team and an extended team. So the core team members are the ones with the little asterisks here on the screen. So it's a smaller group compared to the extended team, obviously, but still a big team. I think we're about 12 to 15 people meeting weekly every Thursday afternoon, Swiss time, morning, Eastern time. And the, on a weekly basis for 45 minutes, at the moment, what we do is we give updates on the yellow sticky notes here, which we called our themes. So there's a group focusing on the validation pipeline, another one on the possible repository setups, and another one on the reference image. So, and then we'll put those three pieces of the puzzle together. So that was, that's what we're doing on a weekly basis. There's a GitHub issue at RiftDoug, you could share in the chat while I speak and you can dig it up. There's an issue where you can sign up in one of the themes. So if any of you wants to join. So this is for the core team. And then we have an extended team that meets approximately once a quarter. So the extended team consists of people who cannot put one hour per week in this work group who are interested to follow its development. So this is similar, we do similar sessions to this one where we update our community on what's happening. And I'll add that the only distinction between the core team and the extended team is like time commitment that people are willing to offer. So we have a bunch of people, our core team that attends regularly and are consistently committed to the working group.  So they prefer to kind of sit kind of on call, you could say. So we kind of tap into those resources when we have a specific task that we're aiming to deliver. And we'll see then who's willing, who has kind of bandwidth at that particular time to jump onto it. So to paint on your availability, if this is something that you're interested in participating in, you're welcome to participate in kind of either capacity. Yeah. Thank you, Doug. I think in the meeting invitation, you will find my email address. So if she would like, if it's too complicated to find the GitHub page of this work stream and to find where you can sign up, because there is a place where you can sign up, but so free to just email me and I will give you, I'll make sure you have an onboarding session and then you can let us know which team you wanna participate in or if you just wanna be part of the extended team, which is also fine because the extended team for me is almost as important as the core team because it shows the support we have from different companies. So I trust that everybody gives the time they can and if you're extended member, it's also fine. I have posted the email address also in the chat. So there are no other questions. This is the last call. Okay, seven seconds of silence is about the time that people get. So I appreciate you all signing in and taking this time out of your day and evening and thank you, Colleen, for presenting. And I think it's time to say happy end of the year and let's see where we pick up next year. We'll definitely look out for those emails that are coming from our Validation Hub email list and if you're not on it yet, I highly recommend you to get onto it. We will definitely make more use of it as the communication works from all the pieces together. Having said that, happy end of the year. Enjoy your end of November, December and see you sometime in January, February. Yes, thank you all. Same to you all. Take care, bye bye. Thanks everyone. Thank you. Bye bye.


There's a great lumbering bus. How about that? I said, what I think we ought to do is tell them that we, you know, since we can't get out of our taxes and we can't get out of our debt, we should tell them we don't want them spending any money on anything but the Northwest Corridor train and Longmont could stand up its own public transit local services through public-private partnerships and we could do a better job than they are because they're doing the worst of all possible things and starving us in terms of capital investment on the local stuff too.  I just want robust bus service in town. That's all I want. Yeah. Well, I want robust service. I want it to be every ten minutes. I want it to be electric and I want it to be small so we don't have those great lumbering diesel dinosaurs roaring around our town getting in the way of everything else and not happening enough to help anybody with their lives. It is transit of last resort and we don't need that at all. And I think it's great that there are no fares. It's the right idea at least that some people should pay no fare. But nothing else is right about that bus service. Longmont understands its own needs and could design its own bus service and fund it. So how could we do that? First of all, we do have the fare box subsidy which wouldn't need to go away because it's not part of the PRPA taxes, right? The other thing is we have progressive businesses who know how difficult housing is in Longmont but want their employees to be able to get to work on time and they understand that good transit is part of having good employees because they aren't having to spend on transit.  And it makes them happy to live here. They get time back, you know? If you have good public transit that doesn't take us twice as long as driving then your employees can read the newspaper or study up for their presentation that they have to give that after. There's a great lumbering bus. How about that? So we could have businesses that would have subscriptions to the service in exchange for getting a say in what the route would be so that their business would be on the route. Subscriptions from real estate developers that want to build high density apartment buildings with not enough on-site parking because we don't have enough space for all those cars, right? So I already have a developer who wants to build two high density apartment buildings and give every tenant an eco pass so that they wouldn't have to pay bus fare and both of those apartments are right on the existing RTD routes so that guy instead of buying eco passes he would buy a subscription to the Longmont Transit Service and I said in 2018 that we need the transit service for our elders to be much more accessible because at that time you had to make an appointment and you picked up by via or accessoride a full week in advance. Sometimes you can't even schedule your doctor's appointments that far in advance especially if you're on Medicaid so it made life quite difficult for our seniors who lack independent mobility and that was the time when Uber and Lyft were emerging and I made a motion in front of the city council that said I want to renegotiate this contract, I want to put it out to bid and I want same day service just in time service for our elders because that's what they need and nothing short of that meets their need and the city did that.