J. Cole has shocked the world and we gotta talk about it. What are we talking about for those who somehow have been under a rock and missed it? Kendrick Lamar hopped on Future's track, well his album, and he dissed J. Cole and Drake. You know what happened after that? J. Cole actually responded, but after he responded, everybody knew that his festival, the Dreamville Festival's coming, right? It could be like a Jay-Z at the, what is it, the Hot 97 Festival, whatever, back in the early 2000s moment, get on the stage and then say, what is it?  I'm all about a dollar. Like one of those type of epic moments that can stand to test the time in hip hop. What is J. Cole gonna do? The world is watching. You know what J. Cole did? J. Cole said, you know what? My bad y'all, like I'm gonna pick up my spade and the world is like, space, whoa, run space, stick bro, you can't pull out. J. Cole pulled out. J. Cole pulled out. But this is actually, I don't know, maybe changing the game. I think it's a lot to talk about because some people are saying it's a career killer, like legit career killer. He can't be considered one of the greatest of all time. Some people think his brand is tarnished in general. So we're gonna talk about all of those things from the scientific, what we think as professional marketers, branding people. That side of it, but we also gotta talk about just straight up like hip hop. What is this thing about and what's next? Like what are our predictions? So first and foremost, man, J. Corey. Hello. When J. Cole, went out J. Cole responded. What did you think? Did you hear the whole album or did you just hear seven minutes? Yeah, I've heard seven minutes and maybe like two other songs. So I haven't got a chance to get through the whole thing first. But I mean, when I heard the apology, I don't know man, I thought it was interesting because we almost came in and made a video title. Is this a rollout, you know what I'm saying? There were a lot of pieces that leading up to this point made me feel very strongly about the fact that like, oh, this is a rollout for somebody, for something, you know what I'm saying? Then J. Cole will drop the album. I was like, oh, this is perfect. This is a J. Cole rollout, right? But the apology made me feel like, no, this is a real thing, you know what I'm saying? This is a real moment. One, just because of, I understand where the potential of thinking he tarnished his reputation can come from if you're thinking about what you may perceive to be J. Cole's core audience. And I'll say that strongly, you know what I'm saying? What you perceive to be, because I think what people think the core J. Cole fan base looks like is not what the core J. Cole fan base really looks like at this point, you know what I'm saying? So I can understand where the sentiment comes from. Because now I'm thinking like, okay, if we were to assume it was still a part of the rollout, let's say I'm wrong, it's like, man, what's the play here, right? No one would see that coming. So it would, you know what I'm saying? Create a lot of headlines. And I don't know, man, this is just maybe where I'm a skeptic. Like I said, like it's just, it's hard for me to just point at things that this is a game plan. And if it is a game plan, I came up with it as a genius. You know what I'm saying? I was like, bro, J. Cole did the impossible. That's why he legit shocked the world. People didn't know that this was an option. Like that's why the world is going. That's a great point, yeah. Like nobody knew you could do that. Like for real, it's like he picked up the spade. He said, yo, man, like, nah, I'm good. I don't want to do this. I don't want to play anymore. I'm taking my ball home. And they're like, yo, bro, come on now. So to me, I understand people are highly disappointed.  But to me, when you really look at it, like you said, J. Cole's core fan base now is not the core fan base that he started with. Not that some of those people haven't come with them, right? But it's not the type of sentiment. Like some people might have come in for one thing, but then they evolve with them and they're looking at the world a new way. Even if you're disappointed with them, I'll tell you exactly why. Like it's not a career killer. It's not going to ruin his brand.  They want hip hop to be different. Hip hop is not a monolith. There's a variety of artists. So we can't keep on falling on this. Like we argument that, oh, hip hop is all competitive. A rap, it has to be competitive. Like there's so many other types. Like we don't want to see Missy Elliott in a rap battle necessarily. Like we're not calling for that. Everybody doesn't have to be in that type of competition. And like you still got battle raps for that. Like that is the pure raw space of that.  But for J. Cole in particular, like, what are you hearing people say? A lot of people are saying, oh, well, it's on brand, but like, nah, man, this is weak. You can't pull out. This is for the sport of it. It's not even that serious, right? They're saying that. Or they're saying, I respect J. Cole as a man, but nah, man, you gotta hop in the ring, da, da, da, da, right?  And now you're doing a 180 on people, right? And also people are still saying they respect you as a man. It was like, look, that's way more important anyway, right? So like, he's gonna be thorough throughout this. He's already at an untouchable level anyway, damn near in terms of like his career is not gonna be over. But if Meek could go through what he went through and come back to the level of relevance that he's at, this ain't nothing, man. Yeah, and I agree. He actually brought up a good point. I do, now that you said that and I think about it, I do think the apology is on brand, right? Cause J. Cole's brand has been, you know, God level MC that's underrated, right? And he always likes being the underdog in the situation. You know, as big as he is, he still talks about himself like he just started yesterday. You know, that's kind of insane. But as much as his brand today is kind of like Kumbaya, you know, brotherly, little more brotherly love type of, type of, I think going all in would have actually been very anti-brand, cause it would have been weird.  Why are you going so hard in this situation? Unless of course we as the audience learn something, you know what I'm saying? That maybe you made us feel justified. It will feel kind of off, you know? And then the second thing, you know, I like the ring example you bought, right? Cause this is literally what it feels like to me. It's like these are two high level boxers who know how to fight to the highest level in the sport. They know how to throw hard blows without killing each other. The audience is waiting for somebody to decapitate the other person.  You know what I'm saying? We can still do that and not tear each other down. You know, if we talk about, like I said, taking it to like a hip hop thing, I mean, you know, for artists of their size, it's probably not the best for either of them to go that crazy at that level. Because historically that very rarely works out for, you know, it might work out for one party, you know what I'm saying? I think that there's still that level of respect where it's like, okay, like we don't want to end the other one's career. You know what I'm saying? That's not, I don't think it's that serious, you know? But then even when we go back and look at, you know, past histories of rappers that have beef to that degree, you know what I'm saying? It's where it does get that crazy. It very rarely works out, you know what I'm saying?  That I think makes people fetishize it a bit more. But if you look at all of the beef and music, you know what I'm saying?  You know what I'm saying? That's real. And then you and Ritter said something that I think like a lot of people really missed, bruh. And let me see if I can find it real quick. He said something that is like to me just straightforward. He said, what classic disc records were the result of two cats that respect each other just wanting to prove who's best? Yeah, exactly. Like, we don't have any real beef. Like, what are we really going like that whole just for the sport of it? It's a hard thing to really maintain because too much ego evolves, like the lights are too hot. You know what I mean? Too many people are watching. And this is what I feel like happened. Matter of fact, I'm going to go through just it's important to recap like exactly what J. Cole said and then provide context, right? So he said, when he was on stage, I'm so proud of that project except for one part.  I think that's important because people are calling him lame. People are calling him soft now.  So I want to get into like the as a man part that I think is really bigger and actually doper than a lot of people want to give credit for. You know what I mean? But we'll get to that. But the first time I was tested, why am I tested? Because I've got the world and I got my niggas like what are you going to do, Cole? I felt conflicted because I didn't know I didn't really, what do you say? Because I don't really feel no way but the world want to see blood. I don't even care like that. Well, I don't really feel any kind of way but my people are watching. And what did he say in middle child? He said, but I'd never be for the nigga for nothing.  Middle child, he said that. I think he said it might maybe twice in that song. I'd never be for the rapper if I smoke him is going to be legit. So it's like, this dude, I'm cool with him even if I'm not super cool with him I just don't care enough. Like I don't have no real problems with this dude. And to me, this goes back to J Cole being on brand. He said this stuff time and time again. Like even in that same project he just dropped might delete later. He had lines going touching on like the fall off over and over again, which your project is still to come. He's always going through something that I don't think I've personally seen from a rapper art form in the spotlight throughout their career journey at like J Cole has which is deconstructing your own ego before people. The reason I say that is because at one point I thought it was just a straight brand play. He was like, oh, he was going this one way was a little bit more rapid, rap-y, boom, bap-y, you know, not like straight up, but you know, very more lyrical and that focus on that. And then he's like, oh yeah, I'm gonna be this artsy. Look, I'm a grow locks. You know what I mean?  You know what I mean? At first and then flip the whole brand. I got the locks and then I'm like, we are the world type vibes. But outside of like the we are the world Kumbaya type vibes, there's been a lot of focus on his own personal ego over and over again. And if anybody has committed to something, like for real, you've changed your walk of life in a very real way, whether that's religion or just you were in trouble and you want to be in less trouble. You know, with the people you're hanging out with, you can always say, all right, I'm gonna make a change or whatever. And for Jay-Caul, all right, is this thing just marketing and branding or whatever like, and this vibe is just working for him. But when you can see that the change is real is when someone gets tested, like in a real way.  Like that was, to me, that's a real ego strip. Like it wasn't, to me, it couldn't have been easy for him to go out there, do that. And then step back. To me, that actually solidified his brand and everything he's been working on the last, I don't know, five, 10 years or whatever, more than anything. Yeah, no, that's exactly what I was gonna say. I think that, you know, like I was kind of saying a couple minutes ago, overall, I do believe it's the better brand play because if you are a Jay-Caul fan and he kind of would have stepped like that, now you're looking back at his whole history and his whole work and you're starting to question things, right? Because what you, now in this real life example of what you get to see him act like it doesn't line up with the picture he's been giving you the last couple of years. This is exactly what, you know what I'm saying? It lines up with the picture he's been giving me for the last couple of years, like these actions. And you know, I think that going back to something else I said, I don't really think it's a lose-lose situation for Jay-Caul because his overall brand is I'm a high level underdog. So if he, you know, was humbly considering himself to be at the top of that, and he gets pushed back down, he can push right back to his comfort zone, you know what I'm saying?  And so I feel like that may have been a part of him that thought about that, like, hey, like, you know, if I move forward in this way, this could be a lose for me. This could be lose-lose for both of us.  And I think the most commendable thing about it, which is something I think is hard for a lot of artists to do, is to stand on your core brand principles when you're put in a situation where your brand, acting on your brand could be a detriment to yourself, you know what I'm saying? And so it's like, that's when you really got to think, like, man, is this really who I, you know, is this really who I am makes it easier. You know, a lot of artists like to live and die by the brand is me, but it's like, this is when you kind of get to see it. This is really you, we gon' see it.  I gotta deal with the X, Y, Z, you know what I'm saying? Because this really is who I am and what I align with. And I think we've seen that in bad ways before, you know, it's like the equivalent of saying like a street rapper crash out.  No, it don't make sense, but I get it. So I look at this as a more positive version of that same thing. Exactly, bro. And it goes back to, again, people are saying it's on brand. You still people, still seeing people say stuff like, well, I mean, I respect them as a man, but to me, right? People saying I respect you as a man, that's better than the reverse, right? Should I rap better and make my life worse? You know what I mean? And what I mean by that is like, there's people who have these great illustrious careers. And then we're like, ah, their personal life. You know what I mean? We see what's going on with Diddy or whatever, questionable. We don't know what he's gonna pan out to be, but there's multiple examples where it's like, ah, the music is great, but the man ain't all that. Or we don't know, question mark, asterisk. But right now, he's standing on. All right, we knew he was great musically. As a man, all right, cool. And we rock what you solid. Now your music is in question. For me, based on somebody who's building a brand that he has, like, and when I say brain, you know, I use that obviously because we were talking about marketing. That's just how people know him from a perception standpoint. But you know what I mean? He's supposedly expressing these core values and going on this journey. Like when you go through something like this, like what you just said, you're in a put in a position that could like really like reverse everything that you just did. And I know a lot of people think it's not that serious, but for him, it was that serious.  So like, that's where I want to go back to what he said. Like, if you think about exactly what he said, right? Cause people just say, oh, he apologized. He apologized. He said, for the first time I was tested. See, he said he was tested. Like, again, I'm going on this old deconstructing my ego journey. I'm seeing the world different. Why am I tested? Because I got the world looking at me and I got my homies saying, what are you going to do? I felt conflicted because I don't really feel anything about this situation, but the world wants to see blood. Like, and ironically, I did an episode, like it's one of them Saturday episodes, last Saturday I dropped it. And it was talking about like the rollout strategy just doing the song a week, but how one of the reasons people could wrongly continue to do it is just the ego trip of, yo, I gotta keep doing it. Just clock and say that I did it, right? Not looking at what was your purpose for doing in the beginning and are you getting that outcome or not, right? Or is it just so you can tell people you did it? You can show other people you did it. And at one point I talked about ego trips, right? And the whole goal is, right? An ego trip, if you look at it as a literal trip, your ego could take you around the world, right? But as you go through the journey that J. Cole is saying, he's going through, as he deconstructs his ego, the whole goal becomes making the trip shorter and shorter.  Now I was able to stop my ego at the airport and turn around and not waste all that time. Now I got in the car and I just turned around. It was like, yeah, you know what? That's a bad decision. I shouldn't even go there. And I think that's what happened, right? We saw J. Cole practicing the exercise and this entire process. And he was able to literally just go to the car. You know what I mean? Ride down the block and then get to the light and say, nah, bro, I'm going back in the house. And that's really what you want.  My whole goal for life is spending a minimum amount of time playing other people's game. He said, I don't even care about this like that. So like I can respect that part wholly. And again, to me that's solidified cause I'm always skeptical of someone who's like going that way with that type of brand and why are they really doing it? And I'm like, you know, like there was nothing to win for him and people, cause the one thing that, you know I'm talking all this like woo-woo about ego trips but the trick is like you can benefit from ego trips. I'm not saying that like it's all like a L. It's only a L based on your personal like measurement but like, bro, people going ego trips, win rap battles, elevate their status over and over again. But J. Cole's always talking about, I don't care about the status. I think even in Mike Deli later, he said, I don't even care about the status or some version of that, you know what I mean?  Like we're just reacting. Cause again, we didn't know that that was an option, we did not that you could do that. That's true. That's why you talked about, you know, possibly making history is like, I mean, he technically did man, cause I can't think of any other situation that is even close to something like this where, you know, or at least not a situation where it garnered the same level of press and conversation. You know what I'm saying? It's probably happening, but it probably wasn't, you know, it wasn't that big. And I think that the message is gonna shoot through the music community as a whole. It's gonna be pretty interesting.   But if you don't like the more like, hey, artists should be cool with each other and cool about y'all.  And I think it's gonna be a good push forward for that, you know what I'm saying? And I think it's a good lesson for all artists to not let your audience bully you into being tough for no reason, you know what I'm saying? It's a great lesson to that. That's right, that's not hard. Legitimately, if people wanna say that's what tough is, like what's softer than doing something only because everybody else wants you to do it?  You know what I'm saying? That's a hard situation to be in, you know what I'm saying? Right, right. See the brother, that's like, that's the double entendre, right? It's often hard, but like what's literally harder, way harder to actually do that than just hop in the ring and keep going. And then- In your home in the studio, you don't make it so easy, you know what I'm saying? Yeah, yeah, exactly. Way harder, dawg, way harder. Who do you think is gonna take, who do you think takes the biggest L from this move that J. Cole made? I mean, I guess if there is an L, J. Cole takes it, but I don't think it's a big L.  Like an L that could develop into a bigger L, but I think right now it's a small L. I disagree. I disagree with J. Cole. Okay, you think Kendrick does? Yes, 100%. I thought you were gonna say Drake, bro, okay? No, 100%, Kendrick took the biggest L of J. Cole's apology.  Cause now it's just like punching the kid and it's like, hey bro, like, I'll set up. I'm gonna go deeper. I'm gonna touch on that reason. And then there's another reason that nobody's talking about. Okay. So reason number one, yes, because that's kind of wild. It's just like, all right, bro, go ahead and hit me on the chin and it doesn't hit the same, right? Cause it's like, all right, I guess that's cool that he hit them, but we can't celebrate it. You know people can't celebrate it the same.  I can give you the bars, but it's not really even worth my time. How hard can I really go? You know what I mean? He might even just have to say something like, I'm disappointed that you went that way. You know what I mean? Nick is getting weak. It might be like that, but he can't just go hard. Or he has to go so hard that we're like, yo, he's violating and being disrespectful and see if he can bait J. Cole out because now it's legitimate beef, right? Yeah. Might be it. This is why J. Cole, why Kendrick is taking the biggest L to me. Like the real reason why he's taking the biggest L out of this to me. Not because what he technically lost that he has, but because of the opportunity he had that he lost. Okay. Kendrick almost got exactly what a Kendrick type, which is like a Koby type of energy would want. Kendrick would, his eyes probably goes red over the thought of I'm going against Drake and J. Cole at the same time. Y'all thought one of these dudes might be on my level. I'm about to slaughter both of these dudes at the same time before your eyes. And I think part of him like hates that that opportunity is gone. Like that's the L for him. And then think about the level of gold status that he would have like just fighting that fight. It's almost like the can't lose. Yeah, but see the thing that makes me think there's still a chance for him is that Drake hasn't responded. And if I think anybody's going to take it too far, I think it's going to be Drake. Oh no, like don't get me wrong. But there still can be a battle that they do whatever they do. But like that's not the same as I just beat the other two at one time. Cause again, that's also some stuff that we haven't really seen. Yeah, that's true. No, J. Cole is the beat into the punch with this, with the exit strategy. He's like, damn man, this dude left through the vents in the room. I thought it was, I ain't leaving no windows. I locked the doors. This man found a vent and slid on out. Man, yeah, I didn't think of it. That's a good point though. I didn't think of it that way. Cause yeah, cause like, but that, and it goes back to the original conversation we were going to have about, is it a rollout? I also feel like, I don't know. I just feel like, like you said it was a competitive moment that Kendrick was looking the drum up.  And he's like, hey bro, I got something for you. Like I'm gonna make sure, you know what I'm saying?  I can see that, you know?  I ain't got nothing coming out.  I mean, future probably like, all right, but, and so that's what I said. I think overall, man, like going back to that, if it was a purely organic moment, which I do believe it is, I still think it is one of the most entertaining moments of music in at least the last like 10 weeks. If it was orchestrated, the person that orchestrated that deserves a raise. Whoever, you know what I'm saying, sees this and is behind it. If there was a person, a group of people behind it, they're geniuses. Because like you said, bro, it's the ultimate rug pulling trick. Nobody would have seen it coming.  It's still gonna get the marketing offset one. Like if it was somebody's idea, it was a great idea. And I want to give them their flowers. But if anybody could have did that, it does line up still because J. Cole already talks about the fall off anyway. Like this could be the entryway into the fall off. You know what I mean? He literally referenced the fall off on my lead later. And then you just lead them to do whatever they do. So it does fit narrative wise. If you really want to go like conspiracy, you know what I'm saying? And as you say that, man, see now you kind of got me back to the marketing side. Cause I was just thinking about, I could see some like marketing, you know, person at the label been like, yo, we should like have you be for the rapper. And J. Cole was like, I like, I don't really want to do it like that.  Like we'll please have you apologize at the end.  And he was like, all right, bet. I can see that. I got a hundred percent see that. You know what I'm saying? I only thought they moved everything up a little bit earlier. And it landed on April Fools.  I mean, a lot like that's it is. It's like, you know, like, and I and Dreamville fast as a, as its own entity is what on like year four or five. This is around the time when people start trying more experimental marketing tactics. You know, they started feeling like, man, we got the core. We need to get more visibility. What can we do? Yeah. Kendrick had popped up at it crazy. You know what I'm saying? But no, yeah, there is still elements of it. Do make me a little bit suspicious. I ain't a lot. We knew something was going to happen, man. Like, like you said, the time of the Dreamville fast shot of the E.J. Jamaica. He asked if I was going to be there. And I was like, nah, but like, I know J. Cole on had that moment on stage. Nobody expected to be this moment. But, you know, you knew they had to be addressed on stage. That's that's what came with it. Yeah. Why not here? And J. Cole on post, so it had to be a festival. You know, you ain't going to make no content about it. You might look on here and knock it out here. Yeah. Yeah, I also think everybody was as surprised as we are that he pulled out. I don't think because things got moving so fast, I don't think the world fully acknowledged how surprised they were to hear respond in the first place. I agree. I agree with that. I agree. It was a cold conversation at first was that no one was going to respond, except maybe Drake. And then not only did he respond, he was the first to respond. Like, I know I saw it coming.  So I was like, man, I ain't saying he ain't never going to talk about it.  I saw DJ Academic say that Drake was like, nah, you must not know me if you think that I would not respond. If you ever think I would do something like what he did, like that type of energy, like he's definitely going to come. That's what I'm saying, bro. Like the energy that everybody wanted from J. Cole is coming, bro. Like we all just got to chill out, wait, you know what I'm saying, find some other things to do on the internet in the meantime, because I 100% believe Drake is going to be the one to take it too far. So, you know, to the same point that you made about Kendrick and him wanting to do that, I could see Drake feeling the same way because it's been a long time since the Meek Mill situation. You know, there's been a lot of jabs at the crown the last couple of years. And what better person to make an example of than Kendrick Lamar, you know what I'm saying? Like who else could he even make up that type of game for exactly as nobody, bro? Like, so it's like he also has a unique opportunity. Pusha T was like a side quest that nobody saw coming. But it was like, oh, actually we can take this seriously because Pusha is pushing. And he found the angle where people cared, right? But yeah, but there was nobody really you could look at for real after Meek aside from Cole and Kendrick. So, let's see how this goes. I think the reality is though, hey man, it seemed like J Cole been in that therapy heavy. And when it comes to him, I'm not surprised this is him because between him, Cole and Drake, he seems to be the most at peace. The mother two guys seem, I got some little insecurities to eat at us, avoid to feel whether it's just cause I want to really clearly be seen as number one or whatever they want to achieve. You can tell they're in a different vibe. J Cole, which is again, goes back to the power of him stepping out. Like, if you're at a certain level of success, right?   I'm not tied to any organization.  I'm good. Like my mind is clear. I'm at peace. I'm just really working for fun, getting into things that fun for fun. That's it, whatever.  And I think that's, well, again, he showed, cause that, nah, man, I'm not gonna get caught up in his game. He broke the matrix a little bit. That's what he really did. He broke the matrix as people saw it. And I was like, nah, bro, like why be written? We always talk about being rich and successful, et cetera, et cetera.  He was like, nah, I'ma actually not be to do it at the top who was playing the game I wanna play. I'ma be that guy that y'all say that your life would be like. I think his life, it seemed like his life might be pretty straight. And I agree. I think he is probably the best case example of an artist that can be large and also not care about traditional industry, politics, stuff, you know? Cause I've been trying to find out. That's what it is. The industry politics, the label stuff, like that type of mentality, but large, yep. Yeah. And it's like, I mean, you know, it's only so many that are even closer than, I think he might be the biggest. I can't think of nobody else. And then maybe, well, I was gonna say, maybe coming here, but that's not true. But so yeah, he might be, yeah, that's not true. But so he has a, he has an indie mentality as big as he is, like the same FU where I don't need the system or want to be involved in X, Y and Z. That's exactly what it is. And he's a good, he's setting a good precedent for those people who are gonna be, you know, actually indie and have a certain mentality. It's like, yeah, you can really just be like, if all of this, I'm running this my way. I don't want to be involved in your games and I'm just playing my own game. Yeah, I agree. I agree. Inspiration on so many ways. Go put them on your wall. No, no, no. I got fans listen to the album first and then I get back, but no. I'm cool with just having the memory to be honest with you. Oh, man. Yeah. But let us know y'all's thoughts in the comments, man. Like is Cole's career over, right? Was this move a soft move or a hard move?  And also, do y'all think Drake is gonna respond for real if he did? Who would win, Kendrick or Drake? I'm not calling none of these dudes by their nicknames. This is yet another episode, no label necessary. We hopped in for a quick one. Let us know what y'all think about the Zoom episodes as well. I'm Bram and Shine. No Corey. We out. Peace. Peace. All right. Let me get some comments, all right. Yeah, that's gonna be an interesting one. She's doing it.


Welcome to ThinkTech on Spectromosi 16, Hawaii's weekly newscast on things that matter to tech and to Hawaii. I'm Elisa Anderson. And I'm Cynthia Sinclair. In our show this time, we'll review the most recent top five ThinkTech talk shows and staff picks. We'll check out the elements of the best of the best and get a handle on the public issues and the guests involved. ThinkTech produces more than 35 talk shows every week in our downtown, high-tech green screen studio. Our ThinkTech talk show offerings are very diverse and their coverage is also very diverse, covering things you might never have otherwise known. Every week, ThinkTech chooses its top five ThinkTech talk shows from the week before based on the number of views each of them has had on the internet. For this past week, the winning shows were as follows. Number one from the series Pinoy Power Hawaii, hosted by Emmy Ortega Anderson. It was called Pinoy Power Extravaganza, Pinoy Power, with guest Efren Extravaganza. Emmy had a conversation with entertainer Efren Extravaganza. Efren performed songs and comedy skits and shared how he was lucky to have come to Hawaii to live the American dream.  I like singing, dancing, everything. So when I was young, my parents brought me by me a harmonica, a gauge of seven. And then I always liked to play harmonica when I was young.  Yeah, that's right. I was in school in Bakara National Comprehensive High School. I was in school in Nagilian Academy in Second Year High School. I was a pastor here in the Philippines. That's why I like to play the ball. The music. He told me to sing with me.  It was called a study of current scientific research, development, deployment and prospects, only on likable science. But guest Leland Worden, tropical landscapes have been extensively degraded due to human activities. Fortunately, many of these landscapes are very resilient and have great potential to recover after major disturbances. Restricted ecology is a really enormous field, but the concept is that you're coming in and basically restoring an ecosystem that has been damaged, degraded or destroyed. So the idea is to try to bring back function that can be species or also just ecosystem services of that ecosystem. And specifically I work on tropical forests restoration in both wet and dry forests. One of the really cool things about restoration is that there's so many steps involved that you really have to take a holistic approach where you understand how a seed germinates and whether or not that species is suitable for the soil type and there's a lot of things, but it's true. It's very complex and really fun to work in because you get to do a lot of different things. Basically the green segment is the protected area and I worked in kind of the northwestern corner of that, this little anvil in the corner called Horizontes, which is the experimental forest. What's really incredible about this system is that about 80% of the forest has naturally regenerated after just removing some pressures such as unnatural grazers, cows, and that actually restricting non- just naturally occurring fires. Where I work, a lot of that basically the pressures were agriculture and the reason that we're able to restore our scale there is because a lot of the production of agriculture products, which was mostly beef, has actually moved over to China. So that is a success story in the fact that we're able to now restore that land, but there are a lot of places, even think about prairies in the Midwestern United States that have this tiny little area now because of agricultural pressures, but we don't really have the ability just because of human land use to restore those systems. Even in natural forests, you can think about actually harvesting of forest products such as collective harvesting of wood, and there's a lot of ways that we can actually sustainably use ecosystems rather than just completely replacing them with agricultural or other uses. Even though that is absolutely necessary to sustain our really incredibly skyrocketing global population, but there are some ways such as agroforestry, where we can actually think about transitioning systems that are purely agricultural over to hybrids where we do have forest products and ag as well. Number three, from the series Lillian's Vegan World, hosted by Lillian Cumick. It was called Vim Vigor and Vegan, surviving and thriving on a plant-based diet with guest David Nichols. Contrary to what people think, those on a plant-based diet are just as energetic as omnivores, if not more. Lillian talks with David about how he became vegan and found the healthy diet he was in search of. Many famous vegan athletes thrive on a plant-based diet. I've been wanting to have you on as a guest because your story is very interesting. I actually interviewed your partner a few weeks ago on my show, the lovely Dukor Murusan. She was amazing and I was very interested to hear that you are also vegan and on a plant-based diet. I would say about 98% vegan. I can't say 100% but I try to watch what I eat. Tell us a little bit about your journey. Again, when I met Ryukul, that's how I met her. I was more vegetarian and I switched. I was really overweight at one time. Genetics paid part of that with obesity, high blood pressure and diabetes because my mom has high blood pressure and diabetes. I was very unhealthy, blood pressure out of the roof, the sugar level going really high, borderline diabetes. The doctor was like, okay, we're going to give you all these meds. I was like, I don't like taking a pill. So I asked him, is there any alternative? He goes, yeah, watch your diet and get some exercise. One of the biggest main questions we get where you get your protein from. Yeah, that's a really interesting fact because the beliefs that I was instilled with is that you're going to die if you don't eat animal flesh, which is animals survive on. I know there's animals that survive like cows and horses. They're healthy and they don't eat herbivores. Their protein comes from what they eat. Why eat their protein when we can eat the same thing? Directly from the source. And they're healthy. I started the vegetarian diet. My blood pressure started going down. I started exercising. Exercise is a huge factor in it. Definitely. It's mainly what you eat. I did come up with quite a few famous vegan athletes that might surprise a few of the viewers. One of them is Venus Williams. She adopted a raw vegan diet after she was diagnosed with an autoimmune disease called Sjogren's syndrome. Wow. She adopted the raw vegan diet which is a very, very strict diet obviously meaning no cooked food. Okay, raw vegan. And so she started her raw vegan diet in 2011 to relieve some of her symptoms. Number four from the series Trump Week hosted by Tim Apachele and co-hosted by me, Cynthia Sinclair. It was called Real Constitutional Crisis on Trump Week. We discussed Trump's orders to Don McGahn and William Barr ordering them not to comply with subpoenas issued by Congress and Trump's efforts to block congressional attempts to obtain his federal tax returns. The subpoena for Donald Trump Jr. to appear before the Senate Intelligence Committee has also been in the news. The administration's moves on China and Iran are likewise troubling. We discussed how Trump is dismantling our democracy and how, if at all, we can preserve it. Before we go down the points, know that Nancy Pelosi said, she said, I believe we're in a constitutional crisis. Now, Jerry Nadler, who's the chair of the judicial committee, he was the first one to say it. Okay, so what leads us to this point? What leads us to this statement? It's a very dramatic statement. We've heard it before. We heard it in 2000 when we had the hanging chads in Florida and it wasn't clear how this election was going to be resolved. I distinctly remember the comments that we're in a constitutional crisis. They certainly said it when Richard Nixon didn't want to turn over the tapes and yet he was being asked to do so. So we've heard it through history. We heard it during the Clinton impeachment. But what makes this one a little more serious and different? There's foreign actors involved. At what point does executive privilege begin and stop? So we know that if you talk to the media, we know you've been called in to testify and you testify about certain things, you have basically waived executive privilege. Now the legal team for Donald Trump says no, that's not the case. Well they say no, that's not the case for everything, which is how we got through this crisis. That's how we got here.  They're just stalling for time is my thought. Because the more they can stall, the closer we get to 2020 and the more likely people are not going to be swayed against him unless they see the evidence that's there. Not only did we have the 800 signatures from DOJ, we also have a Republican who is a very, very important part of the Senate and he's a Republican and he said no, we are going to hold Donald Trump Jr. accountable to the statements he made before this committee versus what we've found to be in the Mueller report. Exactly. And he is taking flack for it. I can imagine just how many calls and how many people are bumping into him in the halls of the Senate and really laying it to him. But this is where again, the crack in the vase is this further evidence that we've gone far enough and I don't know the answer is to that. I don't need it but I can just hope, right? That's all I can do is hope. Number five, from the series Hawaii State of Clean Energy hosted by Jay Fidel and co-hosted by Maria Tomei. This is the 2018 legislative session with guest Kurt Sue. Jay and Maria talked with Kurt about energy policy and the energy bills and resolutions of the 2019 session. Yes, we need to focus on and follow energy in every session. We're trying to partner with a lot of emergency management agencies and elected officials when they host their preparedness fairs. We want to be a part of that conversation and I'm sure you've seen the handbook that we've collaborated putting together to get people ready for hurricane season and all the other associated events, right? So that's an important part of the conversation as well. There's not only clean energy and sustainability but also resilience and preparedness. I do think that interest is growing. People want to be part of the conversation whether it's rooftop solar electric vehicles, lots of questions and opportunities and when we're rolling out new programs within the company, that's a perfect opportunity to meet up with that interest and to talk story and to get to share information. A lot of us get those inserts in our bill, and then that contains a lot of useful information as well. I know I'm guilty myself sometimes of getting the mail and not reading everything as carefully as I should but there's a lot of good information in there and traditionalists who look forward to our recipes, that's also a piece that's built in in there as well. Of course. What else? So what are some of the things recently that people have been asking about? Well, rooftop solar continues to be, we're getting a lot of calls about particularly a lot of projects. When people see things in their community, they want to know what's going on and more recently we're the renewable request for proposals, so we have developers now going into communities and so people have questions about what's happening and why and so those are the types of questions that we receive and even at the neighborhood board level, whenever they see trucks in the neighborhood, sometimes they'll call, what's going on, what's why electric doing in my neighborhood? The guys who run the trucks are perfectly happy to answer your questions too, aren't they? You asked a question. They do great work and so that's the part where we really want to share that information and it goes back to what we were talking about earlier with regards to resilience and sustainability, all of these initiatives are really going towards those purposes where we're trying to harden the grid, make the grid more reliable and that which leads to better service for all of us who use energy and we're also working towards the 100% renewable goal in trying to integrate more renewable energy. And our staff pick, from the series Taking Your Health Back, hosted by Wendy Lowe, it was called From Fast Food to Smart Food, You Are What You Eat with guest Christy Nicks. Christy shared her health journey, Taking Your Health Back is made simple one step at a time. Why is this topic so important to you? Well, because I've been on the spectrum from fast food to smart food and as I started to learn how important it was to eat more fruits and vegetables, I learned a statistic. There's like some statistics that just always stick with you and this one just kind of it hit me. Dr. David Katz said this may be the first generation of children to have a shorter life expectancy than their parents. One other statistic you've probably heard it from Dr. Sears I know you study a lot of what he teaches. He says that 75% of all chronic disease is preventable. 75% so if we have that ability to change, like I look at it as if I had a lottery ticket and I was like here's the lottery ticket you have a 75% chance of winning, you would buy the lottery ticket we need to take that same approach with our health we have such a great ability to be able to shift our habits that this can make a really big impact and I couldn't have done it without being around a circle of people who cared more about my health than I did at one time and now I just want to be able to pay it forward. I can't even tell you the amount of friends I've gained because of one simple change that I've made along the way and you know they always say you're the average of the five people that you hang out with now and I just I feel like I've really leveled up so it's been really impactful to be around a lot of great people that have impacted my life. Pat and I were awful we would wake up and have like snicker bars for breakfast and we just didn't think about how bad things could be and my friends titled me as McChristie because I would go through the drive through McDonald's I mean I ate what I could afford which was total junk that's all I ate was junk. I had McDonald's twice a day every single day and everything else if I did make something it was like I ate like a frat boy it was macaroni and cheese, cheeseburgers, grilled cheese like just anything that was beige with no color I was a frat girl. Well here in Hawaii we call that the local diet okay and you know like I can totally relate to you because and that's why this is so encouraging for everyone watching this because frat boy here frat girl here our local chick I mean the worst diet we lived on fast food so much as mindset you know there's this there's this pattern that we tend to get into and for whatever reason I didn't get into a pattern at a young age about having a belief about a certain thing maybe about food or about my body and then you have these thoughts that create that and then it just keeps going into like this cycle of you know your actions then support the beliefs and thankfully whatever it was maybe it was my mother I don't know but I just never had that bad relationship with food. What I would really like you to take away from today's discussion is the idea that food really is medicine you've heard that before that food be thy medicine and medicine be thy food so simple changes will have massive effects on your overall well-being. You can always find the links to these shows in our daily email advisories if you don't already get our daily email advisories you can sign up to get them on thinktecawaii.com These are only sampling from the top five and the staff pick from across our 35 weekly talk shows. There are of course many more. To see these top five shows and staff pick shows in their entirety go to thinktecawaii.com or youtube.com slash thinktecawaii. Great diversity great community great content at ThinkTech. If you have questions or comments about these or any of our shows please let us know and yes it's okay to share them with your friends and colleague. Thanks so much for watching our shows and for supporting our efforts at ThinkTech. And now let's check out our ThinkTech schedule of events going forward. ThinkTech broadcasts its talk shows live on the internet from 11 a.m. to 5 p.m. on weekdays then we broadcast our earlier shows all night long and on the weekends and some people listen to them all night long and on the weekend. If you missed a show or if you want to replay or share any of our shows they're all archived on demand go to thinktecawaii.com and youtube for our audio stream go to thinktecawaii.com slash audio and we post all our shows as podcasts and iTunes. Visit thinktecawaii.com for our weekly calendar and live stream and youtube links. Or better yet sign up on our email list and get our daily email advisory. ThinkTech has a high-tech green screen studio at Pioneer Plaza. If you want to see it or be part of our live audience or if you want to participate in our shows go to thinktechshowsatthinktecawaii.com If you want to pose a question or make a comment during a show call<PHONE_NUMBER> and help us raise public awareness on ThinkTech. Go ahead give us a thumbs up on youtube or send us a tweet at thinktechhi we'd like to know how you feel about the issues and events that affect our lives in these islands and in this country we want to stay in touch with you and we'd like you to stay in touch with us together. We'll be right back to wrap up this week's edition of ThinkTech. But first we want to thank our underwriters. Okay Cynthia, that wraps up this week's edition of ThinkTech. Remember you can watch ThinkTech on Spectrum OC16 several times every week. Can't get enough of it just like Cynthia does. For additional times check out oc16.tv for lots more ThinkTech videos and for underwriting and sponsorship opportunities on ThinkTech visit thinktechawaii.com be a guest or a host a producer or an intern and help us reach and have an impact on Hawaii. Thanks so much for being part of our ThinkTech family and for supporting our open discussion of tech, energy, diversification and global awareness in Hawaii and of course the ongoing search for innovation wherever we can find it. You can watch this show throughout the week and tune in next Sunday evening for our next important ThinkTech episode. I'm Elise Anderson and I'm Cynthia St. Claire. Aloha everyone.


Well, hello and welcome everybody again to another OpenShift Commons briefing this time Brian Brazil who was one of the core contributors to the Prometheus project and the founder of robust perceptions Are the a new member of the OpenShift Commons is joining us today and he's going to give us an overview of Prometheus So the format of this is he's going to take it away Give a bit of a presentation a little bit of a demo of working with it in context of OpenShift And then we'll have an open Q&A afterwards. You'll notice because we're using blue jeans There is a chat so you can ask questions in the chat I will try and repeat them all and in the Q&A and open it up for a conversation afterwards So with that Brian take it away and thank you very much for joining us today No, thank you So We're not an echo stream of error So I'm one of the four core developers of Prometheus found robust perception She does, you know trying to do consulting in support route for supporting open source I've contributed to many open source projects over the years and they've worked in Google for a while as well here in Dublin So Prometheus was founded in 2012 by Mac Proud and Julius Balls who were located in Berlin at the time and like in 2013 it started as a side project and 2013 it took it into sound head where you're working for at the time Expanded to support Bazooka, which you can consider to be kind of like OpenShift and And they added instrumentation clients for Go, Java and Ruby, which was what was used inside sound head at the time In 2014 other companies started to use it including myself and we had Johannes in Docker So another component of OpenShift and we have a new storage system the V2 storage and a new text format Which we're still using today and then in 2015 we publicly released Because previously everything was on GitHub and it was there But we didn't really tell a lot of people But since then we've seen quite an uptick in usage So to give you an example and there are about 300 contributors to core repositories We have it's actually over a hundred and fifty chart party integrations and we've got several hundred people on the middle list and there are C and It's always hard to tell with open source software But we figured there's over 500 companies using Prometheus in production and there are several companies My own funding Prometheus development In fact, there was a recent stat that came out as well that Prometheus is 56th in the world for open source contributions, so like 56th the biggest project, which is a bit of a surprise On top is Linux itself So previous what is it and so it's a metrics monitoring system. It doesn't do logs. It doesn't do tracing It's not a provider It's at its core time series database with a query language It has clarion libraries to get the data in and a general ecosystem and in general It's looking for a cloud native approach to monitoring services The basic architecture is we've got previous in the middle It's talking to Kubernetes or OpenShift to figure out where everything is going out for applications or exporters Putting in all the data storing it locally sending out alerts and offering that up for graphing If you look over the last 20 years service management It used to be the case your systems were manually configuring everything on the machines and then we went to things like Chef and Then we went to Kubernetes, which is to step up again Where we're just moving ourselves further and further away from the individual machine the individual process And we need to do the same thing for monitoring We need to not look at every single machine and oh that CPU is high But actually care about hey end users care about latency and error rates Let's look at those rather than distract ourselves with alerts for things which fundamentally don't matter to the end user We have to look at what's the ROI of looking at a particular alert So in terms of integration into Kubernetes or to say OpenShift agrees can discover all the positive services everything automatically from the Kubernetes cluster and put in all the labels of annotations Because both Kubernetes and Prometheus have a label based data model So that measures nicely and Prometheus will automatically pick up the changes So you can just set it up once in principle and have everything happen via magic from there on in Then to add your actual application Each instrument your code to capture the metrics that matter to you Because sure it's great to know that the CPU works because we can do that out of the box But you care about hey how many times is this cache being hit or how many times is it a paying versus a non-paying order? And that sort of thing and the great thing is if anyone instruments and you've using the libraries which are instrumented Well, you get that for free get those instrumentations for things that aren't instrumented because well, it's not likely we'll get our code inside my SQL There's export it's looking to work between them So lots of integrations there SNMP for your network stuff console JMX HAProxy and of course Minecraft and Factorio if you're having a little bit of fun and Incidentally Minecraft is started by a friend of mine. I should start a new server last night and Just figure out how well his Minecraft server is performing because he uses a lot of mods So as well if you look across the cloud native computing foundation Cooper to give you an idea of the sort of integrations that are out there and how prevalent integrations of Prometheus are Kubernetes itself is instrumented with Prometheus. So we integrate with each other as you can monitor the health of the cluster Linkerd has metrics in our format and is interceptors for a GRPC and plugins for fluency So now we've got all your applications either directly or via an exporter providing data We've discovered them all from the Kubernetes services covering. What do we do with all this data? We have the promql theory language Which can do basically any math you want on time series data. It can aggregate. It can join series together. It can slice things And it can do some predictions like how close you are to your quota Like I'm a bit run out of space in four hours is more interesting than how I get 95% of my quota because you could get 95% for years Or aggregating across an entire data center for latency Another key point is that there's no distinction in Prometheus between graphing and alerting. It's the same expression language So if you can graph something you can learn so much So here is an example of a more complicated query that you would have with Prometheus so this is taking example of a CPU usage in Docker containers and Getting up per container Something that up by Docker image Not by machine not by service not by user not by data center but Docker image and Getting the top five so you could figure out which containers might want to use some better optimization flags for As an example Then okay, you've got your alerts What's thing is that not whenever we alert should necessarily result in notification You can imagine if you had a rack failure with a hundred machines in it You don't want to receive a hundred pages because that's just silly And so you can actually group alerts together that are related and only get one alert for that rack failure Or maybe one alert for every machine that happens to be currently failing in that data center with you know Hundreds thousands tens of thousands machines root them to the right team and then shot the notifications You're not spamming person every minute that hey This is still firing because that really doesn't help a human who's just gotten a page about a wrecking down It's also designed to work reliably during narrow predictions So the whole architecture is around reliability such that even if your network is falling apart We will do our utmost to make sure you get your notifications You might get some duplicates, but that's better than getting nothing So to summarize then Prometheus, it's a metrics monitoring system a time series database a local hurry language The client library for your data and an ecosystem of exporters and so on Because we're trying to promote here a cloud native approach to monitoring services because caring about individual pods or individual machines Is not sane in a highly dynamic infrastructure like OpenShift There is some links there as well the slides will be up later so you can look at it So I'd like to just show you through a bit of a demo That you may wonder how easy is it to instrument things with Prometheus We're going to use an example from Python. So this is included in client Python at the top So I'm going to copy and paste this off-screen And run that then okay, that's running. So this is just importing the basics Tracking requests time just a dummy function that's sleeping for a while Server for exposition and just some random requests So very simple example, but the core lines of code are here And the decorator here and to go there there are a kind of metrics So we got a whole pot of metrics for free from Python like Also cpu when it starts follow scriptures and then here are two metrics So we've had 39 queries so far which took a total of 18.7 seconds worth of time So that those three lines of code here are just how easy are these two lines Is how easy it is to instrument your code and you can then sprinkle that liberally across your code base So now we can look at OpenShift. So here we set up cAdvisor Which is what you should be using to get all your container stats And as a Prometheus as well running here Now what's interesting if you look at the config file, so this is the Just changes so it's a little longer than usual But this here has not been hard coded to know where things are It's actually dynamically pulling in everything from Kubernetes And then just putting out a node. It's finding all the cAdvisors and using them So you end up also discovering your cAdvisors also discovering all your machines You're knowing where to get the machine metrics So as machines are added and removed it'll automatically show up now here. I've just got oc Running on my desktop So there's only one node But this approach would work no matter how many machines I had And it automatically show up as machines are added and removed. So there's no management beyond the initial setup They've got some graphs here. So here's some Container stats stop putting out too much Here we have the node exporter cities and machine stats We can see how much CPU there is how much memory is being used the disk utilization on my Tree disks and the cD drive which is empty and all the file systems And it's also predicting here as well when my file systems will fill up As it turns out my home desktop nothing's going to fill up. So everything's reducing in size Just kind of handy for once And as well for me is itself also exposed as a whole ton of metrics Now, this is a very deep dashboard that I actually use for benchmarking from ETS But this is a sort of insights you can start to get off an application You can see all the key metrics about the from ETS to no less performance And this is a pretty well instrumented binary And so that's the main things I wanted to show you and so are there any questions There haven't been any questions in chat yet. So I'm going to ask a couple myself So And we were talking a little bit about this beforehand as well The how does the open tracing project surface in Prometheus and how is that related to that this project? Uh, so open tracing and Prometheus are separate because they're kind of different approaches to monitoring So from ETS, it's all about metrics. So time series. We're tracking individual pieces of information over time Open tracing by contrast is both tracing. Uh, so what that means is we're tracking a single request as a passes through a distributed system because if you're getting Into a tail latency isn't that sort of thing or hey a cashless hit here which causes all these knock-on behaviors Things can get very complex So when you've got something like open tracing you could produce a graph that looks something like this and actually see Oh, right Which servers we're all the slow blitz and tracing does that Whereas Prometheus would more tell you that hey 95th percentile latency is blah and then you just trace and say okay Give me an exemplar of when it's slow and I can dig down into it Then we are actually planning on using open tracing inside Prometheus Uh, largely for our queries. We want something in memory that will trace them So you can see hey, here's my slow queries and a few other components But it's kind of two different views of the same data because a metrics based approach Is going to look and say right you got all these requests coming in I'm going to throw away most of the information because it's not efficient to keep around But I'll be able to tell you for each pass any chp method What the average latency is what the bite sizes are and so on so you can get an idea of the general trend A logging based approach will be tracking every single one of those requests But it can't track, you know, every single function to goes into because that'll be too much bandwidth And then tracing is kind of a logs based approach but stitching together logs across machines to track a request for its entire life cycle that's kind of where the Kind of the tree different size of monitoring you have between tracing logging and metrics Well, the reason it comes up for me um as a someone asked me the other day about integrating You have Zipkin up here, but also integrating um Yeager into Sounds like that's not something you would do So The there are some similar coal sites You'd want like if you have a htp routing the htp router to your code You might want to both instrument for open tracing and say hey here's a span Which is what you know each of these here is a span And also at the same time increment of permitious metrics So potentially this has been done to be a little library that does vote But you wouldn't necessarily want to do that all the time But that's where the similarity is because they're fundamentally instrumentation but for different goals A lot of the coal sites might be the same Well, there's a couple of questions that are coming in um The first one is can you provide a comparative feature set for permitious compared with elk? In grafana Okay So that's kind of it's a number of different tools there So grafana is a dashboarding solution. It is what we recommend for usage with permitious and these demonstrations here. This is all grafana and um So that's what we recommend And we kind of seen them as partners. We used to actually have our own dashboarding system Called problem dashboard permitious, but we deprecated in favor of grafana because well Why develop one of those for someone else's a better one already? In terms of elk while the case cabana, which is a competitor to grafana. I believe their feature sets are relatively similar But then the bigger difference really because grafana is a different team is between permitious and elastic search and the difference is that permitious is a metric space monitoring system And the elk stack is normally for logs and these are two different views of monitoring And because with permitious instrumentation You might have inside your binary like a well instrumented binary is going to have hundreds to thousands of code points that are instrumented Uh, but it's only tracking like simple counters of how many times is this line of code executed And you can get that for basically all your system that works nicely and that's metrics based monitoring Logs for example, you couldn't have 10 000 pieces of information But every single request coming in because it would just be too much bandwidth both disk and network You might get 50 to 100 But you can get every single individual request and that's where elk shine is then the logs based approach So they're really complementary systems because the way I see is you seem like permitious or a different metric space system to Figure out roughly where the problem is and drill down and then you figure that okay I've drilled down and I can see things were slow. You've looked a bit It's all the billing end points that i'm going to go over to elk and see all right So who's been making billing requests, which ones are slow at it's these exact requests from these users go over chat with them So they're very complementary solutions. You need you need a metric system and a logging system I think you I think you just answered one of the next questions Um, do you see permitious as a solution a full monitoring solution for open shift? And I think from your previous answer is you need probably both Lodging and metrics You can also do a black box monitoring of permitious via the black box exporter So that's your very simple. Is it turned on probes like htp checks icmp checks that sort of thing But that's just something you can do permitious and works out But you might also need a tracing tool something based on open tracing You're also going to need profiling tools like your gdb your c prof and all that as well Source code so you can compare all these back to the source code to see where your performance problem is And there is no one tool that does everything you need a selection of these tools to tackle different parts of your problem We need a leatherman tool for monitor monitoring Um, there's another question here. Can all components you demoed be instantiated inside of open shift? If so Um, is there a git repo with a good read me to duplicate your efforts here? So if someone wants to okay, so all the components there is actually a blog post on the robust perception blog Which I use just to some of this And all the components can be done inside open shift. However the node exporter, which is our machine agent We recommend not running under docker The reason why is that docker is trying to hide things about the machine and the node exporter is trying to get information about the machine sort of kind of Uh conflicting there. So we recommend running the node exporter on bare metal You can run it inside Inside docker. You're just going to quite as good stats In terms of how to duplicate the efforts, you know, there's the blog post and jenice Just tweaking things to your setup. There's examples inside the primitive stock implementation directory, for example Um, shuresh is asking how can I constrain my instrumentation to class versus method level? So usually you would be instrumenting not classes or methods But like if statements are at that level because you're going in personally choosing What things are important to you from like a business or engineering level? If you're trying to instrument every method or every class that's getting into profiling and profiling because Ultimately in monitoring everything is in the bench So you've got this thing is happening at this time that this context for the context is the entire call stack the full htb request everything Logs works by taking about 50 pieces of those information and storing all of them for every event Metrics instead just temporarily compresses them Profiling just says give me everything And it's just so much data that you have to take Either the logs trade off the message trade off Or only turn it on briefly because it's going to kill your performance You can have to be careful about how much things you're monitoring Uh, but the really valuable stuff is more to do with business logic than every single method in class Because that's like cpu profiling at some point which is a different beast So there's another question um that just came in that's A great explanation of the class method one. Um Can you compare prometheus with heapster ocular? um, okay, so those are two different tools screen lock and uppermost so If you look at kubernetes itself The metric server provided by the prometheus format And the thing is though that that's not great if you're running a graphite or an infox db Although infox db now actually supports that as well capacitor But let's say you're running graphite and you want metrics out of kubernetes What heapser does it goes pulls all those metrics and takes them and Puts them out to graphite for you. So that's what heapser does. It's not a monitoring system in and of itself It's more an adapter for taking this data in the previous format, which is all fully open and putting it to a different monitoring system uh, ocular den is a time series database primarily So it's somewhere where you can store your data. So for example, kubernetes itself is only intended to be ephemeral Uh, it's just for reliability. We don't want to rely on distributed systems Which doesn't mean we're limited to the size of a single machine in terms of how much data you can store and how far back you can store it But then we'd hand off to something else Maybe hawk killer, and maybe we've cortex and maybe open tstb Maybe infox db for the longer term storage of data sort of kind of complimentary in that way Of course, hawk killer potentially you can use as well yourself. I haven't looked too much into it But I know it's meant to be a distributed monitoring system Restorage So, um, since you're one of the core contributors to prometheus, can you tell us a little bit about That what's coming in the next release for prometheus and what's on the road ahead? And where are you? Well, the big thing that's coming up like our point release like we had 1.7 came out yesterday or today And which is largely incremental changes Like with open stack services covering a slight tweak to kubernetes service discovery Um, you know, but the big thing is coming up is for me to choose zero which we're currently working on And that has a new storage engine, which is far more efficient than the current one And it's also one of the things I was working on is new stainless handling, which is a semantic thing Which is important like right now if a Target disappears like you stop a pod It'll take five minutes for the data points that prometheus suggested to stop being returned With new stainless that goes away much faster than that And so it takes only one she's scraping for so that's much better for alerting And and there'll be a few other changes as well. But prometheus two zero is the big one also the alert manager To just the zero seven zero release just out has a new ui, which has all been redone So it's much nicer to use So, um, where are you looking for? Help on the prometheus project Um So we there's lots of places to help So there's lots of integrations if you have some tool that you wish had Metrics, uh, you can add prometheus instrumentation to them or write an exporter if that's not possible for whatever reason So because we've already got 150 integrations the more we have the better And we also got the core repositories is like 25 ish of them There's lots of bugs there that needs fixing features that needs implementing in a variety of languages And as well like helps if you can help with user support because as the project becomes busier It's not always possible for prometheus team Which is about 15 people to answer all the questions and more people would help Look community questions as well the better and documentations always goes front of right a blog post and has to do something That's great because once again the core Like four core developers, but there's a 15 core like on the team who are committers and we tend to everything so It's like can you help with the ecosystem as well as the core? Well, there's a couple of folks asking more questions and gary's talking about integrating jagger with open tracing soon So hopefully he'll have something to blog about around that Um gear gear may is asking. Um, just prometheus plan to support long-term storage of metrics So as I mentioned just there a few minutes ago Previous itself is designed to be for reliability as the first goal and reliability means that your alerts will be received It does not mean that all your data is safe So really the self is more than a femoral cache. So it's not meant for long-term storage of metrics So the question is where do I store my long-term metrics like stuff from a year ago? So like new capacity planning The answer is we have apis. There is a remote write api That can just write out all the data prometheus receives as it comes in And then you can build the distributed system based on that to Do as much data as you want. For example, we've cortex is that If looks to be you're talking about supporting that as well and possibly some other companies And then the idea is that that will be there and because it's a distributed system It might be less reliable under a network partitions and so on whereas for retis itself keeps on alerting And we can also read back that that as well transparently So that's the basic plan is that we have apis for this And the apis are there now The list is experimental because you might need to make some tweaks to them But they fully work if you want to build something around that Cool Um, there's a couple more. There's lots of good questions today. Um One of them is is does prometheus just work with open shift or was there anything you had to do to integrate it with open shift? So because open shift is kubernetes, there was no work because it's already kubernetes integration for prometheus so No, we have integrations like the service discovery is the main thing which is figuring out where are all the pods where are all the endpoints Where are all the easy two instances? So we've got about 10 of those now like open stack was the most recently added one And we've got ones for baritone. Uh, we've got There's nerves nervous being added, which is an airbnb ting There is whatever the twitter equivalent that is as well as it's like 10 of these cross different systems as or Tink we've added gce by now as well But if you don't support if you don't support your team because it's a little too specialized or no one sent a pull request yet There is uh, you just basically chuck us json with a list of all your targets and that works too But everybody else is just normal. This is just another way to run applications Because service discovery is the only thing that's kind of special All right, I think that answered michael's question. Um, there's one more request for a little bit of a demo Can you demo how to to alert on an item since you have it? Okay, well, let me just Do this the easy way so we just pull up the demo Have an actual nurse here somewhere So this is demoed up robot's perception. Leo. I've done weird stuff there. Uh, a simple alert looks like this So we just go alert name that example alert always firing and just an expression and that's creates an alert And then if I look at 9091 I'm sorry 903 Which is the this is an old alert manager. I can see this alerts here And it can be routed from there To email to slack the page review or whatever, but it's two separate steps The primitius produces an alert which goes to alert manager The alert manager takes alerts from all the primitius servers And then lets you aggregate those together into a single notification that goes to page review So the principle being that if you have multiple from media servers looking at different parts of the infrastructure Even if they're in different data centers, you only get one notification for a problem And that's like a key feature for reducing your paging noise But there is I think there's some blog posts out there showing how to do it in full We don't have anything to hand right now All right I think that has answered most everybody's questions. Can you on your slide deck show a slide on how to contact you? I'm guessing you probably have one there or resources for That's all for me. But just go primitius at the most perception.io will get to me All right Because those are the official for me to just kind of like stare I just have one more question. Is it okay? It's a quickie. Ask it Um, I'm just curious how this is tested in open shifts. So I presume Prometheus has unit tests and there's whole testing infrastructure there Then it's further test into the kubernetes community And then from there presumably it goes into the open shift community and more testing occurs and then Open shift qe You know does productize, you know testing other productize bits. So is that how the testing account abilities Lay out here So actually, um integration testing is one of those things that we love help on the primitius Uh, so a lot of this stuff, uh, like it's tested by people doing canaries and so on But we are talking about how to do proper integration testing because a lot of the time we discovered bugs when there's bad code Okay, so that is something I would have helped us So just clearly it's the integration testing of primitius in the kubernetes Or in that case, yeah, but the problem is with something like azore service discovery We need to run azore with ec2 service discovery We need an ec2 with maraton. We need to set up maraton and having then all these integrations is a lot of work Figuring out how to maintain those So that's definitely something we would like assistance with for those who are you know that way inclined Okay, so I'm definitely inclined particularly on the integration testing. I mean In the open shift Yeah, yeah, but if you want to go the primitius developers list is there if you want to chime in there Because I know it's something we really want to do Because we've been bitten more than we would like by refreshments Yeah, and there's definitely Definitely focused on the the open shift team That can I can point you to michael if you drop me an email Um that have been playing with and working with and testing from ethios for use Oh Okay, so there are people that had testing this integration Yeah, I'm not I'm not sure it like we're not productizing primitius In any way at red hat, but I I know people are using it. So And have been testing with it. So it definitely drop me a note. Yeah, I'll see if I can hook you up with them And I'm just curious mentioned regressions and integration. I I mean I seek out problems. I'm curious So I'm just curious. Where is the fragility or where where are the issues? Is there like a So it's normally with well integration of the external systems is normally such a service discovery because a lot of them their api docs aren't great and semantics of some of their fields that they return out great It's not clear exactly what's going on and the error handling can be undocumented and we only discover it when it breaks Especially when you're talking like a cloud provider like, you know, it was or gc Ac2, you know, you only can only discover some problems when they happen to someone So those are very hard for us to test and also if we have to get an account with everyone, you know, we're an open source project On the other hand We've got things like non-export. It was a machine monitor. And so how do we test the free bsd integrations when none of us run free fb sd Or the open bsd integrations and all those sort of questions So figuring out how to test all this stuff like I know Because we're packaged in devian by market Ferrari that's discovered a few bugs in our locking stuff because it breaks on arm So that's not tests. We run ourselves because we're all 64-bit xe6 So being able to expand our testing infrastructure will be quite useful And especially your integrations with kubernetes and open shift because a lot of our users are on that Okay, and just one last question. I don't want to stretch this out to one I'm just curious on the relationship between prometheus and your new company or robust perception.io Just what the yeah So for me, it's an independent open source project. No one company controls it And I'm one of the core developers and then separately robust perception is my company and we do consulting and support for prometheus So that's a relationship like robust perception. We sponsored from convo years now And and you know we help the community in all sorts of ways we contribute development time And then we're going you know trying to make money off consulting and support off prometheus, which is largely working Got it got it. Thank you One more question came in and actually it's a good one maybe to end on now that prometheus is under the cncf What benefits have you found being part of the cncf? Is that helping grow the project? Yes, it does help to grow the project it gives us It's easier to talk to the other projects as well and all work towards the cloud native vision Because you know each of us are seeing Similar issues but in different ways around people getting ready for your idea of cloud native And how you have to move away from the older mindsets of when every machine was special as well like they help a lot with marketing and And just general advice on how to run an open source project because there's some very experienced people there Yeah, no, we've found that it's been pretty helpful too for lots of things and there's lots of new things coming under the cncf and they're all Part and parcel of what we're trying to integrate with it open shift and into the open shift ecosystem It's been really helpful for us to have that connectivity And will be will you be coming to the austin event in? No, no, I'm trying to cut down my travel offer doing too much last year ah But hopefully we'll get some of these demos and with jagger and other things Through the cfp process this time for kubcon in austin and we'll see some more integration Work done. I think someone from us is going to be there. Yeah, I'm sure I know at least the lexus will be there He can't get out of it So he's he's been demoing permeatious through we've for quite some time So, um, there'll be there'll definitely be a few folks So anyways, I want to say thank you very much great presentation I will be putting this and a pdf of the slide deck up on the open shift blog at blog that open shift calm As well as I will dig around for all the references for the blog posts that brian has mentioned and Put the links to those as well there. So it usually takes a day or two to edit the videos but thank you all for your questions and Take a look at the events calendar on commons.openship.org for the upcoming ones We're going to two days a week. So if you have a topic you want to hear about We're going to do Wednesdays and Thursdays at 9 a.m. Pacific. So let me know and I will course stock and bribe people to come and talk just like brian has today. So thanks again brian and We'll talk to you all soon Thank you


All right, I know pilot Pete has to go and fly the plane So I don't think you're gonna make it to the end of the episode with us But you brought a cool stuff out. I'd love to have you tell us about it Pete. I did. Thanks. Yeah, and I'm actually gonna get to go to my daughter's commercial ops class. That's where I'm going. Oh, that's even better So I'm not sure I could pass the class. I've only But the things they discuss so anyway cool stuff found so Not trying to disparage the air pod pros loved them absolutely loved them and When about a year and a half into a year a year and a half into my second pair The right ear pod air pod started going and when I would talk I'd get a buzz in my ear And then it got so bad that even when I wasn't talking it was buzzing when I was trying to listen So that's no bueno. I really don't want to drop another two and a half airplane dollars in two into a pair of Airpods and so I started looking around is there a replacement and I found these these are the A and C which is active noise canceling skull candies And I have the little case that I got on Amazon with with an air tags I don't a reminder that we've got people most of the audience listens So if be please show it on the video, but explain what you're showing it up Yeah, so I'm holding up a case that's slightly larger than your pods pro case But it's a charging case and it does she charging wireless charging It also has a USB-C on the bottom so you can do wired or wireless charging And unlike most other in ear Earpods air pods These also work independently like the air pod You can put the left ear in and listen or you put the right ear in and listen and have a phone Conversation with either ear so that that was a huge selling point to me because I've got another pair that I really like But only the right ear works independently and then of course active noise canceling here through ambient sound it has three other modes too So it's got a podcast mode a music mode and a movie mode and then there's a little bit of a they call them ear gels, but it also so in in Addition to the rubber portion that fits into your ear There's a little bit of a rubber gel portion that helps hold it into the ear as well like a bum on top of it Yeah, yes. Yeah, they don't quite feel as substantial as the AirPods but other than that I got to say I've been super satisfied with these the sound is nice The quality has been good the the independent wireless the wireless charging the independent operations and Here was the big part part. I got them for seventy nine dollars Yeah, that's so that's less than an airplane dollars speaking with I got to figure out. I'm we're in the wrong Business here John because we're gonna start earning airplane dollars. That's the key Dollars How do those relate to kid in college dollars, that's the part that I need to figure out so Yeah, those are many hundreds of airplane dollars. Let me tell you Twice right now. Yeah. Yeah, my sympathies to your wallet. So yeah But based on that I'm gonna duck out of here and say thanks again for having me gents Yeah, man, I'm gonna do the commercial operations and see my daughter. So enjoy my friend That's what's paying for college. That makes it all worth it. So yep for sure