excited to be here. More than I thought you would be. Wow, okay. Well, very good. Well, it is my pleasure to be here. I am Adam. You can find me on the interwebs. Oh, wait, hold on. No, you can't. Now you can. You can find me on the interwebs on the Twitterverse. Please feel free to post right at me, whatever. I don't post a lot of things that have a lot of real relevance or meaning to anyone but me. So if you're going to follow that thinking, you're going to get a lot of really smart, intelligent things from me after something like this talk. You won't. But in the meantime, you can definitely find me there. That's a great place to go. Now real quickly, I work for a company called Zeal. We're a web and mobile applications consultancy. But if you can kind of tell from the name, we're kind of a lot about the energy of things. And that's kind of what this talk is a little bit about. It's not just about positive energy or positive thinking. Really, it's about things like difficult conversations. And more specifically, what this talk is, is a practical guide on human psychology. And I'm highlighting the practical part here because more specifically, if it's not able to be put into practice, it's very, very hard to really create any meaning in your own life. Now, what I discovered was that clearly I'm a person that has a big and extroverted personality. And that was not that was something that I found that was really fearful for some people. And that really was kind of like hurting quite a bit to me because I realized like there were people that wanted to say something, wanted to talk to me as an employer. I had members of our team that didn't feel comfortable coming in and talking to me. So I had to take this real kind of reflective stance back on things. And in doing so, I realized there were a lot of things I was doing that were not helping the situation. But similarly, there were simple things that they could have done to help me as well better understand what was going on for them. And so this talk, again, is about the human psychology after a bunch of research that I had done and practices that I'd put in for myself, as well as helped other members in my life, not just on our team, but in my own personal life to help us to communicate a little bit more so. But we're going to start here. And that is, we're going to talk about state. Now, I'm not talking about the OO object-oriented side of state. We're not talking about programming here. I'm talking about things like state of mind. And specifically, the things that we do that empower us mostly start from here. And that is state. The state of mind that we are in moving forward. Now, this can be broken up into three different things. And we're going to start here with physiology. So here's what I want you to do, is I want you to take a moment. I want you to put yourself in a semi-neutral position in your chair.  And just breathe in and breathe out. Just simply and easily.  Exactly. But I want you to just breathe in and breathe out for a minute.  And you were put in a position that was very uncomfortable. I want you to just visualize that. And with every breath you take in, I want you to breathe that moment more and more into your body, physically into your body.  I want you to just breathe that moment in for a moment. Think about what you're visualizing in your mind. And think and analyze your body.  And where does this come up? Do you feel it in your toes?  Where do you feel it? Physiology can change everything. So go ahead and open up your eyes. Raise your hand if you identified a spot on your body in which you could feel the confrontation, the anxiety, or anything along those lines. It was easy. It was relatively palpable. Now, raise your hand if you could not. Okay. So the next quality we're going to talk about real quick is language. And specifically the language that we use and the story that we tell ourselves when it comes to this moment. Right? So I want you to think back just to a handful of seconds ago. And I want you to think about what was the language you were using in that moment to yourself? What were you telling yourself? Were you saying that, oh, I'm weak. And whoever this individual is is going to beat me down? Or was it the opposite?  Right? What was the language that you were using? Because this is really important because the language we use ultimately defines the story that we tell. And the story that we tell ourselves ultimately creates the belief systems that we're going to have as we go into a given situation. Now, here's the beautiful thing is we can control all of this stuff. Because the last and final bit of state is focus. And the thing that we ultimately choose to focus ourselves on is the thing that will drive us and empower us forward to either be able to handle the situation, get out of a bad situation, or turn a situation around from something that isn't good and constructive to something that's productive and is. Right? So again, it goes back to these three items. First is physiology. The second is language. And the third is the things that we focus on.  And in reality, in that order, I would be willing to bet you that more often than not, your physical state is the most radically affected by your physical change. We know this in terms of human psychology when it comes to, you know, if somebody is standing over you or the proximity to you, that physical difference has a lot in terms of state of mind. It can very easily affect those things. Now, also, we've learned over time, especially, well, let me ask, show of hands, how many of you felt bullied when you were a child under the age of 18 felt bullied? I'd be willing to bet that most of the time in that situation, because this is a predominant tactical strategy, is physiology, physical difference. Oftentimes, we're bullied by individuals that are substantially larger than we are at the time. This is part of the reason, or one of the many reasons why this happens a lot in middle schools, because that's when growth spurts start to happen. And similarly, the things that, the language that gets used in context at that time can either be really divisive and derogatory. And also, for some individuals that might be going through change or emotional transition, that can be a really, really hard time for them. And the language that gets used can really affect that. So those two are really the predominant drivers that affect state. The third is the one that we often control the most, but it's a derivative of the first two. So if we're in a negative physical place, the language we're using is highly negative. Realistically, we're going to focus on that, the negative. The next bit that goes into this was the ultimate driver and the ultimate like powerhouse was connection, right? Ultimately, when it comes to human communication, this is kind of the goal, is we want to connect with the other individual. And this is the thing that was the most harmful to me personally in this situation. I was like, like, ultimately, what this is telling me is like, I'm not able to connect with these people. And that's like really sad, because that's anything but what I want. I absolutely want to connect. And what I was losing sight of is that all the other things that were affecting their state, I was impacted. And that wasn't okay. That's what made this conversation with them very difficult. Now, if you were here last year, or if you've seen another talk of mine online, I talked a little bit about these, and that's the human needs. And these are the six human needs as defined by America's guru, quote, unquote, Tony Robbins. So I am entirely ripping this off. But I found that this was really resonant because it put a lot of things into perspective for me. Going over these briefly, basically, he quantifies that all things that we do in life are every action that we take in life is to drive towards the fulfillment of these six basic human needs. The first top of the list here is certainty, uncertainty, significance, love and connection are referred to as the basic needs.  I want to know that the taxi cab is going to arrive and take me where I want to go, or I know that I want to get a pay check. I know that the people around me care about me. Things along those lines, certainty. Uncertainty is the inverse of that. But it is also a need, right?  You're like, I'm all in, y'all. You are clearly driven by uncertainty, right? Or if you're an individual that struggles a little bit with the managerial responsibility, right? And it might be something where significance is a major driver for you, that it makes you feel significant to be a person of authority in a situation, right? Or similarly, that you feel significant with somebody that you love and appreciate, friends, family, spouse, someone along those lines. And the last is relatively obvious, but it's just the simple act of love and connection, which I said just a moment ago was something that was threatening to me and the most disruptive for me personally was a realizing, realization that I wasn't having that with the people I cared a lot about. Now the fulfillment of the first basic human needs, these top four, generally the way in which we fulfill those are via egocentric means. In other words, I, to fulfill the sense of certainty, uncertainty, significance, love, and connection, I will do things that are for me specifically. It's not about other people around me, it's about me, right? But then there's this bottom chunk, these last two, and these are referred to as the spiritual needs. It's growth and contribution. Now here's what was found is most people, when you ask them, what are the two that drive you the most in your daily life? Not the ones that you want to drive you in your daily life, but the ones that probably do, most people responded with one of two, certainty and significance. But when they were asked if I could be driven by any on this list, regardless of whether or not I do that today, which ones do I wish I was driven by, most of the time they say growth and contribution. Now here's the beautiful thing is, when you were driven by growth and contribution, realistically, you were probably more prone to serve others to get there, right? There's a lot of individuals in this room and I'm actually very honored to know many of you that have contributed greatly to changes in our entire ecosystem, as well as helping individuals enter into this ecosystem, either via code schools or extension programs, things along those lines. And I'd be willing to bet that most of the people that have founded those organizations and continue to drive them forward are, if nothing else, driven by the sense of contribution and that they want to do that, and realistically, they had a great sense of personal growth that occurred at that time. So remembering these two things, it's also really fundamentally essential that you remember that every single person in the room and every single person in our life is realistically working to fulfill these six. Every single one. Every single one. Even the craziest asshole-ish of people are probably working towards the top four.  I'll bet you can pretty quickly pick which one is driving them. Now, look at the greatest people in your life and now identify which one are driving them. I'd be willing to bet asshole's top nice people bottle. That'd be my guess, right? Now, ultimately, all this comes down back down to is connection. So we're going to talk about something that is very controversial to some people in this room, maybe none of you, but we'll find out. And that is mirror neurons. Now, mirror neurons is, to a certain extent, it is not fully validated, but I find it very fascinating. And basically, a mirror neuron is a neuron that fires both when a person acts and when the person observing the act feels the same thing. It is also referred to as the empathy neuron. So there was a study done by neuroscientists and there's a neuroscientist at UCLA, his name was Marco Lacoboni, who has argued that mirror neuron systems in the human brain help us understand the actions and the intentions of other people. In the study, which was done in March 2005, he and his colleagues reported that mirror neurons could discern whether another person was picking up a cup of tea or planning to drink it based on these three fundamental principles. The first is context, right? This whole thing, to a large extent, is effectively just pattern recognition, right? So the first is context, right? If you put it inside of a scenario, the top example is what it looks like before tea is an example here, this is for Sam. And then the bottom example is what the situation looks like after tea. If you look exclusively at the action that the individual is taking before they're interacting with everything else around them, effectively very similar, but when you combine the two, ultimately what happens is we derive intention, pattern recognition, right? Now, here's what's interesting is, again, going back to mirror neurons is, to a large extent, this is the reason why this conclusion is drawn, is this is what helps us understand the world around us and act accordingly. And what has been found in situations like this is those that might be on the spectrum of autism often have challenge combining these three together. And that's where some of that problem arises in terms of processing and so forth. But here's what I want to do.  Stand up and find a partner, yep.  All right, now here's what I want you to do.  All right, excellent. Once you've determined A and B, put your hand up. Excellent. Okay, very good. So here's what I want you to do. Person A, I want you to listen very, very closely. Person A, raise your hand if you're person A. Excellent. Person A, listen very, very closely. Here's what I want you to do. I want you to maintain totally neutral, unemotional, unattached, like nothing. You are blank slate.    You got me A? Say yes. Excellent. Okay, person A.  Person A, did you not commit that? Yes or no? Yes, you did. So you will do this person A.  Look to B. Person B, look at person B. Okay.  Be neutral. No emotion. Just be neutral. Person B, I want you to smile softly at person A. Smile softly at person A. Five more seconds. Just smile softly at person A. Okay, you can stop your gaze. Okay.  Okay, now here's the thing. Person B or person A, you've already been given the hint here. You are mentally ready for this. Am I right? Wrong? I'm right. Okay, so I want you to do this again. So person A, you were going to become, wait, yes, totally neutral. No emotion. Hold true. Person B, I was wrong the first time. Yes. Person B, totally neutral. No emotion. Plain slate. Okay, you've already been ready, you're already prepared. You've got nothing being thrown at you. Okay. Now person A, look at person B in the eyes and smile gently. You got five more seconds. All right, very good, very good. Okay, give yourself a hand. Okay, so I want you to be very honest with me. This is not a competition, but I want you to be very honest with me. I want you to raise your hand if you were a person that was able to maintain totally straight face, no emotion. Totally straight face. Okay, now look around the room. So we're looking at like what? A percent of you maybe, right? What's wrong with the rest of you? Right. Yeah, those are the sociopaths. The point of this is the point of this is that the reality is, is that when we are put in a position where we are to witness the things happening around us, this is really so my background is in theater. We're in the right place for that. And this, there's a strong argument that live presentation, whether it's live music, live theater, live performance of any kind, live, live, live is more fulfilling psychologically in ways that we might not be able to acknowledge right away because of an example such as this, which is mirror neurons. This is a very simple and rudimentary example that when we witness things, our bodies react. Now, of those of you that we're not able to hold it, hold it straight faced, how many of you got joy or pleasure or positive emotions out of it? This is very, very, very important when it comes to the interactions and the ways we connect with other people. When we do simple, simple things like illustrate to them the type of relationship and emotion and connection I want to have with them, the probability is that they will feel that. Right? Managers in the room remember this when you're going into meetings with your team. The way you set the stage when it comes to just something this simple can radically affect the psychological output of that entire experience for the other person involved. Huge, almost immeasurable. Now one more example we're going to do and it's okay, you don't have to worry about that. Can you laugh uncontrollably for me? That works. Why are you clapping? Because there's crazy people in this community. Right? Absolutely. Okay. So this is the thing is that again another example of that. Think of all the really cute baby videos that are out there just little kids laughing that we watch on YouTube. Why does that matter to us? It's because of this same reason. Keep this in mind. Remember this that empathy is the currency of connection. The more we empathize, the more connection we have. So remember that. Empathy is the currency of connection. Okay, so let's get actionable here and let's talk a little bit about navigation in terms of how we navigate through this stuff like things like difficult conversations in our life or difficult situations one way or another. To remember this, that my feelings are based on my perceptions, right? My feelings are based on my perceptions. There's a term in neuro-linguistic programming. I was going to use the NLP, but that means something different to our group. But neuro-linguistic programming, that is the, it's the concept of anchoring. Now, how many of you have a negative association to just this image? It's just an image. How many of you have the exact same alarm clock? Right? He's like, that's my alarm clock, dude. Where's my credit? I know, totally. This is true is that that there is, oftentimes what happens for us is we have anchors. If you look at all of the negative experiences that you've had with interacting with other people, I would be willing to say, I would be venturing a bet that realistically, those interactions that you've had are, are making it harder for you to enter very similar situations. There's an example that's given when it comes to those at a funeral. And the simple example is this, that one of the things to be mindful of is when people are in a really strong emotional state, right? Especially something that's highly negative. Physical touch, remember, physiology affects state and that physical anchor can lock itself in to where quite literally, when I touch that same spot, it can trigger subconsciously emotions to be discharged for that individual. And as an example, if I was to go up to somebody who had a grieving, who was grieving the loss of their father and I was to walk up and say, I'm so sorry for your loss and touch him on the shoulder. The next person walks up, walks up and says, oh, I'm so sorry for your loss and touches him on the shoulder. Five other people do that. You have any idea what could possibly happen if that happens again, right? Now here's the challenge with this and I realize that this is the other side of the difficult side of conversations that exist is you may have no idea what anchors exist for another person. In fact, it's highly probable that you'll have absolutely no clue and that they might not even know. But you have to know that that is very likely for them. So the only way to really dive in and understand what the balance of that looks like and what all those components are is to understand the story. Because ultimately, while my feelings are based on my perception, my feelings can be negotiated. They can change. I can change the way I see and feel things, especially around the things that I am most connected to. So preparation for conversations like this and situations like this is really essential for both individuals that are involved, right? It's really essential for a lot of reasons, but mostly for those three reasons I was giving before, physiology, language and focus. If you're not in the right mental state and you walk into a situation and you're focusing on the wrong thing, then your preparation and that conversation may very well go the wrong direction or at least not be as predictable and as certain that you might want it to be.  And that's perfectly fine too. But the first thing I'm gonna go over is breathe. So here's what I want you to do. I want you to take a second. I want you to put yourself and sit in a neutral position again. I'm gonna teach you a very simple breathing exercise that you can do at any point in time, anywhere in your day, and it will literally only take you less, it would literally take you less than a minute. Okay, so here's what I want you to do. I want you to just sit there for a moment. If you need to put your hands on your legs, that's fine too, but just be as neutral as you can. I want you to breathe in on a count of five. I want you to do your best to hold it for a count of 20. Do your best and then let it out on a count of 10. Just do your best, okay? Count of five, ready? One, two, three, four, five. Count of 20, hold it. One, two, three, four, five, six, seven, eight, nine, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20. Let it out on 10, nine, eight, seven, six, five, four, three, two, one. You do an exercise like that five times. You will change your physiology, right? So it's really essential that you get into this mind, this head space before as you're preparing for this so you can stay in the right state of physiology and ultimately move to the right state of focus that you need, but breathe is number one. The next is map your assumptions. So oftentimes in conversations or in situations, we make a lot of assumptions about the other person's interpretation of what we did via our own interpretations. So just like navigating a map, it's imperative that you understand and map the assumptions that you're making about a given situation. I don't have the opportunity to go into too much detail about this, but if you have questions about this when I can go into it a little further. But basically what you're looking for is what are the things that are not driven by fact, right? What are the things that you heard, feel, kind of thought happened? Like what are those assumptions? If you're drawing the conclusion about another individual that draws you to the belief that oh, they don't love you, they don't care about you, they don't value you, more often than not, it's based on just assumptions we're making and most of the time from what I can tell it's based off of previous experience it's drawing us to that conclusion. So it's really, really essential that we map our assumptions first. The next is to map your fears. What are you afraid of if this conversation goes awry, right? What happens if this doesn't work out, right? Same thing to mapping your assumptions, you wanna do the same thing with this. And I literally will create a list. So I keep an index card on me most of the time. Let's see if it's on my pocket, except right now? Yeah, there it is. So this is for this talk, right? And it's very important for me that I write down specifically like what is my goal, like what is my outcome that I have for this talk, like if nothing else, what am I gonna land on and what are the other things that I wanna avoid mentally, right? And it's really easy, it's just an index card. I mean, I could probably make an app for that. Again, we're in the right community but it's an index card and a pen. I can do this just about anywhere. The next is what do you want to learn or understand about the other individual? This is also very important. Going into a conversation where you're seeking to learn automatically moves you from the top four basic needs into the bottom two of growth and contribution instantly, just like that. If I'm looking for a reason to connect with you because I wanna learn something about you, even if you're the most detestable individual I can imagine and fathom, it immediately puts us in a space where we can be really constructive and productive. But remember this, understanding does not mean acceptance. Just because you understand where they're coming from doesn't mean you agree with it, okay? So remember that. And this can be very helpful again when you're in a situation where things are tough and there's a lot of people in this world that I can't imagine sitting down with because there's nothing about what they think that I agree with. But if I go into it trying to learn something, then at least it can start to be constructive. And then the last is what I said, told I wrote down on this card, which is know your outcome. What's your outcome in the conversation? Even if somebody's coming to you, what is your outcome? For me, my outcome most often, not especially right now is to connect and learn. I just wanna learn. What? I don't know yet. Well, I don't know what this is gonna be about, but that's okay. So I'll just connect and learn. Now, you've done all the preparation, it's now go time. So we're gonna breathe again. Make sure you breathe. If you're going into a situation like maybe a salary negotiation, or in my case, we worked a lot of clients where they've got budgets or they're gonna try and talk us down on numbers. That's my story, right? These are the types of things that I need to make sure that I've got my head in my state in the right place. The next is to make it safe. This is really important as well. And as somebody that wants to meet somebody where they're at, I will try and be the one that puts myself into their situation, their physical situation. If it's their office, their location, I will call them, I'll put myself in a place where I'm mentally ready to have a conversation with them, but I'm gonna make it safe for them. And similarly is to ask them or to try and pick up what makes them feel safe. And if at any point in time, you feel like you're not achieving that or there's something that's not going, that's going awry, it is perfectly acceptable to acknowledge that and perfectly acceptable to reschedule in life. That's okay too. Then the third is to explore the story. And this is oftentimes in a situation where there's confrontation or conflict. It's very important to remember that belief systems are a derivative of old experiences most of the time. And if you start to explore the story, how somebody's understanding an action or a thing that's happening around them, then it's easier to empathize with them. Again, just because you understand, doesn't mean you agree with it or accept it. But what it can mean is that you can understand it and help identify how you can work and grow together. Also very important is to acknowledge their contribution and acknowledge their feeling whether you agree with it or not. By simply acknowledging it, it just helps level the playing field. It's like, you're human, I'm human, we're just, we can find a way to work through this and I recognize that what you feel is what you feel regardless of whether or not I agree with how you got there, that's irrelevant. What's relevant is that you feel that way. And similarly is to recognize the contribution that somebody that I have made to this positively or negatively and to allow them the opportunity to do it, not necessarily do it for them. So I can say, I feel like I've contributed to this because I just didn't give you the chance to speak your mind at the time. I feel like I must have cut you off or just kept this from being constructive and I just want you to know that. I'm acknowledging that that could have been a major player in this. Then inquire to learn, ask questions, do as much as you can to learn what you can from the situation that you're in and then ask open questions and here's some examples. So instead of you're full of shit, I could say something like, can you help me understand better your perspective? Right? Can you say a little more about how you see things? What information might you have that I don't have? That is really helpful and that's something that I realize a lot is I'm privy to in our company, I'm privy to information that I forget other people don't have.  Well, it's an easy question to ask. What information do you think I have that you don't have that could help you? What impact have my actions had on you? How do you see it differently? Again, not that I have to agree with it, but just I want to know, I want to explore the story. Last two things and then we're good to go. So, confrontation. In the event that you experience confrontation with another individual and it happens a lot, raise your hand if you've ever experienced a confrontation with somebody. Keep your hand up if it's been physical. Yeah, totally. It happens to a lot of us, more than the psychopaths in the room, I'm just kidding. Yeah, this is true that confrontation is really, it's a very powerful thing that most of us have experienced in our life. So I'll give you a couple of real quick things. Number one is physiology, language, and focus. If you can change those three things, then oftentimes the confrontation will disperse. A simple one is in the event that like, anytime we're having an argument, so anytime my spouse and I get into a case where we're just like, war button heads pretty hard, the very first thing we'll do is change location. Very first thing, just physically move, right? That will more often than not, at least bring the defuse it down to half or some arbitrary percentage, just help move that along. Language we sort of talked about, but oftentimes when I start to ask the question to help understand them a little bit better, that gets their language identified, that helps them identify what language they're using and helps me understand as well, and I can start to mirror the language so that we can again have this sense of common ground. A lot of this stuff are the things that we just experience. Again, this mirroring model. And then the last is focus. So experience this with family members that struggle with depression or struggle with anxiety and again, most of the time it's what they're focusing on creates this downward spiral from them. Yes, there's many other factors as well, but that is also one that can be very helpful to kind of address and change. So if you can help change their focus onto something different, and I don't mean make a joke or something along those lines, I mean just help them see another way as an example, we do a lot of hiring as do many of the companies that you work for. And I find it's very common for individuals to come into an interview and they're focusing on their lack of experience. And my favorite question is asked is, yes, but what are you the most experienced at? And everybody's got an answer, everybody does. And it's just a change of what it is that they bring value to in just about any situation they're in. So it can be just as simple as helping them change their focus and similarly language, right? The very last thing is problem solving. So the goal in all of this is to get to a solution of the problem. And similarly, it starts with the breath, it starts with understanding and it ends with getting to a point of clarity, right? Once we can communicate, we can identify what a good healthy solution would look like for the problem that we're talking about. So identifying this, so a couple of small things is identify a standard of fairness, right? This can be really easy because I believe that for the most part, most people, there's definitely outliers, but for most people we want something that is fair. Sometimes we can't articulate that nor do we wanna consciously admit it, but it is something that we want. We do want something that's fair. And similarly, unfortunately, compromise to a lot of people has become something that's a negative anchor, that it doesn't mean something good. So we won't say that, we'll say fairness. Like what is fair? And if you can define what the standard of fairness looks like, then now you can strive to solve a problem that is within those boundaries. And it's kind of hard to say that, well, we've discussed what's fair, we've set that tone for what becomes fair. Now, once we meet that, isn't that good? And if we can be good with that, then it can resolve. Sometimes this happens over many, many sessions. Sometimes it happens inside of a minute. Time is irrelevant to this, but when that becomes the goal of finding something that's fair and reasonable for everybody that's involved, it's pretty easy to get there. So this is difficult conversations. I know that you've all experienced it. I'd be more than happy to talk to anybody else if you have questions about it, some tools or a situation that you're in that you'd like some maybe thought or I would love to learn more about your story, okay? My name is Adam Cuppy, I'm from Zeal. You can email me if you need to. It has been my pleasure, thank you all so much. Thank you.


Now we're going to look at SunSpot Rotation and just to show you real quick where I'm getting the data from, this is from the SDO missions website under the data. And when I browse the data, I can put in a start and end date, which particular instrument I want, whether I want high or low resolution, and whether I want every image or just like every fifth image, which speeds it up a little bit. I've already gone ahead and submitted this because it takes a little while to download it. And this is the movie that's playing. So this is showing a high resolution image, which is why it overfills the screen. And it shows the sunspots as they're moving across. And again, this is sped up, so it's going sort of in fast motion. Now for my lab students, you have to watch this several times through and note which of the behaviors I mentioned you can see. I'm going to go ahead and let it run through here several times before I stop the movie. But you can also replay this YouTube video if you need to.


You may have noticed that some of your subscribers have a personal email address, like Hotmail or Outlook, that they're using to sign into their subscription. This happened because of the activation processes that were in place for the legacy subscriber experience, causing the sign-in email address to be different than the notification email address. How do I fix this? Simply select the Connect Emails button and the system will attempt to match the accounts with Microsoft accounts to existing users in your organization's Azure Active Directory based on matching the first and last name. If there is an error, you can remove the match by clicking on the X. You can also use the Search Directory functionality to correct the errors or fill in missing information from your Azure AD. If all the matches look correct, you can choose to select all matching subscribers rather than selecting them one at a time. Next, click on Continue, which will take you to a screen outlining the changes to take place. If you agree, click Save, and the changes will be made. Your subscriber will also get a message informing them of the change. Something to keep in mind when you edit the sign-in email address is that this only updates the email used to sign into their subscription on my.visualstudio.com. If the subscriber has already activated benefits such as Azure or Pluralsite using the other email address, they will need to continue to use that email address to access them. For any new benefits that they access, they should use the new email address. If you have any issues, contact support aka.ms forward slash vss admin support.


Oh, there it goes. Turned over to right on the dot, the time o'clock. All right, welcome. We're going to talk about Cinder Performance, which it's kind of a misnomer, but performance of storage in OpenStack here. I'm Ed Balboff with SolidFire slash NetApp. My two colleagues here, David Black out of our North Carolina lab. He's more NetApp than SolidFire, but he's coming to the dark side. And John Griffith, who everybody knows. He's infamous. I'm sorry. So the agenda we're going to go through here, we're going to talk. John's going to talk a little bit about LVM performance, and he's been doing some work there. I've been doing a lot of kind of investigation around the hypervisor, the VM, those kinds of things. David's got some stuff he's been doing on storage networking, iSCSI sessions, and QoS. And then we'll come back up with a conclusions and maybe take a couple questions. So David and I are going to step off and let John go through his cup bits. All right. So I wanted to kind of talk a little bit about LVM, mostly because as some of you know, LVM is the reference implementation. Gets a lot of flak. People talk about how it's no good, it doesn't perform, and I need raw disks, or I need that, whatever. So what I did is I put together a little bit of information to talk about some of that. So the first thing is, as I said, everybody always says, nobody in the real world actually uses LVM, or why do we bother with LVM? Nobody uses in real life. And my response now is don't make me flip this table. That's a new Cinder thing. We flip tables when we get angry or upset, so. The point that I always like to make to people though is this is the last user survey. If you take a look, you will notice that the second most popular backend in use is LVM. Not only that, it's actually growing cycle over cycle. So to sit there and say that it's not something that's used or gonna collect it is probably not really a wise choice on our part. So that being said, there are a few misconceptions out there about LVM. A lot of people seem to have the impression that LVM adds some overhead and slows down your performance. The other one is that LVM does not have an HA solution. And then my favorite one that has been kind of purported around for a while now that drives me absolutely crazy is that I must have raw disks in order to get performance to do things like Hadoop. Completely untrue. So the facts are, LVM and even with an LIO target on top of it, they really don't have much of an impact on disk performance for standard workloads. As far as HA solution, there's actually a number of options. Cinder has had a mirrored LVM option inside of that driver for a number of years now actually. On top of that, there's DRDB now is in tree and you can use that. There's still some work to go before I would probably put it in production and fully trust it. But they've got some performance data out there and things like that. And it's actually coming along really well and it's a pretty good option. And then back to the last one, I just don't even know what to say about that anymore. So Pishposh, I took that from Simon. Now, I'm not gonna just go crazy here and say LVM is the greatest thing in the world and everybody should use it and stuff like that because that's not true and we all know that, right? I'm not a zealot much but it is pretty good and it should be something that we continue to consider, I think it's cheap, it's free, it's easy. So I think it's a perfect fit, especially for a reference implementation. It also behaves more like most of the backends in Cinder. By the way, there's like 85 of them. So again, not saying everybody should beat the LVM drum and go down that path. It's not right for everybody. It doesn't fit all use cases, absolutely that's true. But I am saying we should continue to nurture it. We shouldn't just dismiss it as a thing that's only good for the gate, right? When you get into large scales, production environments, service providers, things like that, it may not have a place for you but then again, it might. It's a fantastic solution for TestDev. It's a fantastic solution to augment something that may be higher performance, higher reliability storage inside of your private cloud. So those are the types of environments that's really good to look at. And then of course, you can't beat it, right? If you wanna just do a POC of OpenStack and do some evaluation, it's cheap, easy and free, right? What more could you ask for? So it's kind of a good deal. And in that respect, I say if that's gonna be a potential OpenStack deployers or users first taste of Cinder or first taste of OpenStack, we should probably try and make it as good as we can, right? It's kind of important. So what I did is I put together some basic numbers. Nothing really like crazy or anything like that or intensive. So rather than just sit up here and try and tell everybody, hey, it really does work. I know it, I use it all the time and it's great. You should just believe me. I put together a little bit of data because everybody likes to say things without any data to back it up. So I took a machine with some raw disks. There's some Intel SSD 320s in them. I've got LVM version 2.02 130. It's running on CentOS 7. I've got a 3.10 kernel on there. You can see all the information at FIO 2.2. And I used RDO and PacStack in this particular example. I've duplicated this on a identical machine with Ubuntu 16.04 and I get almost the same results. I mean, it is within 5% for everything. So that's, by the way, using DevStack, which you're never supposed to do, right? So the FIO settings I used for these tests, I used a block size of 4k just because that's a common thing, right? And that's what most people do. LibAIO is what I use in there. An IO depth of 16.  I used 50 gig test data sizes and I used direct. I didn't use any buffering or anything like that. And I only used a single job. And the reason I did that was, particularly in OpenStack, you're hopefully booting up a lot of instances that are fairly small. One vCPU, two vCPUs, doing small things. So I want to make this somewhat realistic. And then I run each one of these for two minutes. The device, as I said, it's a raw disk. What I do is I start with that. I run this test suite on it. I run it 10 times. And then I go ahead and I put LVM on it and I run it directly to LVM 10 times. And then I do a striped LVM across three of those disks and I run that one 10 times. Finally, I put an LIO target on top of it, run it 10 times. And then finally, at the very end, attach it to an instance, go inside, run FIO inside there to the dev VDB, run it 10 times, get the average. You get the idea, right? So this test that I'm running is a group of tests. I do a sequential reads test, sequential writes, random reads, random writes, and then a 60-40 mix. The thing that's kind of cool is if you look, I was, actually, I didn't think it would be this good. I thought it would be a little bit worse than this. But if you look, compared to the raw disk, I'm not taking a hit at all.  It's fantastic. And it's really cool.  Unfortunately, I didn't have a bigger server available to me at the time. I'd love to have done something like this with, say, 10 drives, 12 drives, something like that. That'd be really cool to kind of check out. But anyway, you can see the difference, and you can see all the way down the chain to the VIRT IO, that's when we're going through the instance of BDB. There's not that big of an impact, so that's good. It's good stuff. The 4K random reads, again, pretty much the same story here. This one, you know, there is a little bit bigger of a difference that you can see on the striping. So that's cool. I'll give you a second to take a picture. Hurry up. There we go. All these slides will be on SlideShare. So if you want to get them, you're able to get them up there. Again, 4K sequential writes, same type of deal. And the random writes. I have a thin column in there. So FYI, I forgot to take that out. I had thin data in here as well because I thought, well, maybe that makes a difference. In terms of these tests, it really didn't have any impact or make any difference at all. Where it did make a difference is I started playing around with what happens if I have a whole bunch of snapshots of volumes while I'm doing read-write activity. And that actually started to make a significant difference. But that's for another talk. And then the mixed workloads. The mixed workloads, we, you know, this one was a little bit interesting. I expected to get better numbers out of this than I did, honestly. So I ran the suite, I ran my script like three times. And it just, for some reason, there's some really weird latencies that came up in there. I don't know if it's an FIO thing or my disk or, you know, I'm not really a performance engineer, so I did the best I could and sort of came up with pretty consistently. So at the end of the day, I just want to say, you know, you look at that and it's really, it's not bad, it's not bad at all. This little handful of examples I gave is definitely small scale, no doubt about it. But you can scale it out from there. And the other thing that, you know, that I've noticed over the years playing with some of this stuff is it's a fairly linear scale. So the more drives you add, you know, you can extrapolate it pretty easily. Striping is a huge thing. We don't actually do that in Cinder by default. You would have to set that up on your own. There's a really easy way to do it, set up your thin pool and use thin provisioning and have that thin pool striped. And I'll take care of it. You can also go ahead and mirror that thin pool or that striped volume group, whichever one, or striped volume, you can do both. So there you have some redundancy. Since this is iSCSI, 10 gig network, dedicated data network is essential to this. You start doing this on one gig and you'll be really disappointed. The other thing is, keep in mind in terms of scale, if you don't have a large number of disks and you don't spread it over multiple disks, you're gonna get completely hosed if you have more than a couple of users inside of your cloud. So it's kind of key to play with that and factor that into your equation. So everybody calls LVM the pig. So as far as I'm concerned, that'll do pig. All right, so John talked a lot about kind of the smaller scale stuff with the LVM and the reference implementation. I'm actually gonna take a look at some stuff I did with a much bigger setup here. So this is the quick version of the diagram. I actually had eight solid fire nodes, so I had 80 SSDs in there and a full, fully redundant 10 gig network going out to the compute nodes, the control nodes. But I did have them going through top of rack switches and a core switch. So you can see the diagram there. We had 40 gigs up through the core. The goal being not to limit ourselves with the network. So we're trying to make that as kind of production type as possible. A little more about the details here. I used, they were all Dell R620 servers. You can see the specs on the core count and the memory utilization there. I used a Fuel 9 OpenStack deployment. So the hosts were Ubuntu 14.04. The only change I really did with that was change the force config drive, driver typo in there. So I could vMotion stuff around so we can kind of test this stuff on, put it all on one hypervisor and put it all on three hypervisors. The image I was using is CentOS 7.2. David will talk a little bit. He used the CentOS 6.7. So there's some different tuning parameters. I'm gonna try to point those out here. So we've got some differences in versions. So that's kind of important. I'll talk to you about some conclusions there. And so I have a heat template to deploy the instances and the volumes. I can build a eight gig root volume, 20 gig test volume, and I can do this automatically to turn up how many instances I want. None instances and we can build as many as we got storage for. And then I automatically pre-fill all the volumes. So I've got a random non-compressable, non-dedupable workload that I fill the volume with before I start running my stuff. So I looked at hypervisor tuning, VM tuning, other things like that. I guess I'll tell you the conclusion to begin with really here pretty quickly is that a lot of this stuff doesn't matter a lot. It's in the, again, the older versions, there was more tuning as we've built the newer versions of OpenStack and the new deployment tools would kind of do a lot of this automatically. But you can see here, we changed, I changed the scheduler. I added the number of requests like rotational gets set like it should be. You can put all this in Udev rules. I actually had a little script so I could set these as I wanted to back and forth. So we'll show you some results here in a second. And then we look at the KVM and the VM itself for some tuning issues here. Interestingly enough, in the VM with CentOS 7.2, you can't, for instance, change the scheduler. So the scheduler is none. And that's the way it should be, right? In the older versions, you had to set that to no op, but now it's set to none. You can see the green things here are highlighting the things I changed. I meant to put that on the previous slide. Within the VM. And then for KVM tuning, all the best practices are around using Vert.io, using AIO for the driver mechanism or the AIO mechanism there and turning off the page cache so you're not buffering there. Now you may want that for some other reasons, but we set that all up, made sure everything was set correct. And then I started making a lot of graphs and I have a ton of data. So I'm trying to present the stuff that's most relevant here. So I'm gonna, in this one, I'm gonna go through all of the four of these and then for the subsequent sequential read or sequential stuff and random writes, we'll put a little less data up there. So you can see the basic conclusion here is this, you know, I've got tuned hypervisor, I've got the defaults and then I've got the tuned VM and the default VM. And really, not a lot changes. You tend to get a little bit better latency when you tune things and you push the knee out. So if you're looking at these curves, these are all latency on the vertical scale, IOPS on the bottom, and this is all coming out of the VMs, right? And this is running, in this case, it's running one VM. So I got some tests where I ran lots of VMs, but this is just like focused on one VM to try to see what happens as we tune these things. And again, you'll see the needs on these curves move a little bit. We set these all up. If you're familiar with SolidFire on the back end, we can do quality of service. I set some minimums here to keep these things sane, but their maximums are turned off. So they're all free to fly as much as they want. So when we look at this, kind of the summary on those slides, right? The three millisecond crossing, which is a value where you would start to see some issues potentially in your application, was at roughly 18K IOPS on all those graphs. It didn't change a lot. So the nominal one millisecond for the hypervisor didn't seem to change much, but when I started tuning things in the VM, we see that our nominal 1.75 milliseconds, we could cut that down to somewhere in the 1.25 to 1.5 milliseconds. So there's some tuning differences you can make here, but in your overall performance, it doesn't really affect it a lot. So if you look at the random writes, now I've got a lump in my defaults here. John probably ran something on a cluster while I was in the middle of running a benchmark. So we're gonna ignore the little lump because that should be a standard curve. And you can see the changes here. So we've got the three millisecond crossing at about 12K IOPS for everything, nominal 1.25 for the hypervisor. The VM adds three quarters of a millisecond of latency going through there overall. Again, these are all single VM performance. So we've got them here for sequential reads. This changes a little more. So you can see the curve in the VM is very, we just, there's no knee, it just kind of goes up, right? As the IOPS, and of course the bandwidth goes up, the latency affects that pretty quickly. And that's the conclusion there, right? The VM really adds a bunch of latency and you start to see that with these sequential things. And it cuts our performance down from about 275 megabytes per second to 195 megabytes per second, just that added latency. You can see on the, it's my right, but I guess it's also your right. So the hypervisor doesn't quite have that. It pushes the knee out a little bit. So those tests were actually run right on the hypervisor on the same volumes.  And similarly for sequential writes, again we got another lump in here. I don't know, somebody else will blame that on Chad who probably ran something in there, but a little less so, right? The writes are, the latency is buffered a little better here somehow through the stack. We queue it up and therefore we're not so dependent on getting the stuff from storage and through the network. But again, you see that the knee, and in this one there is a bit of a knee here on the VM, basically I have the data for the tuned VM but it didn't change that much. So I found a place to put my conclusions in the slides there. So that's my look at what happens with the hypervisor in the VM when we tune it. This was not messing with the network and some of the iSCSI session stuff, so that's where I'm gonna turn it over to David here because he looked at a little bit of that so I guess I'll let you bring up the next slide and then I'll come back up and talk about what happens when we scale this out to a lot of VMs. So, David. So we've seen a couple different environments here for some LVM testing with John and then some testing in his lab that Ed did. As Ed mentioned, I work in the customer proof and concept labs in North Carolina for NetApp. We set up multiple environments for testing for demonstration, that sort of thing. This was conducted on a certain environment that is actually our first one with OpenStack. So this is something my group's getting to know and I'm kind of spearheading that. But anyway, we use that for demoing, benchmarking. It's running OpenStack Liberty and it is using Mirantis Fuel 8.0. So I think I'm down a level from what you were doing there. That's because it's stable and that's what I wanted to stick with for now. We're using nodes Ubuntu 14.04 and notably IO equals thread. So some of the previous testing, especially with new versions of Sentos and Ubuntu and so forth, we're using native IO. But this is using threads just to let you know that. So this configuration is one controller and seven compute and the networking on each servers, we used one one gigi interface for Ethernet for management and two 10 gigi trunk using LACP per server. So this is how we got data in and out of the servers. Sender storage is configured as a multi backend and I got a list there of all the different storage backends configured, notably solidifiers. So that's what we were testing. Hardware looks like this diagram, mainly for solid fire testing for nodes and you can see some of the other gear there all connected to a Cisco switch. And we have eight IBM X3550M3 servers. So a little bit older servers and those will become relevant here in a minute. So what I was interested in finding out here with our test scenarios is to find out what happens when I increase the default of one for iSCSI sessions for open iSCSI on each of the initiators upwards. So my reason for this is I wanted to cause storage contention. So I wanted to see what would happen. I want to push the storage as hard as I can of course for a demonstration reasons. So the configuration has 30 VMs total and each one has a 10 gigabyte solid fire volume attached and on those compute nodes you might notice there were seven, we had one out of service right now. So I had six to use and five VMs on each of six. So they're evenly spread out. And the default host and VM storage settings applied for this test because I just wanted to find out what happened when I increased iSCSI sessions. The benchmark used here is VDBench. I used the benchmark load generator tool and call it four corners. We saw some results in ads where we have sequential reads, sequential writes and then random reads, random writes. And then the block sizes that accompany those. So my test here went from one to four iSCSI sessions. And then scenario two which I'll also show you was not for storage contention but I wanted to contrast it with one that did have contention. So I just had one VM and one volume on a single host. So the results I got for those one to four iSCSI sessions are graphed here. And as you can see, they're not super different but something to keep in mind if you can see those numbers and yeah, I think you might be able to see them there. Pretty high numbers. So the random read is the one I want to point out because we were seeing about 200,000 random read IOPS from the system at this point but with that increase to four sessions that gained us about 15% which you might think that's not a lot but that took us up to about 230,000 IOPS with random reads. Little bit less deterministic results with the other three but it definitely did have an effect. So this is something that I may investigate further increasing the sessions for instance two as many as eight on it. And then test scenario two this was with the single VM where we weren't continuing for storage bandwidth. So really not much difference there and things kind of bouncing around for sequential writes but got a little improvement for random read and a little decrease for random write. So in a situation where there is contention the storage is being heavily loaded I would tend to want to use more iSCSI sessions maybe four or more where I don't have that kind of contention it doesn't really seem to make a lot of difference. Okay part two of this got a couple other test scenarios. The first one is a back in QOS test and this is where I wanted to find out how can I keep one volume up in its IOPS how I kind of keep the response time down how I can get the IOPS up while I have some contending volumes. So here I used a nine to one ratio so total of 10 VMs and each one had a 10 gigabyte solid fire volume attached and all those VMs are on one compute host so this is actually over provisioned somewhat not a huge server. And here this time now I use the optimized settings that Ed mentioned a few slides back so using all these different settings trying to get the most I can out of the host and the VMs and so the scenario I used was to leave what we call the slave one VM unlimited so it could run as fast as it wanted and this could be say an important application a database for instance the other slaves two through 10 were throttled to 4,000 IOPS this 4,000 4K IOPS so I wanted to find out what was that one VM doing I think the pig we call it sometimes and the other one's piglets when all of this was under heavy load so same test run exact same four corner test at the mass rate and here I vary the I SCSI session counts and I say averaged here because I want to tell you this up front that in this case when I was on one host I didn't actually see a difference between using one to eight I SCSI session counts fact they're almost identical so I use that as additional test data and averaged all those for the numbers I'll show you here. Second scenario was using front end QOS so I'm going to save the interesting part here for a couple slides but I switched to using the front end sender QOS type okay so just FYI the type specs I use for solid fire if you've ever worked with solid fire on the back end these are what the QOS specs look like and here I set minimum maximum and burst IOPS so you'll notice the max and the burst are the same number actually and this is so I could try to get an apples to apples comparison with front end QOS so this means I had a really pretty much a hard cap at 4,000 IOPS similarly on the front end I set total IOPS to 4,000 per second and total bytes per second to about 16 megabytes which reflects a 4k block size okay so it's kind of one thing that was kind of unexpected and someone might know more about this than I do for sure but something that surprised me because I worked mostly with solid fire in the back end is that when I used front end QOS and I wanted to do volume retyping like to set up for another part of the test I got this error message so I couldn't change the front end QOS settings either to actually change the numbers or to reset it, remove it or add it so what I had to do simply is to detach the volume set them and reattach the volume which was fine for what I was doing because I was testing raw block devices but for instance if this would have been a mounted file system that would have been somewhat more complicated so something to keep in mind it's not required for back end QOS I can set those on the fly so that was kind of cool but I thought I'd mention it okay so under the results this is random read IOPS versus QOS type so looking at front end versus back end and as I mentioned I had a total of 10 VMs running and I just included that on the left hand side there for your information so we were seeing about just under 45,000 IOPS with the front end type and a little higher than that with the back end but what I want to focus on is the middle graph there with slave one so this is the IOPS figure and I can't really see the number there exactly but that was about 35% more so we're seeing maybe 8,500, 8,600 with front end and more than 11,000 with the back end so I got higher performance there of that one VM and volume that I wanted to perform well slave two and 10 were even as you would expect the throttling worked as it should so those were sitting right at 4,000 IOPS for those okay so you remember the IOPS we had more performance on that one high priority VM and volume now we look at latency in the same scenario I'll say the best for the latency of the slaves there's about 17 milliseconds almost exactly the same so they were being throttled their latency was higher because they were queuing but again we see the QoS type here for front end I had about 35% higher latency than I did for the back end and I think that they were both 35% just a coincidence but so I enjoyed low latency for random reads as well as higher IOPS on that high priority one okay and last to try to get these all came in one graph but the sequential IO latency looking in the same test was even more markedly different for the slave one for both reads and writes it was very even and very low for both of them so that's a good news but you remember it was a different scenario for random IO on that but when we look at the slaves it looks like there was a lot more queuing on the host so a big difference there and for sequential reads I think that was about 67% higher latency using the front end QoS and actually about the same for the right so very different scenario so the net of this is what the back end I was able to enjoy lower latency especially from the throttled instances there and so hope this is useful to you and I'll turn this back to Ed. Okay cool so I got one slide left here and then we'll talk to some conclusions but this is kind of a pretty interesting slide so what I did was I set up my scenario I set up 32 VMs tuned them all and pushed them all the one hypervisor so I had four I pushed them all the one then I ran my tests, all those tests lots of graphs and then I started moving the VMs apart so I took the same 32 VMs and live migrated half of them over to the next hypervisor and then I ran the test again and then I ran, I took half of off of each and put them to four so I'm gonna throw this up here as the one example of this this actually happens to be random reads but they all look fairly similar to this so I didn't wanna death by too many graphs and slides but interestingly enough we look at this, the one hypervisor with 32 VMs and you can see that the latency goes up pretty quickly so this is kind of as we would expect as we spread this out on the hypervisors our latency knee goes out the interesting part is the maximum performance of all of these and these are all scaled the same so if you're looking at these I made sure that they were all scaled the same our maximum performance when we get to two and four is the same but our latency is much higher so that contention on the hypervisor that's its effect is to really add latency so the moral of the story is spread your stuff out and that's kind of, I guess I got two slides here Nova VM layouts are important and that's what it's going for there Nova does some weird things so spread the stuff out is the moral of the story but when you do things like I put five I have five hypervisors and five VMs you would think that Nova would put one on each hypervisor nah, it puts two on one hypervisor and none on another and oh by the way I had one off VM and it put these two on the same node with the one that was off so one hypervisor had nothing and one hypervisor had three VMs one of which was turned off when I do the 32 VMs on five hyper I had five hypervisors and then somebody stole one, John which is why I did it with four because that actually made the math easier but I was looking at this kind of the layout stuff and so 32 VMs on five hyper and I got one hypervisor had two VMs one hypervisor had 12 VMs and yeah, so that's why I started off I'm like oh this is just gonna work it's just gonna put these 32 VMs out there they're gonna be evenly divided not the case so I wound up doing a lot of scripting to move these back and forth live migration so I think if there's Nova guys in the audience we might wanna look at kind of why this is the case I didn't mess with the schedulers I didn't do anything all my hypervisors had exactly the same RAM and CPU in them so something to kind of normalize that the other thing that from those pre yeah, okay, see he's suggesting the RAM waiter changes how that if I hear and you're correct from here they all should have been set the same right, these are the other identical machines with one controller oh right, okay so you do group them yeah, okay so there's some more investigation to be done there's a lot of investigation I think that's what all three of us have we spent a lot of time working on this stuff and there's more investigation but also we might wanna look at how do we look at what the volume type is is part of the how do we do the Nova schedule I don't know, that's tough right because we're cross-projecting but how do you get take if this is a high performance volume and take that into consideration and sort of where you put that with when you lay out the VMs something to think about and then there was a Veritas v. Brown bag about residual IOPS scheduler I didn't happen to go to what David did and he said we should reference that because I guess they had a pretty interesting way to kind of get some feedback about what was actually happening in each hypervisor and then feed that back into the scheduler right, get that right so and with that, we're gonna hit some conclusions here we'll take some more questions they didn't want us to ask to go to the mic so yeah, I'm gonna tell you that but so let me talk about the conclusions and bring these guys up here a little bit basically we think OpenStack works pretty well a lot of the stuff that's happened here in the recent versions has made it the defaults are pretty good there's some tuning to do if you're trying to eke out every bit and do the tuning but it doesn't help that much and especially if you're using a lot of little workloads the multi-session I SCSI stuff is interesting and I think John and I have talked about how do we maybe incorporate that into there back in QoS is more fair to mix workloads and spread out your VMs as best you can so I think that was it and you guys got anything else you wanna add into there? No, I steal your stuff and I'll keep doing that okay, all right, with that can I ask you to just tell if we can hear you, we'll repeat it and we'll get it on video so well, so that was kind of a misnomer I think we should have said storage performance yes, we did we did look at some of that and John and I are gonna have a conversation in the book that part's great we're gonna have a conversation over a beer tonight if we're still awake after we spent so I've actually been working on that a little bit lately especially the past couple of weeks because there's something not right something really not right I don't know what it is yet I haven't figured out but something particularly at the API and it looks like it's something going out to Keystone but I'm not sure but I am aware of I think I'm aware of exactly what you're talking about but I don't know what it is yes, I think we all saw that and that's why we started kind of looking through it a little bit here so that'll be next summer we'll talk about that sure, yes yeah, that was one of the tuning things that we kind of looked at there's a lot of information out on the web like if you Google this stuff and you start like you wind up reading a ton of documents and some of it's older stuff that's not that's one of the big things I was finding was that oh, I read this recommendation okay, I gotta go check that and make sure I'm doing that and a lot of the modern the Senthos 7.2 really the guest's image really fixed a lot of that fixed it in the sense that it made it what the recommendations out there were and so you don't have to go mess with it to get it right 6.7 was a little different on that note so the Vert.io SCSI is the default in OpenStack and that's what it uses for pretty much everything except for Ceph which uses the RBD stuff and then of course NFS one thing that has come up in the past is people have talked about using the Libvert and KVM iSCSI stuff and going direct that way and using Libvert as the iSCSI Attacher or the initiator there's a lot of foot out there folks under the impression that that gives you better performance and stuff like that it actually is almost exactly the same there is almost zero difference between the two it's easier to manage and easier to maintain because it's built into your Libvert version and you don't have to worry about it doesn't work the same on Ubuntu or it doesn't work on you know, Senthos or Red Hat or whatever the same way and the Nova guys have to have all this code to make sure that it works right so that's the only advantage it's not a performance thing it's a maintenance thing and if you're gonna do your favorite fiber channel then you gotta have yet a different way to do it fiber channel is dead to me I could add to that too there was a session yesterday I went to that was for QMU and KVM IO did you did that one? I know I recognize you so there we go so you had some great data there okay so did you actually get you set that up and ran the whole iSCSI Libvert stuff so that was one of the things we thought we were gonna talk more about that too but I didn't put the data in there because it wasn't very clean I had a really really difficult time getting it to run it turns out you can't you cannot run it in open stack right now period so what I ended up doing was my own KVM box on my workstation and I did it all manually and set it up and everything and I got to work that way but I saw absolutely no difference whatsoever right I think we're at our time we'll be around here if you wanna if you wanna come up and beat us up some more that's cool we're good thank you


Artless just updated their site with new discoverability features that help you find the perfect asset quickly and effectively. First new feature is suggested sound effects. When browsing through the footage catalog, the system will automatically suggest relevant sound effects to the footage you select. The second is AI powered search. With this feature, literally just describe what you're looking for and AI will help you find the perfect match. The third feature is music search auto suggestion. With auto suggestion, you'll get real-time recommendations for music collections and categories based on your search. Just type in a vibe you're looking for and you'll have a whole collection of tailored songs. The fourth feature is sound effects sorting. Sound effects sorting allows you to quickly filter sound effects by staff picks, newest and top downloads, so you can discover the right sound effects for your project in no time. These four new features makes it so easy to find the perfect sound effects, footage and music for your project so you can spend less time searching and more time creating. Download this .io and explore the new discoverability features yourself.