The bones of these animals here, that belong to someone. How disgraceful. Mass grave out the back. It's almost like with bones, people don't blink an eye if the bones come from the pig, but with our human bones people will be like freaking out. So pigs just don't matter in this society. So what's actually more disturbing to me is that I used to have these bones on a plate on the dinner table. So I was eating the body parts of these victims of this atrocity. And I wouldn't blink an eye, it was just normal. But now I just see the remains of dead body here. Very disturbing. Instant perspective, like everything. Just the whole vibe here has changed now. Just a heavy vibe here now anyway. I feel like when those trucks stop you gain instant perspective. Boom, it hits you. It's pretty heavy. It's sad. But it's important that we're all here. Yeah, I know. I know it's like dogs aren't they? I think these places, because they are obviously getting murdered in there. And I don't think they truly know what's about to happen, but I'm sure they're a bit unsure and scared. So yeah, a bit of love always helps as well. Sending them some love. A bit of compassion. Beautiful. I love you, Baba. I love you, darling. Four babies need some love. And they're not going to get shown lonely love. As a sick world treats them like property and like products and like bits of meat. Four babies.  Unsure of what's going to happen to them. So just a bit of care and compassion and love is important too. Baby killers, eh? Look how young they are. Terrified little prisoners. It's like they're born to get killed. They're bred into existence to be slaughtered. It's not fair. Thanks for coming here. The world could use a little bit more love. I'm happy with the turnout. I'm happy that there's a lot of motivated individuals here to bear witness and to share the pigs' story. I'm not happy that this system still exists and exploits and kills animals. I don't think it's justified. I think it's an atrocity and it never gets easier being out in front of these places. It's the haunting sight of empty trucks. Used to be able to hear the screams of the pigs from inside at this slaughterhouse. But I think they've soundproofed the walls in there because you can't actually hear them screaming anymore. Which was very obvious last time we were here. A lot of people might see a truck with animals on it that are going for human consumption. What do you see? I see individuals each with their own personality. Terrified, infant-like animals who are being used against their will. They will all be butchered and murdered. I see an injustice when I see those trucks. Shut the gates, man. Are they all shut? The only reason those slaughterhouse workers are stabbing animals in the throat in there is because consumers want to eat their flesh in their funding these places. So, you know, you can pay lip service to veganism and go, you know, I'll support your cause, I'll respect your cause. I'll support your cause, I'll respect your cause.  The knife and the blood is on your hands. One thing I've noticed while I've been watching is that there's been a huge change in activism. Anyone can be an activist. They only need to have a mobile phone and a social media account. Yeah, and, you know, I sense that this is wrong. And the only way things change is if activists speak up about it.  Anything can be changed. If something like this gets out to enough people, good compassion at human beings wouldn't want to be funding a place like this. And we could get millions of people to be on board with this, and we have. If they were human beings in a truck like that, there's one word for it, there's one word for it. Human beings have been sent to either a gas chamber or a slaughterhouse to be murdered, but the only way a holocaust can take place is for good people to stand by and do nothing. And that's why we're up for this. Probably the only real love they'll ever get in their entire life and it's out here before they, you know, meet their fate in there. I don't think holocausts like this can exist with enough love in the world. It just wouldn't happen. It's a prison truck. Look at it. These are sails, breeding sails. They use them to breed piglets for about three years, three or four years. The clients, they send them here to be slaughtered. So they're just prolonged suffering.  If this is the start of something, where is it going? Well, it's already happened. In my eyes, it's already happened. We just haven't caught up to it yet. This is a very objective truth. I mean, animals are being slaughtered for an unjustifiable reason. Most people are against animal abuse and animal cruelty. It's cruel to rob someone of their life and to use them against their will. I mean, we just have to get enough people aware and on board with this and out of their conditioned lifestyle and onto some good vegan food and we can make a change. When you bear witness, it gives you a reality.   If you feel anger, that's good. Feel anger. Channel it productively though. You know, don't let the anger and emotion consume you. Use it to fuel you. You use that as motivation to stop these places. I love you. I'm proud of you, mate.


Hey guys, it's MJ the student's act tree and this is going to be another audio lecture on business objectives, which is subject CT2 and this is still chapter 1 part 2 So like I said, it's business objectives. We're just going to jump straight into the material and the first thing I want you guys to consider is what is the objective of a business and This is quite a It's quite a fun question to just think about, you know, go through all the various things, you know What why do we form a business? What is its purpose and We could come to the answer that the objective of a business is to increase the value of the shares so a business's goal is to become worth more and The next question that follows up from that is how can this be achieved? Now in the short term this can be achieved by making more money. So the more money business is making You know, its value will go up shareholders Would like to see that more money being made But in the long term The value of a company goes up when it starts Contributing more to society starts making better products. It starts becoming more, you know just better for for the whole economic environment and This now leads us to to what our business is and we spoke about what businesses are In the previous lecture, but another way to think of businesses is that they are a collection of economic projects and they all have They all have the following components. They have the inputs inputs being capital and effort So remember in last lecture we spoke about the shoe factory So the capital would be buying the shoe factory the effort would be running the factory and actually making the shoes Then there is the time the time process You know, you don't just buy a factory and then suddenly you've got shoes It takes a process and it's going to take a few days a couple of weeks Before the shoe is completed and then you can sell the shoe and this is where we talk about the outputs You get money and you get value. So with a shoe, you can sell the shoe You get money for what people paid for it, but there's also the value and this is a more wishy-washy thing in the sense that You can't really quantify it. It's not very scientific But there's a value had in somebody having shoes, you know protects their feet, but it's very difficult to put a number on that value and this is unfortunate because it means a Lot of the time people when people come to business they now focus on capital as the input and Outputs as the money and they don't tend to think of the effort or the value Just because those things are hard to to measure up now Where actuaries come in or where actuaries can help a lot in Businesses and day-to-day is that these things capital effort time money and value They're all uncertain. They're all random variables and Yes, in the beginning they have an expected value and you know this expected value that the share The shares are priced But as time progresses, you know, these expected values can change certain events can happen or Uncertainty can decrease as you get closer to the end goal and thus we can see that the value of the share can change and now what happens in a business is that some shareholders might choose to sell out at this time and you know get a little bit of profit or or you know use their money elsewhere and The great part of the business is that when shareholders buy and sell their shares They shouldn't have an impact on the business so it shouldn't disrupt the business because some of the time or with some of our really big large corporates you have these really wealthy individuals with these wealthy consortiums and they're investing and They don't have the time to you know actually do the actual business or do the effort So they hire professional managers. So what we're doing is we're splitting the input shareholders provide the capital They then hire professional managers who you know contribute the effort and then What happens is the money and the value will then get distributed between them so with some of your bigger businesses what you have is you have your shareholders and they appoint board of directors and Your board of directors will then Choose a whole bunch of general managers and your general managers will hire a bunch of employees. This is a general Corporate governance that a lot of firms adopt and Like I said, there's the pros you're getting guys who are good at managing to manage and you're getting guys who have the Cash to invest to invest okay but now sometimes or Most of the time is there are some cons to this separation of duties and that happens because they are conflicting objectives. I mean you look at our shareholders They want a high return, you know, they there to to chase the profit. So they want to maximize The capital to money ratio they want to put in as little capital as possible and get as much money back You know that that is the perfect situation for the shareholder Whereas if you look at the manager, I mean he's putting in the effort side. So If he's gonna be putting in effort he wants to make or choose the projects that maybe on a little bit more interesting a little bit more fun to do and Maybe require a little bit less effort Because the ratio they're looking at is the effort to value created because at the end of the day They want to feel, you know fulfilled that they've done a meaningful job and that you know, there's there's purpose in their work So they're gonna be looking at the value created They're also concerned about the money, but they're also concerned about the value And that I guess that's what it is the shareholder, you know, he wants to maximize returns or build an empire It's all about the money and and all that returns Whereas the manager might want more of a luxurious working lifestyle and just he'd be happy with stable returns and This is where we're coming into this whole agency theory and information Asymmetries, I mean and you get it also with other parties. So with lenders People who lend money to the company They're more focused on short-term goals on, you know, the security of the company because they want to get their money back Also with say employees they Employees are an interesting one. I mean Let me let me give you guys a quick little example So like say today I went to I went to the gym and I bought a smoothie from the the smoothie bar Now the person selling the smoothie is receiving a fixed salary in in most of the cases which means If they sell one smoothie or they sell a hundred smoothies, they're going to get paid the same So if they do a really good job and they make a perfect smoothie and they're really nice to the employee To the customers and they really create like a real nice vibe going What's happening to them? They're creating more work for themselves because more people are going to come by smoothies and they're going to do more work more effort So what I've normally found with the smoothie people is they're not the friendliest They take a long time when they make the smoothies and I actually once even got smoothie that had you know shredded plastic in it But there was a once off once of occasion But the whole idea is that they're more interested in they're getting their fixed salary They have to look they don't want to work terribly because then you know, they could always get fired But I mean, especially in South Africa. We've got such strict labor laws that it is quite difficult to get fired So once you get a job, you're actually quite comfortable and you can slack off like that because the law is in your favor and Because you're getting paid a fixed salary if you sell one smoothie or a hundred you'd rather sell the one smoothie and This is where the subject tries to To look at this problem and say well, how can we fix it? You know, this this actuarial topic is how do we avoid this conflict of interest and how do we motivate and I mean the one one case which a lot of smoothie places do is they might have a manager so managers there He's monitoring them making sure everybody's working making sure everybody's being nice to the customers But this introduces another cost, you know, the whole agency costs because now you have to pay that manager And he also might have conflicts of interest and stuff like that It would be much simpler to just align the incentives, you know, tell the tell the smoothie guys Say for every smoothie you sell you get a Piece of that price. So if you're selling it for say ten dollars, well, that's sorry That's a very expensive smoothie But if you're selling something for ten dollars You get to take two dollars and this would incentive us the smoothie maker With the shareholders to maximize the amount of smoothie salt so that they get a higher salary and the shareholders get a higher salary So this this is this whole You know When incentives are not aligned and this is also one of the reasons why I don't like Investing with asset managers.  We'll invest it And we'll pay you a return and what they say is We've aligned the incentives by if we get over a certain like if we get 20% or more then our fees increase and You know, when you first think of that you're like, okay, well, that's great Our incentives are aligned the more money they make or the more money they make for me The more money they receive therefore, let me give my money to them because they're gonna chase the highest returns But there's a big problem with that thinking because when it comes to investing and business and all that type of stuff there is the risk dimension and There's kind of that, you know Basic rule that the more risk you take the higher the potential reward Now if I go with my money and I give it to one of these asset managers They are incentivized to take higher risks because if let's say they risk in this really Risky business and it succeeds. Well, I'm happy. I get lots of money They're happy because they fee increased, but I'm more well-off in them because I've got a lot more but what happens if They invest that all my money in this really risky business and that business tanks will then, you know, business fails What they lose is they just miss out on their fees Which is a very small amount compared to all my capital that I've invested that has diminished so what we see is These this agency theory problem exists in quite a lot of economic activity Whether it regards to risk or with regards to you know selling and efforts and all of this type of stuff And so I guess to just land this whole lecture on business objectives We should maybe redefine the business objective as maximizing shareholders wealth But within external constraints and what exactly those constraints are well That's what the rest of this subject is going to be exploring. So this is just very much of an introduction To all the corporate governments agency theory and all the weird and wonderful things that can go wrong In the mechanics of the business. So, yeah, I hope you guys enjoyed it Click like ask questions in the comment section share with your friends and subscribe I will be making more videos like this Thanks guys and job as always study hard. Cheers


What's up everybody? Jason here for jaspyscasebrakes.com. Jaspys six box opti prism basketball mixer just sold out It is a random teams break number one and this is also an early bird break. We're having a chance to win a free spot So buy one of the first 20 spots in this break and get entered a winner free spot We're randomized the first 20 customer names and top name at the end of the randomizer wins a free spot on this break so we use the dice really here for the early bird customers and Add you here in the randomized customer names and teams which is a lot of why this little mixture that includes Box of choice 1920 fast break optic retail a optic mega box a blaster of prism 1920 and an optic blaster So good luck guys Roll it and we got a three and a two five times Customer names at number one after five times gets a free spot one two three four and five Boom five times Tony congratulations. You are the early bird winner after five times So let's add you in here little early bird spot Mojo and five times on the teams and names one Two three four and five Five times Thomas down a Kevin There you go five times on the teams one two three four and five Five times nuggets down to the thunder I have Thomas you got the nuggets rich Richard Schmidt with the Hawks Robert with the spurs chat with the jazz Robert Mancini with the enix calling with the Wizards Chris with the Pacers Teddy you have the Sun some Matt with the pistons Diana with the heat Derek you have the Mavericks Mike with the Hornets Richard Serrano with the Bulls Rich with the Warriors Alexander the Kings Anthony with the 76ers Brandon with the Lakers TJ with the Clippers David Sullivan with the Magic Daniel Bucks last while Mojo Tony with the Blazers Brandon with the Cavs James with the Wolves Derek V with the Pelicans Tony your free spot is the Nets David Bergstrom with the Rockets Kevin Mano with the Raptors Chris with the Memphis Grizzlies DMACC with the Celtics and Kevin Crouch with the Thunder There you go Now I'll put ties I give you guys time to trade all right someone just actually got the Giants and encased One of the bigger teams remaining. So now we're down a four left straight up Again Cardinals Broncos Jaguars Retzkens Jinx you guys remember those twins from you guys seen a The movie is it a 22 Jump Street those comedian twins Like jigs, but it's over pop Like saying the same thing cuz they're like twins That's funny. All right, Kevin's gonna stick with the Thunder then TWC No, there's no more paper There's worse teams in the jazz, right? I mean at least there's a rookie in there Goga all right prison blaster Servetus Chimo Kiki Rosdaikis Tim Hardaway, Jr. Purple Wave And wow, look at this out of a blaster box Zion Williamson Silver Derek with that. Whoo, man. This makes you start off hot Off-centered, but still At least what four or five hundred bucks not more six hundred That's the way to start off the mixer boom The mirror Carol Marvin Bagley There you go It's been a minute since I pulled a silver. Oh, maybe Nick sold it out I think I think a random number block is getting put up Matthew I don't see any orders coming in. So maybe it's just around number block Yeah, I think around a number blocks getting put up. So Nick listen to you guys Yep, here it is. I mean if it sells out, I can probably do that. I mean it's a quick break drew Eubanks or E banks spurs phobia whites Cow guy Retail Tyler hero. This has been a great mixer. Wow. Look at that. Even the Zion came out of here Two Zons two for two One silver one base Cobing there's a jaw Loaded mixer and Jordan Clarkson checkerboard Sorry Elvis. I mean these are from a separate case though. They weren't this part of the same case as the as the other 10 two boxes Let's go with the optic mega box, which I actually haven't ripped any of these mega boxes Not that I remember of you get the special two other packs, which was leave those last and then some regular optic packs and I think we can run back this mixer as well guys We have a handful of these boxes to run back. Yeah, so in case we'll be coming up soon, then Yanis Is it a Thomas?  Oh my god This is crazy guys Shaq they moved over my house You believe this right now guys, this is literally one of the craziest Optic prison mixers I've seen this is our first time doing it Man three for three in Zion's. Can you believe that? hollow base Silver it's crazy. All right. How not a Zion, but how about an Achille Alexander Walker? Little pinks. I'll finish leave all these.  Okay guys. I'm done. I can't top that Something crazy like that Carson Edwards Celtic's gonna DMAC. Let's go with the retail then we'll go with Fast-breaking and off with choice prison Can we go four for four guys? Can we go four for four?  Yes You're probably a little upset because you wish you could have got the pelicans. But yeah, honestly This rarely happens. So it's actually a treat to see. Oh, yeah, because So many people grab the pelican straight up in a full case or this and that and just never get anything and I Mean, it's gonna happen every once in a while Right pelicans are gonna hit big Yeah, it's crazy I mean for my enthusiasm, you can see I haven't seen many Zons in a while at least like hollows and silvers and The fact that it came out of a blaster. It's just even more Insane to think about. I mean if you can get another zone base or something here, that's gonna be crazy Already. Oh my god, really?  That's crazy, Derek Fiondhu, Ron, my house. Oh Man, man, that would have been crazy right there. Jackson Hayes Hollow Scofield retail I would have been nuts Derrick Wow, it's a one-of-one Dirk new whiskey. Look at that. That's so cool man. This makes it has been loaded Jesus Mavericks Derrick there you go Derrick's the only one that said he's very refreshed seeing this and there you go Rewarded awesome Tim Duncan Man, let's run back this mixer guys. We have we could probably post another one. Honestly, I'd love to run this one back Now that we're not doing the big football mixer We have plenty of time to do another one of these after like encased T-minus a little alert magic All right, no Zion in here, but this one had a one-on-one Dirk Dirk Dirk This mixer was what 60 bucks People want to run it back down. Let's do it. There you go. Nick. I just literally texted you There you go. The next one's already been posted. Have you seen this mixer so far Nick? This is nuts 101 Derrick hollow silver base and you have a job as John Moran in there This has been a nuts mixer And the person that got the pelicans. I don't recognize Derrick with that last night. So it must be a new customer. I only had one spot 60 bucks That's crazy. I know I had never ripped him in person to Nick. It was pretty cool. All right Let's go fast break now I told him if I pull if I pull the tiger stripes on it. I'm done. You're gonna have to come in and break There's just no way that I just I just won't I will never top this break. Oh I actually have another I actually got a text message. Nick. I have another Jordan we can get So I'll text you I'll text you the photo very nice Jordan Good price as well. Hi stare. That's awesome. Yeah, I already bought a bunch of that late 70s 80s early 90s Yeah, so in case won't take us more than like 20 minutes so I can definitely run back another one of these optic mixers easily And again, fortunately the football mixture won't go today, but it'll be the first thing that Joe breaks probably tomorrow Mitchell Robinson in 95. Oh, yeah, and we can break the cup. That's right So that'll be a nice last three breaks of the night would be awesome.  Try the hero I'll clear for take off lay grip. Wow. There you go four out of five zions in each box Just a base, but still what is that like a hundred bucks still there's a jaw and Jalen Noel T wolves autograph go into James Will Barton Oh, nice Little disco of Luca Sam Nick And we got a Jeremy lamb gold 10 out of 10 for Indiana Pacers Chris what No, it's nine o'clock Probably three more Derek favors Tidra. All right, you guys four out of five boxes in this mixer have had at least a Zion base Can we make it five for six? Let's get a tiger stripe. All right. Good luck guys tiger stripe Chances here with this choice one little pack so much heat. I got a Roberson for the Thunder you got Adam Rose Goldfield silver Wizards going to call him and the tiger stripe Steve Nash nice. Oh My god, I wanted a tiger shot, but I guess I'll just settle for the auto. Oh my god The Zion Williamson Autograph out of the choice box. God, I Mean tiger shy, I guess it's still as good as a tiger stripe. Jesus Penny hard of it at 88. Well, there's three of them. Sorry Deondre Jordan, I know I don't even think the customer is watching I mean he might be or she might be but oh my god. That's crazy. And then Steph Curry Wow, what a mixer for the pelicans. I Mean, we don't get to see this every day guys Honestly, it's really tough to hit but when we do definitely appreciate it I know we all wish we had the pelicans But still awesome to see Zion autograph. You got a tiger striped Steve Nash John Moran Zion base dark one-of-one nice break for the mouse John Moran Zion base again silver and a hollow all in the same mixer Whoo, all right, we have another one up. I'm not sure if it's gonna be just as good But I hope I can top it which I'm probably not but the second one's more than halfway done Jasper's case breaks calm


How are you doing today? Be very nice for the skin. Take some alcohol. That's going to burn his skin. We're going to train you today. We'll give you a like and what you can see. It's going to burn down. What do you do? I think the shift change went pretty well.


Alright, time to- time to loot some cave mushrooms. Oh no, he's gonna get my mushroom, isn't he? Look at him. Straight to it. Little f***er. Did he get it? Didn't get it. No! No!   That'd be instacarma. I'm just gonna nuke him, yeah.  Alright, here we go. Resist. Resist. Going back. Gonna reset. I'm not starting with double resist. There's no way- and now it's Voidwalker's back. Oh, this f***ing game, man.  Oh, holy- couldn't resist. Yeah, well. Figure it out. Game is f***ed. It's- it's because it's not a- a resistance-based resist, right? In- in level 60, there are hit-based resists and resist-based resists, but they both say resist, not like in Wrath. In Wrath, it says miss when it's a hit-based one. So, likely it was a resist on him because he's 24 and I'm 22, so my hit cap on him is, like, 97%, some s*** like that.  I have some shadow hit to relieve that. You can't get rid of the last 1%, but that'll bring me up to 99. So, we just need to get lucky, basically, or not unlucky. Oh, that's an unlucky break on the fear. It resisted, man! It f***ing resisted! Are you kidding me? I can't risk going back there. I'm gonna die if I go back there. Oh, for f***'s sake! Now, I have to wait for respawns. Minus 5 minutes. Good stuff. Oh, that's so towing, man. Here he is. Noob's coming. Don't want to run into these boys.  So, we have to clear them. They got emulation aura as well. If we're quick after this next mob, we can do it before these sh***y fel hunters respawn. But this next mob needs to spawn now. Any second. He had 4 HP, but if I go for another one, the sucker buscatches up to me, then I can get dazed, right? And he can- he can maybe get another cast on me, or the f***ing voidwalkers will come to me. And if I get dazed by this guy and the voidwalkers come, then I'm dead because I have no fear of- So it's like Omega risk. And if another one resists, then I'm 100% dead, right? Because he will just cast on me. Where the f*** is the- the fel hunter? The- the- what is it called? The Fel Guard. There it is. Okay, let's go quickly. Fel hunters are coming. Tell you that much. Dude, can I even loot it up there? Surely I can. All right, it's f***ing run boys! I need an evade spot. I- there's surely an evade spot around here, somewhere, but I don't know it.  Sh** things. I think f*** for that, man. I'm gonna kill this while we wait. There he is. So if you look at this macro right, I can target a guy, press D. And then the other macro, if I'm like looking for the mobs that are called Blackrock Champion, I can just press it. It will target one and mark it if there's one nearby that's alive. This way you don't need to constantly change the- the first macro that actually does the targeting. Because this macro edits this macro. So you- all you have to do is target stuff. You could probably do it for mouseover as well, but... Yeah. So this makes it really quick and easy to look for mobs of a certain name. Oh my god, I haven't spent a talent point this whole level. Morn, Cas, boys. I'm just gonna half after this one. Whatever. Unlucky turtles. So I've lost my play, 22 hours. It's not bad. 23 hours to 30. That's fairly good, I'd say. You can see here how much I'm beating my previous time by. So like every level we're outpacing ourselves, which is really nice. Really good XP per hour as well for this level. We reloaded when we were doing the Elite Quest, so that was like around here, I think. Let's hope we get good drops here now. If you're enjoying the content, you'll have to hear it, mate. No, my boat! Wait, I'm going- that's the wrong one anyway. Not my boat. That is my boat! Go, go, go, go, go! No, my boat! I'm missing two boats in 10 seconds, by the way. What is your spec? So we started off with one spec, then we came here to Spirit Tap, one point in shadow focus, then we came down here to five and five, so we got the extra DPS on our Holy Spells, and then we're coming down here and we're taking three-inch shield, then two into martyrdom, and then we're going to put one in a focus, three meditation, that's going to bring us up to about level 40. At round level 40 or 42, you want to respect a full shadow. Bad pathing, boys! Right, so what do we get in next level? Mind Blast.


about NVMe error logging and the reason I wanted to talk about NVMe error logging is because now that we're having more of our customers starting to use this technology, particularly NVMe over fabric, it's become apparent that it can be difficult to troubleshoot things like connection issues or other problems with the logging that we currently have in place in the kernel. In particular, if you hook up a storage device and it doesn't work, it can be very difficult to figure out why that is. So there's a couple of things. One is that there's several differences between the kinds of errors that we log for SCSI, which is at least for the type of technology I'm most familiar with, it's just FiberChannel. The types of errors that you get with SCSI over FiberChannel versus, for example, NVMe over FiberChannel. But it's also a somewhat of a more generic issue. We see this sometimes with customer reports of errors with NVMe PCI devices where a customer tries to use an NVMe PCI device in the system and it doesn't operate as expected and it's difficult to figure out why. So one of the things was that SCSI error logging in general has undergone a bunch of changes over the past several years. In some cases, we've slimmed down the error logging.  But also we've added a little bit more detail for things like errors on commands where we actually show the CDB, we show how long the command was outstanding, we've got the error, things like that. So there's been some proposals. I haven't actually submitted any patches for this yet, but I have some patches I've had of development for a while that I'm still toying with. And really I kind of wanted this to be sort of a more open general discussion about what do we want to actually have this look like. I don't think necessarily want to go to a very verbose model for NVMe, but we do want to provide enough detail so that when something's wrong we can pinpoint what the cause is. So did anybody have any questions about that before we go into a little bit more detail about what I'm talking about? All right, so with NVMe over fabrics, there's sort of effectively three phases, right? There's the discovery phase where you're finding out about the device storage end points that you're going to make connections to. And then there's the connection phase where you're establishing the connections. And then there's when you're actually doing IO and you get errors and what type of errors you see there. So with the discovery phase, one of the problems that we have seen is that the way that NVMe over fabrics operates, when we do get discovery log page, we typically see all the entries that the device is reporting. Even if you don't necessarily have connectivity to those ports on the device. Or even if it's not the same transport type. There are some devices, for example, that will return both NQN information for Fiber channel as well as say NVMe over TCP. And your system may not just have the capability, may not have the adapters, network connectivity to connect to those. But you'll still get errors about it with the current approach.  But we won't really display any information about why this would be, right? It could be because the device isn't allowing you access. It could be because there's some kind of physical connection problem between the two. Could be a number of things. And then when we get errors on the actual IO, they can be errors for the device actually reporting on some error attempting to execute the command. Or more typically we get errors on the transport, trying to connect to the device if there's a connectivity interruption. And you'll typically get error messages to the form. NVMe and IO error minus six, and we log this. And this isn't very useful for a customer. So what I wanted to have a conversation about was sort of, what do we want this to look like? And do we want to have the same level of detail that we've had with other transports in the past? Or are we really trying with NVMe to make this sort of as a seamless thing as possible? I think that the current approach of having, well, the bare minimum of logging is not a bad one. Because one of the things which constantly bugged us on the SCSI side is the sheer flow of information coming from the SCSI device, which really caused the whole system to slow down due to the design of print gate. And so really the only good way of getting rid of that is simply by not printing anything, or only print a print which is absolutely necessary. So what I would love to have is to delegate error messaging over to either dynamic debugging or even to tracing. So the depth debug thing and so which then use the dynamic debugging really helps here. Because that means you have to actively enable the error messages and then you will see them, but in the normal case, you simply won't have error message. So you won't be off of all the information. That is actually the direction I would love to take and also to rely more on tracing because that allows us to, while really figuring out what this specific IO do whilst being in the stack. Sadly, there is an issue with that because the multi-parting kind of breaks the tracing approach. Because the tracing approach requires to be tracing to be enabled on each device. But for the parts, we don't really have a device which we can enable. So we can only enable tracing for the upper level for the, essentially for a struck NS head, but not for the path devices. So we will never see the actual error in the tracing because we can't enable it and that's an issue which, well, probably would need to be addressed. But I really have no good answer how we could address it because it would mean that block trace would need to enable more than one device which currently simply doesn't. Now, one of the things that I hear from our support people as far as doing things like enabling tracing to provide information and this is similar to what's been done with some, I know at least one fiber channel driver is that often the only information we'll get, if there was some kind of intermittent failure, we'll get a log file with the system log messages. But if you didn't enable tracing, none of the information's in there. And they may not be able to replicate the problem easily.  And it would be preferable if there was some way to have, I don't know, using debug FS or something like that where you could go pull the information out of essentially recent activity or something. So the problem with doing that is then you end up buffering a whole lot of information in the kernel and this tends to not be particularly a good idea. So when you're saying debug FS, are you saying it should be on the target side or on the host side? No, this is on the host side, right? So it has to do with the concept of your system might run along for months and be perfectly happy until there was some upset, some disturbance in your switch fabric or whatever you were using to connect to your storage. And then you get some kind of outage and you don't know what happened. And the only information you're ever going to get is whatever you happen to capture in a log file at the time. So the trend had been to go and put more and more and more information into the logs so that if something ever happened, you'd have enough to try and at least start an analysis. But then what happens is if you have an outage where if you have a large number of devices, you can suddenly get flooded, like Han has said, with a huge amount of information that's not terribly useful, right? So this is what I'm talking about in terms of what should we do for NVMe relative to what we've done in the past for SCSI is I don't necessarily want to take what we had for SCSI and try and provide the equivalent in NVMe because there are some good things and there are some bad things about what we currently got. So I guess it's like if we could do it all over again, what would we want? I agree with what you even said about mechanisms that have to be enabled dynamically, of these usually enabling these mechanisms after something happened is too late. So that's why 20K is interesting and I'm aware at first that 20K could solve that significantly, especially if the output was sent to a serial console because at the time 20K was still synchronous. If I'm not mistaken, I think that 20K has been made asynchronous for that kind of purposes. Then I think we should take a closer look at 20K and see whether it can be used as a mechanism for building that kind of information. Okay. Another thing that comes up since Thomas mentioned NVMe native multi-path is that sort of the concept behind NVMe native multi-path was that it was going to be more of a transparent thing for handling the multi-path than DM has done. So when you get errors, when you're running with NVMe native multi-path, right now, if you know exactly what messages get logged under which conditions, you can make inferences, for example, about whether you have no access to your storage because you don't have any good paths or because none of the good paths that you have have an ANA state that will allow access. The two messages that are logged are very similar but slightly different. So one of the questions is, should we make this more explicit? Sort of, you know, I can't do this IO because none of your paths are in the right access state. Because right now, yes, there are messages about a lot of things in NVMe and if you can go dig into the kernel code, yes, you can go find where it got logged and infer exactly what conditions would have triggered that.  They're just, yes, they are distinct but they're not necessarily meaningful. Yes, I mean clearly, we should be updating the error messages there that it's not just trending the, well, the error number but really some meaningful thing. Because even the error numbers are just the mapping of the actual error being presented by the driver into something which then the upper layers do handle. So it's really pointed just to play what error minus seven so it doesn't tell you anything. Or the geologic driver printing, LLDD failure seven or six or something like that. It might be an idea to just, to formalize that that the error is actually logged at the NVMe layer in some meaningful state and then have it requiring the driver to provide the correct, well, return value such that we can log it correctly and not leave it off to the drivers as it is now. So on the NVMe side, it's supposed to log page even in a target, we support error log page. That is much more... This is not applicable. Because yes, the target might be able to tell us what has happened but the problem is we can't talk to the target. So an error log page doesn't really help because we can't access the log page. So it's interesting. If there is an error log page right now, I'm not sure if we can currently read that out through NVMe CLI, correct? You can read it out, right? So to the extent that the target can tell you something about it, you could go and send those commands to the target but that's an explicit thing that right now you do from user space, right? So we have, as a distribution, we have utilities in the event that we need to capture the information that goes scrape the whole system for these kinds of things and so that is something that we could do. And recently, from Oracle, we added the logging verbose error feature so you can tune it and you can get verbose error on request completion. Yeah, there's another facility that Broadcom added to their fiber channel driver that essentially has a trigger that will avoid logging things until something interesting happens and then it will display the most recent, I forget how many, but things in a buffer. Did you try the recent addition of the verbose logging with target? I'm sorry to say again. So did you try the recent addition of the NVMe error logging that we have added with the verbose option configurable? No, so we haven't looked at that yet. What I was primarily concerned with is what Juan was saying was the kernel, NVMe core code and transport drivers and multi-path code, the type of logging that the kernel is performing when it detects errors either on discovery, connection, or IO. But this is another thing that could provide more information about, particularly if there was some intermittent event. And it's really about, I think, making this mechanism a little bit more comprehensible to an end user. So things like eliminating numeric values in error codes. Things like that. So, like I said, I've had some work on this that hopefully I'll have more time to devote to in the future to try and get some patches posted. And so, if and when I get that in the near future, and get some review comments from people, I'd be appreciated. That is all I had. Anybody else have anything else? Yeah, that's one thing which is continue bugging me also is the error messages on connect. Yeah. Because the problem is there that the connect command we send is not the connect command which NVMe is using in the NVMe protocol. Heads up, we have some people that want to ask questions on Zoom, Hey, John. Hey, John. Hey. So I want to talk about two things that you brought up, Ewan. One is this idea of multiple transports being reported in the discovery log pages that leads to different errors because maybe you've got a TCP transport and you've got a fiber channel transport on the subsystem. And as you've noted, it depends upon the discovery service, that I've worked with, that we've seen. They're just going to report all discovery log page entries for everything. So I think that's something that could be fixed with a patch. And a question that we need to ask is do we want to actually support NVMe multi-path access over different transports? Right. So if you've got a TCP transport which is really slow when you've got a FC NVMe transport, it's like do we really want to be trying to support a multi-path access over both of those transports to the same namespace? I would say no. I mean, traditionally we discourage this for things like arrays of support, for example, both fiber channel and SCSI and for ISSCSI. People tend not to export the same volumes as LUNs on two different transport types. There's nothing that prevents this in any of the software. It's just not necessarily a good way to use this in the center of practice. What we have seen, though, is people wanting to use the same storage array over multiple transports with disjoint sets of logical units, or in the case of NVMe for disjoint namespaces. And so the question is to what extent, number one, can you actually tell, I mean, can the storage array even tell that it shouldn't report conceivably? It could determine that it shouldn't report discovery log pages for a different transport type than the one that it was receiving the request on? The discovery request on? Yeah, I agree about that. Offhand, I don't even know if that's if that's required by the standard. No, it's not. It's not. There are use cases in customer environments where they have a high-speed primary network and a backup secondary network. And maybe they want to have, you know, their 100-gig ethernet and an 8-gig fiber channel or 128-gig fiber channel and a 10-gig ethernet. And that might be the choice that they make in their environment. As for SCSI and NVMe, there aren't any common identifiers so that would have to be set up manually. There's no way for you to automatically know that those two data sets happen to be the same identifier because there just isn't a common identifier. No, the issue right now is that, you know, if you with the discovery Udev rules for fiber channel if you go and do the discovery you'll get back the log page that reports transport types to which you will never be able to connect. And we will try to go and do connection requests to those. And we will always get errors. But the storage doesn't know that either. Well, no, the storage could know that. No, I agree you can't really we can't really require the storage to do one way or the other, but we can certainly, you know, filter things, you know, even an NVMe CLI to just say, hey, you know, we're going to we're only going to report discovery log page entries to the host, you know, that matched the, you know, the transport that the application asked for, right? It's just a matter of paying attention to what the address family is with the, you know, if you send your NVMe CLI request over an NVMe TCP transport, then you could literally just filter that out in the results that are reported back from from, you know, from NVMe CLI. I guess, circling back to the higher level is, I mean, I think what we need to get to for the area handling with NVMe is something that's more comprehensible than what we currently have in the kernel. As kernel developers, yes, of course, we can look at it, we could probably figure it out, but those aren't going to be the consumers of this type of stuff in an enterprise environment. So I think we need to make, we need to be a little more gentle on our users. So I agree, Ewan, but I still think there are two fundamental things that you brought about. One is this idea of, you know, multiple transports, which in some situations is going to lead to, you know, any configurations that, you know, may not, I mean, they're possible, right? But, you know, I don't know if the question is, you know, do we want to support those, right? Do our partners want to support those? And I've asked this question specifically some of our partners, right? The second is this idea of connecting to devices with no physical link. You know, I see that as a configuration issue, right? I mean, filtering out log page entries based upon whether or not there's a physical link, you're never going to be able to fit, you know, you're never going to be able to comprehend, I consider the discovery log page entry to describe what we would call a configuration space, right? And whether or not there are actual wires, you know, between the host and the subsystem, you know, at each of those subsystem ports is really something that's completely up to, I don't think it's anything we should try to solve that problem. That's a configuration problem. Obviously it depends on the transport for fiber channel, you know, with traditional SCSI, we don't do the discovery that way. We typically, you know, find out about the remote ports from the name service and the fabric environment and we will only try to connect the ports that, you know, we actually can see the R port. Whereas with the NVMe discovery mechanism, the array is telling you about all of these ports that it might have somewhere that you have no connectivity to. The question is whether you can know that you don't have any connectivity. But whether or not you have connectivity, right, but whether or not you have connectivity is something which you're never going to get that right, right? If you try to modify things and it's perfectly possible to have a link that goes down, right? So with fiber channel, if the connection changes, then you're going to get, you know, the name server is going to be updated. You're going to get RSCN, right? And today what that means is that it doesn't mean that we're going to actually, you know, have to call you Deb. It just means that one of the NVMe controllers gets removed. You know, you just have one less path, right? And then when that port is plugged back in, you get another RSCN, right? And the path is restored. With NVMe TCP, it's not quite so simple as that, right? Because we don't really have like any type of a sophisticated RSCN mechanism with NVMe TCP. So I just see these all as problems which, you know, we've been dancing around and struggling with. And I think we need to ask some fundamental questions like, you know, what is it that we want to support and, you know, are we going to try to solve these problems? Some of which I don't think need to be solved. Certainly want to streamline the error logging because we don't want to be creating errors in all these different things. But basically you're not going to lose connection to your name space until every single path is gone, right? Well, so I think John has wanted to have another topic later on. Yeah, this week on specifically on the different TPs on NVMe Discovery. So. All right, so I'll go back on mute. Okay. Do we have anybody else on? Sorry, there's one thing which I want to raise. That's again the errors on connect. But we do submit quite some just submit a string of arguments. And then the driver has to figure out which is which and do actually connect underneath. And but the return we are getting is just an error code. So in case the arguments are wrong for whatever reason or not being accepted by either the controller or the NVMe stack or something, we have no way of figuring out which of the argument was it was which caused the failure. Occasionally you do get something some information by looking in the message log, but then again this is not really a nice way of doing error handling. So really we should be ideally we would have a way where we could either return a pointer or some indication at which point the parsing failed in the connect string. This is actually something that was pointed out by our QE engineer who works with me on this stuff. It's actually worse than that because when you do an NVMe CLI command to do a connect for example, it's not a synchronous thing, right? What that does is it instantiates a kernel object to do the connection. Which will then sit off and asynchronously try to reconnect. If you can't connect you cannot return an error on the syscall for the right for the NVMe connect. So the command actually comes back and says sure I did it and the kernel is off trying to connect to this thing that isn't going to work and you just get a bunch of error messages and there's never any feedback to any kind of script in user space or anything that didn't work. The other problem you have you in is that the error reporting is asynchronous. When you send an NVMe command it will come back and tell you that it was either successful or it failed. If you want exactly what failed, you have to go look at the error log entry, the log page and then you have to match up a log page entry with the command that you sent to look for the details which will point to the field within the command that tells you what's wrong. But they're completely asynchronous. Two sets of commands with no connection except some random command identifier and as long as you didn't overflow the log with too many errors or send too many commands. So there is a mechanism there to do it but it doesn't scale very well quite honestly. This sort of begs the question too whether you really want to implement that kind of state tracking in the kernel or whether you want to have some user space thing that can collect this information and tell you... You can't because as Fred said the error log is tied to the command ID and the command ID is completely ephemeral. User space doesn't know anything about the command ID so it wouldn't have a notion how it could track it. Plus that by that time user space gets along to actually asking for information, the command ID might have been reused. So you have no track whatsoever what really has happened. It's essentially a bit like the old request send to send command on SCSI it's actually the very same thing. It's actually modeled after ATA in the way they did doing an ATA rather than the SCSI auto sense. It's not that we can't handle it I mean we do it in ATA we actually do retrieve the error log page in the error handler figuring out the information and set the correct bits when returning the command. Yes you can do it.  But yeah it's doable. The question is whether we need to do it because as of now we don't and surprise surprise this is not an area where customers have been complained about. What the customers have been complaining about are errors running up to that point where you could retrieve the error log page or any command whatsoever because that's really where the issues is. What do we do if there's a failure somewhere in the chain before the command is being sent or processed by the controller. That is not being handled consistently or not at all occasionally. All right we're at the end of the day. Next up we have James Smart talk about commonization of NVMe transports. James here.