So, I have to say, I'm kind of in awe of the statistics around PyTorch, and I think when we... Why is there a number here, that's weird, huh, weird, let's get rid of that, I'm incredibly inattentive. You know, when we were, oh, weird, actually something happened here, there we go, aha. When we were conceiving the PyTorch foundation, honestly, it's amazing, I would say I pinged Jim on, I think, LinkedIn messaging, and it was like, hey, we're thinking about doing something with the foundation with one of our projects, right, we didn't tell him which, and of course he's like, oh, it's a PyTorch, please let it be PyTorch, please let it be PyTorch. I can't tell you, but we need to talk, let's have a conversation. And so, yeah, honestly, I'm so pleased at the way it worked out. We had no idea if it was going to be successful or not, if anyone would show up to the meetings, if anyone, you know, if it would grow, and I think it was... The results have been spectacular, so thank you, thank you so much. So I'm Joe Spezak for those who don't know me, and I'm going to talk about PyTorch, a project that has near and dear to my heart, no matter where I go, even if I go to Google, I'm still going to love PyTorch, which I am, of course, at Google now, and I'm going to talk about how PyTorch became the foundation of this crazy revolution in Gen AI and LLens that we're in today, and I think it's like the overnight success is like one of the ways that, you know, we think about it, like in the last six months, this craziness has happened, but I want to talk actually about the foundations of it. I want to talk about like leading up to this, the last five, even ten years of where all of this came from, because it didn't happen overnight. It was actually born on the blood, sweat, and tears of open source developers, people building libraries, people merging pull requests, fixing issues, which is obviously near and dear to this conference here in open source. So before we jump into that, very briefly on myself, again, I'm at Google working on Cloud TPU mainly, but also going to be investing, of course, in LLM and large model platform, as well as working on some applied ML work in my next, I guess, increase in scope, so to speak, I also do a lot of work in the VC world. I'm an executive resident at Canvas Ventures, if you're familiar with them in Porto Valley. I also angel and as well as my advice, companies like Lightning and Thropic is a couple of stealth mode companies I invested in recently, like contextual folks coming out of fare, as well as Run House, which is a former PM of mine who actually worked on PyTorch. So really awesome group of companies I'm investing in and partnering on. Of course, I was a founding founder of the PyTorch Foundation, where Sumith and I worked really closely to build it, and then previously I was at Metta for about less of five years, working on PyTorch, building partnerships, built a large model scaling team, but then also built an organization in fair research focused on math and science. So I have a bit of a full stack, I would say, background in the space. So this large model wave, I want to actually put this to some numbers really quick to actually give people an idea. So the old timers, I guess, like myself, who kind of look back in the 2013 time frame there, machine learning was like this less than 20% Google search term, where it was like, okay, this is kind of cool. And I remember at the time I was actually at Intel, and I attended a Berkeley retreat, and it was a free trip to Tahoe, so I couldn't turn it down. And I came back and I said, holy, you know, actually I'm on camera, yeah okay. I said, holy cow, this is gonna be crazy. ImageNet had just came, you know, I was surrounded by academics that were just pushing the state of the art. I met Yanching Jia, who was building Cafe at the time, which ended up being a foundation of Facebook's machine learning, of a lot of other companies, Yahoo's machine learning at scale. And you can see, you know, like it was kind of, it was perceptible, but it wasn't where it is today. And then you fast forward 10 years and you can see it's everywhere, of course, but what's really interesting in this data is if you look at large language models and you look at generative AI just in the last six months, it goes from basically being esoteric to just absolutely everywhere. So I think what's really happened here is we've brought it into the mainstream, chat GPT, and, you know, we've now kind of hit virality to the point where my parents, who are the least technical people on the planet, know what it is and they actually play with it. My father's a criminal psychologist for the record and he knows nothing about machine learning. He just knows we have free food at Google and he asked me about it every time I talked to him, but he's 75 and he likes these things. So and if you look at it in another axis, you look at the VC finding, which is interesting.  And this is going to scale even further because there's been dry powder in the VC world the last couple of years with the pandemic and, you know, holding people holding back on investments and so on. It's it's going to start scaling even crazier. And of course, the amount of compute actually these days, semi dictates how ambitious you can be. So we actually see this kind of feeding frenzy with startups trying to get compute deals and trying to, you know, secure the ability to train really large models. So this is actually poised to even go crazier. And I think one of the things that's interesting, so obviously being at Google, I have a different perspective now after kind of being a metaphor while, you know, the attention paper, we all talk about the attention paper is actually the semi inception point. It's wild to look at the start the founders of startups have just come off that paper alone. You can see, you know, well, I'm technically she should Nikki have actually left adept, they're actually starting something new as a few months ago. But you can see, you know, Noam, obviously a character, folks leaving for cohere near open AI. It's, you know, inflection obviously just came public with with pie. But just the amount of money being raised and the scale of these companies and just the the dispersion of the talent from that one paper has been incredible. And obviously it's given a lot of heartburn to Google as a company because we've lost a lot of great talent in the process. But what we've learned is scale actually creates these emergent properties. This is from our palm paper, which is now available as an API.  And this is in a lot of this progress is actually being pushed by the startups and obviously large companies as well. And, you know, I have to talk about parameters, even though parameters don't mean that much these days. They it is still one way we look at model scale. But what I'm kind of proud of is my old team that at Meta creating the Lambda models, they prove that you can actually do a lot. You can look at the 540 billion parameter model from Palm or you can look at GPT three. But with the new scaling laws, you can actually do a lot with even under, you know, 100 billion parameters or even the models are being released by say mosaic and others in even in the single digit, parameter billions of parameters, they're still quite capable. So I actually firmly believe there's going to be a bifurcation of this. And we're going to talk more instead of parameters, we're going to talk about capabilities. And those capabilities are going to be, you know, there's going to be a smaller parameter count or smaller size models. But then there's obviously going to be these world models that only big companies or people that can access a lot of compute can can actually train. And so we're in for an interesting few years, I guess. But we're here to talk about PyTorch, right? So let me let me give back to PyTorch for a second. So I want to give everyone a little bit of a history, historic perspective on where things came from. So we started out, I guess, who is a user of Lua Torch? Anyone? This is going to date you. So be careful if you raise your hand. Because, you know, yeah, okay, so no one's no one's brave enough.  But really, PyTorch was born out of the Lua Torch project. So we're not in Colbert who's at Apple now, if you actually one of the more interesting things as inside is if you go to the PyTorch GitHub repo, look at the licensed TXT. It's really interesting. It goes back like a couple of decades, actually, there's a lot of people who actually have copyrights on that code, including many folks you wouldn't expect. But this really all started, I would say in the research world, I was at that ICML, where I think they were hosting a workshop at Microsoft, and, you know, they announced, you know, support for Python on Torch. And and that was that was it. It was boring. It was it was kind of Python on on Lua, basically as a binding. And the goal there was to, to basically be one with the community, right? The Python community was was really growing was was diverse had a lot of libraries. How do you connect with that, that community? Well, John, the Coon will still say that Lisp is the best, right? You should go and use Lisp even though no one uses Lisp. But I think Sumith is right. Sumith was right at the time. And so was Adam Pashka, who was actually the actual developer who created PyTorch. And I'm not sure if you can actually read that tweet, but that that is the sun will tweet from under carpati that goes back, you know, I've been using PyTorch for six months now, my my skin is clear. You know, the air smells fresher, you know, I have more energy, right? And that kind of started off as kind of a silly tweet at the time, but it kind of set off this this virality. And and people got really excited and started using it. And then he fast forward a year or so here. And this is kind of what I joined from. From the other side, I was actually at Amazon prior. And we started to really build the community around it.  Fast AI was just Jeremy Howard, and, you know, whatever time he could get from Sylvan, Google, and a few other folks, the open mind community around privacy, they were very early days for a lot of these communities, but they were building on PyTorch because it was easier to build on them because it was Pythonic, it was defined by Ron, it was just easy to integrate with Pyro out of Pyro team out of Uber. And then, you know, boom, we announced PyTorch 1.0. This was our first first year we did a DevCon. And we started to really in earnest build out our partnerships. So you can see some of the partnerships there. TensorBoard, I remember bringing a case of scotch over to Google at the time and after they landed all of our pull requests, so you didn't have to have, you know, a TensorFlow dependency, you could just, you know, from torch.utils and port tensorboard and, you know, specify your log deer and you were like up and running on TensorBoard. Beautiful. Torch serve, we built with Amazon, MLflow, we built with Databricks, Kuflo with Google. Obviously Lightning was one of those projects that came out of FAIR actually as an intern, Wil Falcon. And then obviously we started to see the demand for the production ecosystem start to take hold. So it wasn't like we were going into Amazon and Google and Microsoft anymore and saying, hey, please support PyTorch, they were coming to us and saying, you know, help us support PyTorch in our platforms. And you fast forward all the way today and you see what's become of that. If you go to papers with code, which I highly recommend that site, by the way, you can go to something, I forget what it is, it's like trends, I think it is. If you go to paperswithcode.com slash trends, it'll actually show the framework usage and how many libraries and papers actually get generated by framework. And you can see that big orange blob in the center is PyTorch. It's just incredible. ChatGPT obviously built on PyTorch. We built that partnership with OpenAI, they went all in PyTorch, stable diffusion, built on PyTorch. The foundations are there. It's pretty incredible. PyTorch basically is the revolution. So I do want to actually talk a little bit more about why we invested, though. So I get this question a lot. And it's always, well, why did Facebook, or why did Meta, why did they do that? Was it altruism? To some extent, yes. I think we built PyTorch with that in mind. But there were six reasons, I would say, that we fundamentally, when we actually had to go to our C suite and say, here's what's successful, which you have to when you're burning hundreds of engineers, and you're, at one point, I was having to stand up in front of our exact team and say, we are actually prioritizing the bugs that Uber is finding instead of our own production teams. Those are hard conversations, but we actually were able to convey the long game. And the long game was when you have a common platform and framework with the community, you get things like a timeline edge. You get to see, in some ways, the future. People were building libraries on a daily, weekly basis. I would get, you know, it was a very small team at the time. And I would get these, you know, pings over chat or pings over Slack, people would want to get onto our ecosystem page. And you would see all kinds of new things happen, whether it was around computer vision, whether it was on NLP, whether it was new training loop abstractions, different tools and things that would make things more productive to use PyTorch, take PyTorch into the world of reinforcement learning, whatever it was, they were all built on a common framework. This was incredibly powerful. So we started to pull these things actually into Facebook. We started using them. Corny, I was one of them, for example, in the computer vision world. PyTorch Lightning today is used at scale for a lot of the models. Obviously, software leverage. We took these, you know, I mentioned geometric lightning. We took these internally. Hardware. So one of the things that, you know, being kind of self serving a little bit is by creating a ton of demand in the ecosystem. Every hardware vendor was forced to integrate with us, right? So this was a nice little Trojan horse that said, okay, well AMD or Graphcore or all, you know, and then it becomes much, much easier when you're on a common framework. Then you can benchmark, then you can evaluate. So there's, you know, I'm not sure we actually ended up taking any of that hardware internally and in front, but we had the option because we were on that same platform. And then obviously there's other pieces that are more, I would say, like vanilla open source, but controlling our own destiny was actually pretty important.  But controlling our own destiny was actually a big reason why we did what we did. And, you know, truthfully TensorFlow, for example, was on the list of things, you know, one of the options we considered. And it was at the time, it was growing like crazy and we considered it and we looked at the governance, we said, okay, what happens when we're in a situation when our production team needs to land code and we are at the mercy of some other team at potentially Google, right? Not a good place to be in. So I think controlling our destiny, we actually governance, if I had to boil everything down, governance actually was a very, very important reason for the decision. But there was one big thing that we need to think about for this to actually all work. We had to take a community first approach. And I think we probably took this to heart probably more, more than I would say than most believed. And so if I kind of boil it down to a tagline, I would say PyTorch needs to be seen as a neutral brand, giving first class participation to every major stakeholder investing in the ecosystem. So this wasn't that, you know, PyTorch was an open source project that we kind of owned and held and there were these contributors out there and they were nice and we just kind of gave them, you know, the ability to access the code because it was an open source. No, we, they were literally on the same level as anything that was coming out of the company. So whether it was a library coming out of fair, whether it was, you know, a new capability that was getting launched, a blog, whatever, everyone was equal. And we fought on a daily basis to make sure that happened. And we took that really to heart. And I think that's a lot of the reason why we were successful when we were. I mean, I think if you look at some of the projects over time, I mentioned Open Mind, CAPTOM, we went to the point of almost detriment. Lightning, for example, you know, if you look at the training loop abstraction library, I mean, who uses lightning today? Anyone? A few folks, okay. So how many, who knows, like, about all the other training loop abstractions like Ignite and Poutine and Catalyst and no, no, okay. There's a bunch of them, right? At one point I was getting at least one or two a week where people would find a cool way to abstract the training loop because we never had a model.fit in PyTorch and still don't today. Like there's no Keras like model.fit. And that was actually by design. And we had to hold the line there because we wanted the full kind of exposed functionality for researchers. And if you want to use an abstraction, you can use lightning or you can use Catalyst or you can use something, you know, that is maybe a little bit simpler and easier to use. That's totally fine. That's where the community comes in. With projects like Open Mind, for example, you know, we were releasing our own libraries around differential privacy. There was a project called Opaques, which was a great differential privacy library. We actually went and shared it with the Open Mind community even before launch and let's figure out how to integrate. Let's figure out how to collaborate versus going in community and competing with the community. So we went to great lengths to make sure that we did take a community-first approach. And I think it was actually really successful. And I guess, you know, the results kind of speak for themselves. I would say, you know, network effects is what I would boil us all down to.  But it came with a strategy and it came with a vision and we had an operating model and we had, you know, air cover and we had a vision and, you know, we were able to execute it. So why is this guy at Google talking about PyTorch? So, by the way, there's a few folks here from my team definitely hang out with Milad and Shaheen. You'll want to chat with them a little more. They're definitely, they've been working on PyTorch a long time, so definitely chat with them. So at the highest level, Google is investing in every layer of the stack. We build Silicon, we build servers and racks and data centers and all the way down to, you know, to infrastructure as a service, both internally and externally. We have an ML platform that provides basically, you know, your kind of traditional things like notebooks and MLops and all the way through to models as a service and to kind of agents that you can then deploy in your applications. And you can see it's kind of a full stack. So depending on what you're talking about, we're deploying more and more obviously models as a service these days or providing APIs for models like Palm and providing BART as kind of a platform for experimentation evaluation. So you can basically, you can say we're doing pretty much everything you can in the AI space, I would say, every kind of niche of it. And if you look at PyTorch, especially down in these two levels, this is really where we're investing. But what is it concretely, I guess? So this is a project that we actually started back in 2018 when I was at Metta and actually working with, you know, Vishal and Shaheen and Zack Stone and it was it was a collaboration between between the two companies with the single focus goal of basically supporting cloud TPUs kind of cutting edge hardware on PyTorch. And I think if you if you're at the first DevCon we did in 2018, you saw glimpses of it, right?  But you can see basically we made the project was available, it started in 2018, started to become available in 2020. It basically the core of this was a lazy tensor idea. So, you know, you kind of think about the continuum of ways to execute operators, you can compile operators. So you have a defined by run, which is kind of the eager mode, you know, way pythonic way that PyTorch executes operators at the whole other end of the spectrum, you have something called AOT or ahead of time, full graph compilation. And then we found this basically this approach that's kind of sat in the center called lazy tensor. And lazy tensor is essentially you're really only executing executing ops when you need to execute those ops. So it's kind of a it kind of it's in theory, it brings you really the best of both worlds. It brings you the performance of the hardware, but it also gives you the near kind of pythonic experience that you're, you know, used to kind of running an eager mode. Is everyone familiar with like the lazy tensor or eager mode or your mode in general? Okay, cool. And then, you know, in 2.0, we started to integrate that with torch dynamo. Hopefully folks have kind of heard all the compiler noise that came out with PyTorch 2.0 dynamo and inductor. And you can start to see, you know, really the performance for both TPUs, but as well as GPUs. So this project really I would say sits kind of just below the PyTorch layer. It provides a pathway to something called OpenXLA, which is a new open project by led by Google and many others, Waze here from Intel. And this basically provides you kind of this open compiler back end, which then can, can then connect to any XLA device. And it doesn't need to be a TPU. It can be a GPU, it can be XPU, whatever type of device you're looking for. In fact, it could be even a mobile device.  You have Ray, Hugging Face, Lightning, you have the whole thousands of libraries that are out there. And so a lot of people ask, like, what do I need to do? I'd like to get all this great performance. Well, actually, the code changes are pretty minor. So if you look at what I kind of highlight here, if anyone's familiar with, you know, setting a device in PyTorch, right, this CUDA devices, GPU, CPU device and so on. So you're going to set an XLA device on this case that then would would propagate through your, your step all the way through, you know, through your model. Basically, your model code may change like a couple of lines here and there. But the main thing is you need to import, say, the Torch XLA library and then basically set a device and everything kind of works from there. So pretty minor changes. But what we get is actually a lot of performance. And so I actually expected GPU data on this Milad, but I never got it. But this is all TPU data. And this is all latency. So we've actually been doing a number of optimizations for Lama, for example, working with the meta team, even at 175 billion parameters, which is of course, the GPT three scale, the latency is incredible. And this is, I think, probably an order of magnitude lower than what the original latency was. I think it was around 200 milliseconds. And now you can actually unlock the ability to actually serve, you know, these models, these large language models at scale. And of course, latency for 7 billion is like negligible, right, three milliseconds. So incredible performance for both TPUs and GPUs. So I'll cover a little bit about ways to get involved. But honestly, this is all motherhood and apple pie for an open source person. So I'll go through this. And by the way, feel free to ask me questions. Actually, don't mind being interrupted. So I want to call out a couple things. So one Jeremy Howard is awesome. I've known him for ages now. He continues to build his courses on PyTorch. And so he just actually released fairly recently the practical deep learning course. He moved in his family all the way back to Australia. He's but he's still advising and still heavily involved in community. I highly recommend taking his course. So way number one of getting involved, take a class or take a take a MOOC or play with tutorials. PyTorch has incredible tutorials. When I was there, we worked really hard to make them great. The 60 minute blitz is still probably the most popular thing. I think on the PyTorch website. And that is, you know, at this point, quite a few years old. The coolest thing I think that we actually did on the tutorials when I was with PyTorch is we actually had co lab integration. So who uses co lab? Okay, I'm going to see some hands this time, hopefully. All right. So co lab was amazing. So one of the things that I did with my team at meta was we taught a class at a master's level course at Georgia Tech on deep learning that we then brought into something called the AI learning alliance to drive more diversity into AI. We taught that course almost entirely in co lab. And then the students would actually go and pay the 50 bucks for co lab pro and get you know better GPU and a longer runtime. And it was it was absolutely incredible. So you can actually click through on most of the tutorials on PyTorch or right into co lab. And you can start playing with code right there. You don't need to spin up a VM or pull out your credit card or whatever. It just works. It's beautiful. And there's of course GPUs available for free. So I would definitely check that out. Another thing you can do is you can compete in a Kaggle competition. So this is kind of a cool. Well it's a bit bittersweet but mostly cool graph on the left here. This is actually deep learning usage on Kaggle a PyTorch versus TensorFlow. And you can see that dark blue by the way is PyTorch. So PyTorch is everywhere in Kaggle which is pretty awesome. I was actually looking through the challenges when I was putting these slides together. There's a million dollar challenge. It's got a baseline in PyTorch if you want to play with it and try and win a million dollars on the Vesuvius challenge the ink detection. Go for it. Kaggle has really incredible statistics have 13 million users million monthly. They have up to date PyTorch 2.0 Docker images 36,000 monthly PyTorch users on Kaggle. So it's pretty cool. And the challenges are awesome. And obviously it's a great badge of honor to win a competition on Kaggle. And I think lastly, just again, motherhood never pie, host a meetup or join a meetup. I saw Imad, its stability is going to be hosting one, I think in two weeks in New York over at the Lightning AI office. So, you know, join a meetup, get involved in the community, come to these events, right? Talk about your work. I think, you know, we've been blogging a lot obviously with the LF. It's been awesome. The I hate to say how the old blogging used to do. We basically it was a pull request that I used to just merge myself, which is dangerous. It was a loophole in the system. But talk about your work. You know, reach out to the LF folks. It's been actually pretty incredible. We've been blogging from the Google side on pytorch.org. Tell your story. There's actually a community stories page as well.  And if I can help in any way, either personally or if it's from Google, whatever, I'm happy to help. I love building this community. And I'd love to help tell your story as well. That is it. Thank you. And I can answer any questions you have.


One institute, one focus on sport and movement. Five departments, let's get started. We researched the mechanisms and effects of physical activity in childhood and youth. We improved talent selection, career development and transition out of elite sport. Mastering and perfecting movements, that's what we focus on. Sport is socially relevant. We analyse conditions for successful integration. Our vision is moving towards better health, interdisciplinary and multi-method research to move the sport system. Welcome to the Institute of Sport Science at the University of Bern.


Welcome again, everyone, to another NARC Survivor Live video. Just checked into this new room. I'm near Osaka in Japan. Still just exploring. Driving around, travelling. So straight to it, in this one, a very important topic.  Before I begin, do show your support down below by giving this video a thumbs up. It helps to support the YouTube algorithm and gets this message out there to other survivors. So let's begin, let's get straight into it. I know it may seem like very egotistical. They're very full of themselves. They think they're the best. And it's as though you need them. They don't need you. That's the narrative that they portray, which doesn't really make any sense because they are always there. They're always around you. It's like you can't get rid of them. And yet they make you feel like you're the one who needs them and they don't need you. That's what they make it seem. So those of you who haven't gotten to this point where you have cut the narcissist off or got no contact, you might assume that maybe they just won't even care. They won't even bother them. They won't even notice. But actually they will notice and they will be very sensitive to that rejection because there is nothing that hurts a narcissist more than rejection. So when you cut them off it will cause a narcissistic injury which will be followed by narcissistic rage. Yes, they will get very angry. But don't get it wrong. They're not angry because they love you or because they miss you. It's nothing like that at all. Now this goes for whether you're dealing with a full-blown narcissist or someone who has narcissistic traits. I mean just anyone who is not empathic, anyone who lacks empathy, anyone who is not an empath which means that they do not possess the amount of affective empathy and what affective empathy is as someone who has high amounts of it I can tell you if I see someone in pain in distress, that hurts me. I might even feel it as if it's happening to me and it doesn't have to be physical pain. It could be someone going through emotional distress, psychological trauma. I will feel that just as much as they do. I will take it on as if it is my own pain and before I became an awakened empath I might have even have assumed that it was my pain more than anyone else's and it's kind of a blessing and a curse at the same time. I mean, if you're around another empath you can both use this beautiful gift to share with each other, to love each other, to understand each other but when you're around narcissists they will use this gift against you. Yes, they will use your affective empathy against you and it will feel like a curse when you are around them because they feel a lot of shame and a lot of pain which they will project onto you and because of your empathy you will identify with it you will take it on as if it's your own but as I said, when you cut them off and it makes them very angry that has nothing to do with you as a person it's not that they miss you or love you or that they care about you it's not that they are angry with people who are not empaths people who lack affective empathy because it's that affective empathy that gives us the ability to connect to another person to be able to feel and understand their experience and emotions as if it was our own I mean, that's really what a true connection to intimacy is and yet, believe it or not but only 1-2% of the world's population are empaths who possess high affective empathy which means that true connection and intimacy is actually very rare but back to the narcissist or those who have narcissistic traits you've got to look at it like this everything that they perceive it's from their brain their brain just as ours does, it gives us the ability to perceive things it allows us to have these 5 senses sights smell taste hearing touch these are our 5 senses which are controlled by the brain take the brain away you can't perceive of anything so everything that they perceive of is through their brains and when they're interacting with us as with anyone else it is their personal experience which is separate from our experience so how they perceive of us when they're interacting with us it may have very little to do with who we actually are or how we perceive ourselves because even if we are presenting ourselves authentically it doesn't mean that they are receiving us in that same way unless, of course if they do possess high amounts of effective empathy if they are one of the few people in this world who are empathic because as an empath myself and I'm sure many of you can relate to this you can feel another person's emotions and pain and especially if you are an awakened empath you know exactly what they are feeling even if they don't openly reveal it to you because you can feel it as if it's your own emotions but when you become an awakened empath you can then dis-identify from it you can then recognize hold on a minute these are not my feelings or emotions and what can help you to do that as well is when you realize that before interacting with them you didn't feel that way so where did that energy come from? obviously it came from them but also when they come around us as empaths many of us are high vibrational people and when they come around us they sense that we are happy and contented that we can experience joy and because they can't experience these higher frequencies it makes them very mad and you will see it straight away they will try to pull you down to their level and they don't even have to do anything all they have to do is just be near you that's enough to bring your vibration down because as an empath you can immediately sense it so you really need to see it like this because this is the reality of it and it may completely change how you've thought about them up until this point not just how you've thought about the narcissist but people in general I mean just remember how everything that we perceive of it's all operated by our brains and without that affective empathy it's like we are kind of cut off from the world a person who is low in affective empathy they kind of live in this world inside their heads with a population of one which means it's just them there's no room for anyone else to come in because they're unable to empathize they're unable to feel or understand other person's emotions and experiences as if they were their own and this is why the world is the way that it is today this is why empaths are so rare why there's so many wars so much problems with divorces relationships, families it's because most people they do not have the ability to feel and understand another person's emotions and experiences as if they were their own most people do not have the ability to do that so even when you cut off the narcissist and it looks like they're so mad that's all just anger frustration and shame and it's disconnected from any experience with you it doesn't mean that there is a connection between you and them it doesn't mean that at all they're just at the effect of you cutting them off and then they are responding to it to try to keep you down so that they can then elevate themselves at your expense they have cognitive empathy where they can read, they can detect how you are feeling I mean even a child can do that they can tell when you're happy, when you're sad they know what to do to push your buttons to make you angry or upset even a child can do that but the effect of empathy is to actually feel another person's emotions as if they were our own now narcissists cannot do that and this is how you may have noticed they can really hurt you they can cause you a lot of emotional or even physical pain and it's like it doesn't even bother them it doesn't even affect them but for me if I see someone in pain like if I see a homeless person on the street I just have to help them out if I see a cat or a dog crossing the road and it looks like a car is about to hit it then it will affect me I might flinch I will experience something in reaction to it and although at times people may make me angry they may upset me I don't desire to hurt them even in the rare case where if I did do that I would have to make it up to them somehow what I'm describing is affective empathy this is something we feel as empaths empaths are 1-2% of the world's population other people do not feel that they can quite easily watch you in pain leave you in pain and not do anything about it and not feel any need to be there for you to support you or to resolve it most people can quite easily do that and feel nothing it won't even affect them because they lack effective empathy they do not feel or share feelings and experience as if it was their own so they have an inability to really be vulnerable or to connect and this is how you know that even when you cut the narcissist off and then they go straight away to their new supply they flaunt it in front of your face to try to make you jealous what they have isn't even real it's not even real they have an inability to be vulnerable to connect or to experience true love, intimacy, joy and you should already know that because if they have the ability to do that and to care about another person why didn't they care about you how were they able to watch you in distress and not step in not do anything to support you or to resolve it this is how you know wherever they go whatever they do whoever they are with they are still miserable they're still not happy they still don't care about anyone but themselves they're still imprisoned psychologically in their own minds in this world inside their heads with a population of one meaning there's no room for anyone else but themselves in there and it's because they chose a different path as most people do when they're children when they experience a traumatic event when they're going through something as a child whether it's abuse or neglect whatever it is I mean we all experience trauma in our childhood and the reality is that most people they are not strong enough to choose the right path to choose to be empathic rather than narcissistic so what happens is that doing this time in their childhood when they experience abuse or neglect or whatever it is because they're too weak to deal with the shame they avoid it they deny it they deflect it they just don't go inwards to deal with it and their parents aren't there to guide them through it to make them feel better to help them to regulate so what happens instead is they become shame-based people and they never learn effective empathy which means that they never have this incredible ability like many of us have on here as rare as we may be to actually connect to someone and feel what they feel as if those were our own feelings so this is what you have to realise with not just these narcissists but most people in this world and I'm sure many of you have realised this already most people do not feel what you feel they do not care the way that you care I hate to say it but the truth is a lot of you who are watching this I mean we have to consider that most definitely there's no more than 15% of the world's population who actually have full-blown narcissistic personality disorder of course yes there is COVID narcissism as well and those who just have narcissistic traits but I would say that many of you you may actually just have been dealing with someone who's not too far off normal but you may assume that they are narcissists because they don't seem to care they don't feel what you feel truth is that's going to be the case with most people in this world they are not going to feel what you feel they are not going to care so much they are going to be more worried about themselves more than anyone else because most people are not in paths most people are not looking to put other people before themselves like you or I so it doesn't always mean that you were dealing with a narcissist of course if you have witnessed the traits such as the arrogance the sense of entitlement the grandiosity the narcissistic injury the narcissistic rage if you've witnessed all of that definitely makes it more likely that you are dealing with a narcissist but even then we cannot diagnose what you need to know what you need to understand is that there are very few people in this world who will care the way that you care who will feel the way that you feel and how do I know that? well for a start you're watching my videos this information resonates with you you obviously care enough about that narcissist you are dealing with and that relationship that you are in to come on to here and watch this video and right now it's over 28 minutes in shows how much you care shows how much it means to you reality is that most people in this world they're not going to care that much but I'm not here to judge whether or not you actually were dealing with a narcissist I'm sure that many of you were but also many viewers likely were not they've been someone with narcissistic traits maybe a covert narcissist or could have just been a normal person normal people they actually do not care that much about other people they really don't I mean just look around at the world there's wars all over the world there's people who are homeless they've got nowhere to go no money no food do most people seem to care? no they don't care like I said most people in the world are living in a world inside their own heads and they're the only ones in there because most people are not empaths they have an inability to connect to other people to feel their emotions and experiences if it was their own and this is very important for you to be aware of and this is why I really want to drive this home because of my own experience I remember when I tried to cut the narcissist off when I rejected them I was shocked because I was thinking all of this time you devalued me you made me feel like I'm not important like you don't really care about me like I don't matter and now I'm trying to leave I'm trying to cut you off and suddenly now it's like I'm like the most important person in the world and you could be fooled especially if they come back and they try to love bomb you again they make you believe that this time they're actually about something it could fool you into thinking into believing that they actually care like they actually feel your pain after everything they did to you after how they treated you yeah it could fool you into believing that as it kind of did for me on a few occasions but no when you see when you try to cut them off and you see them react in that way where it looks like finally they care so much they actually don't care about you at all it's still all about themselves they only care about themselves most people in this world only care about themselves they do not feel what you feel they only feel for themselves I'm sorry to say but that is the truth we've only got to look around at this world and see all of the pain all of the suffering in this world there's so much poverty there's so many things that people are going through and the reality is that most people do not care they really don't and it's very different to many of us who are here right now because you're watching my video I already know that many of you are just like me you may be highly intuitive highly perceptive empathic you have this ability to read into other people's experiences you can put yourself in their shoes and for that reason you may choose to spend a lot of time alone because it's hard taking on other people's energies all of the time you don't really like being in public spaces too much but when you see someone in pain it's like you just can't help it you want to step in even if you know that it's going to affect you you might even put your life on the line just to help someone in need you might do whatever it takes you may be the type of person to jump in front of a bullet for someone someone you don't even know because it's that empathic concern it's something that you've learned in childhood sadly it does tend to come from abuse we were abused as children possibly by a narcissistic parent who made us believe that their feelings and experience are more important than our own so we will sacrifice ourselves for other people just to make them happy even when we're not getting anything out of it and that is real connection real intimacy but believe it or not a lot of people they are actually repelled by that and you may have noticed this with the narcissist that you were dealing with Dr Ramani did a video on this as well where she said that narcissists actually hate our empathy and they are very jealous of it because they can't experience at a deep level like we can they can't connect they can't get the same fulfillment because they chose a different path they were too weak in their childhood to deal with that shame we weren't, we self-loathed we confronted it and many times we confronted it on our own our parents weren't even there for us we regulated our own emotions but narcissists couldn't do that so instead they abandoned their true selves when they were children and then they created a false self who's all powerful but also very manipulative they have to manipulate people abuse them so that they can regulate their emotions because they never learned how to do that on their own in their childhood so when you cut them off they're not actually feeling anything for you everything they're feeling is just for themselves they have an inability to connect to you and it may really shock you when you see their reaction it's like the entire time they never even cared about you it's like they never even knew that you were there half the time unless they needed something from you but then as soon as you cut them off now they're all over you now they're pulling out all of the stops to sabotage you to prevent you from moving on they will spread rumors behind your back they will gossip about you to other people they will try to get you fired from your job they will try to affect you financially they will start a smear campaign against you they will stalk and harass you but even then all of these things they are doing it's not because they care about you it's not because they love you or they miss you or that they want to be with you it's nothing to do with that at all it's all for themselves because they're losing their supply and their entire identity their false identity it revolves around you it revolves around manipulating and controlling you and keeping you in a certain state or condition because if you can move on from them and you become successful in your work you get into a new relationship you're happy, you get married you start a family, whatever it is all of these things it affects their false sense of self which is tied to your position in life it's tied to you being unable to move on so they have to prevent you from moving on and of course, since they're narcissistic they don't have effective empathy they can't be vulnerable, they have an inability to connect they don't care the way that you care they don't feel the way that you feel so all of these things that they're doing to you whether they're stalking you, harassing you gossiping about you maybe they got you fired from your work you lost all of your money because of them maybe they sabotaged your new relationship they turned everyone against you they're doing all of these things and they do not care they're not feeling your pain or distress it's not doing anything to them we are nothing alike they are nothing like us their brains are very different I mean, I can tell you if I see my cats Nala as an example when I would see her being sick sometimes it would happen when I'm travelling it's hard to get the same food sometimes so there are occasions where I've had to change the foods or the treats and then later that day or the following day I would see her being sick on the floor and it's such a horrible experience for me it just goes right through me I feel it in every part of my body and it makes me feel sick as well it makes me feel hurt and it just makes me want to do whatever I can to help her from going through that experience and of course to prevent it from happening again because I feel it it's affecting me but believe it or not when they're doing these things to you it is not affecting them they do not care the way that you care about them they do not feel the way that you feel about them they just do not feel the way that we feel but they try to trick and mislead you by making you think oh no, it's not it's not that I don't care it's not that I can't be vulnerable or I can't be intimate no, I just can't do that with you that's what it is that's what they want you to think and then they run off, they ride off into the sunset with their new supply looks like they're living their best lives but that is not physically possible it's not possible for that to happen because if they can just move on and care about someone else they would have cared about you they would have felt your pain and they would have done everything in their power to prevent it to make things right so it is all a lie it is all fake how you do one thing is how you do everything if you can just sit back and see someone going through it and not step in that means you are not an empath it means you lack effective empathy it means that you are essentially imprisoned in your own mind and you are unable to connect to another person that is exactly what that means as crazy as that sounds that is exactly how it is for them and it's not something that is ever going to change because they have a different perception a different concept of winning of being powerful, of being strong quite clearly to us what's most important it's connection true love, intimacy that is what matters to all of us here I mean that's why you are all here isn't it it's because you didn't get that from the narcissist and you didn't get that because that's not a concern to them they can't even experience any of that anyway they are more concerned with power and control through manipulation and deception that's all that they are concerned with so anything that you see where it looks like they are involved with another person it is all fake there's no connection there's no love, there's no intimacy it's just potentially two people who are having separate experiences I mean they are there physically together when you see them in the pictures but they are not actually there they are not fully experiencing each other that's how it looks like they can just move on they know everything is so much easier for them everything is going so well now with this new person because it's different it's not like how they had to deal with you where you are this you're this empathic person you actually care, you actually feel you're worried about a connection about intimacy, about love all of these things the new person is not too concerned about that it's not a big deal for them like I said, like attracts like they find people who are like themselves we just got tricked we weren't really attracted to them you're not attracted to the narcissist you may think that you are I mean I don't want to dictate your feelings but I want to help you to see that this person that you think you're attracted to no actually you're attracted to the false character it doesn't even exist I mean look at who they actually are this manipulative, abusive, deceptive person of course you're not attracted to that you don't want to be with someone who abuses and manipulates you really want to be treated with love and respect and that's not who they are so you don't really love them you don't really even want them you never did they just tricked you into thinking that you did and they tricked you into thinking that they've moved on and now they have what you wanted with someone else when that's just another lie they've just found someone who's more like themselves someone who's more narcissistic they lack empathy, they have an inability to be vulnerable or to connect, or to be intimate or to experience real love their needs are very much the same they don't really care about each other they just don't feel and something like that it can actually work quite well especially in the beginning or at least they can present it very well because neither of them are concerned with intimacy, love or connection neither of them can experience that they're just more concerned with superficial things the image so they can put on a great show for you and they can make it very believable too but you couldn't do that with them because you cared about them they didn't care about you you felt their feelings as if they were your own they did not feel your feelings as if they were their own so there were all of these arguments and they would start arguments a lot too because they do know they're not completely detached they know where they belong they know that we are in paths and they know that we feel on a very deep level they're far more sensitive than they are at least in terms of intimacy and connection when it comes to anger, hatred envy, jealousy and frustration they're far more sensitive to that than we are but they will try to project these emotions on to us as well but this is what you have to understand when you are dealing not just with narcissists but most people in this world because if you google it you will find statistics show that only 1-2% of the world's population are in paths and even then how many of those are healed and awakened in paths probably very few of that 1-2% and this is why I already know those of you who are in paths you feel very alone you feel like no one understands you no one feels what you feels no one cares about other people animals, the environment and the way that you do the reason why you feel that way is because it's true your feelings are correct 100% correct and it makes sense because I don't believe that it was always this way if we managed to make it this far as a population but now at this point empaths are a dying breed there's becoming less and less of us in the world and narcissism is being promoted I do believe that there is an agenda at an elite level they're deliberately pushing this on us and they're using every possible method that they can to do this and it's because they don't want us to reach our true power which is where we are empathic it's our higher ability higher frequency they want to keep us all down because then we are easier to control if everybody is vibrating at a higher level we're all at the vibration of love that is very difficult for the powers that be to control of course if we're all experiencing a lot of anger fear, guilt, shame even desire or pride that is very easy for them to control so this is why when you look around it's why the world is the way that it is today as sad as that is that's why and the reality is that most people do not care about that even if they are already aware of it most people only care about themselves they're not concerned about other people in the way that we are that's why it's very important for us to have our own community of like-minded people other empaths it's really good that we have this where other people understand because that's really what we need is more understanding people who understand each other most people in this world they are more about separation they're more about their own groups of people they're not about diversity coming together when that is exactly what the world needs that's what we should be encouraging in this world but it's unlikely that it will ever get to that point because the reality is that most people only care about themselves and they're just using each other to manipulate and control to get their own needs met because that's all they're really concerned with it's their own emotional regulation and maintaining this four cents of self around manipulating and controlling us back to the narcissist how they feel when you cut them off they feel a lot of anger and frustration but that's only for themselves it's not for you they don't really care about you they don't really see you they don't really feel you it's just all about themselves they're present in a world inside their minds with a population of one it's just them and no one else there's no room for anyone else in their heads keep in mind everything is experienced through our brains but with empaths it's a very different experience we can share another person's feelings and experience in our own minds in our own brains that is incredible that is true connection true love, true intimacy I do see a lot of hateful troll comments coming in now though I'm just going to call them out we've got to start shaming these people Manda Jamali says you are a demon a karsh batacharya says you look like psychopath Kartik Katar is telling me to remove my glasses what else have we got here Gary Beerboy says you're like a villain in a movie a karsh batacharya says he's just talking shit Farah for you says hi Baldi hello Shambu says WTF are you speaking these are all perfect examples of what will happen if you go to a narcissist and actually I'm telling you if you go to anyone most people in this world if you go to them this could be your own family, your own friends people who are meant to be the closest people to you and if you feel like they're not hearing you they're not listening to you, they don't care about you just go to them and tell them I don't feel like you care about me I don't feel like you're hearing me you're not listening to me you're neglecting me I don't feel like you're my mother, father brother, sister, friend relationship partner, whatever it is husband, wife, child go to them and say that and watch how they react that will tell you what exactly you are dealing with because I've experienced it myself in my own life with many different people and what do they do when you speak on it and you suggest that or you just express your feelings that you feel like they don't care about you in my experience they got very mad they kind of went to making accusations against me and acted as though I'm selfish and I don't care or they just started making demands they became very entitled they went into denial or just blaming but you know what they never did they never actually took any action to correct it which is very funny because in pretty much every situation in my life whenever someone ever came to me they confronted me and they may have been completely right in doing that and they told me that I should care more about something or someone I need to do more, I need to be more I'm thinking yeah, what have I got to do? just tell me what to do and I'll do it and I did do it every time I went above and beyond, I went out of my way to please them, to satisfy them, to make them happy most of them though, they never really cared they never said thank you they never appreciated it they just became even more entitled and made even more demands but never did anything for me in return and I know I'm not alone I know there's other empaths out there who have been through this exact same thing as I've said before most empaths in the world they're likely to be alone and isolated they're almost always the scapegoat child they're targeted by their family maybe in their workplace, their school in their community this is just how it is this is just how it goes and I would just like to say I would like to speak on behalf of everyone I am very sorry for those of you who feel like you are alone in this world you feel like you don't matter you feel like no one cares about you and I want you to know that I do care I read your comments every day I've been doing it for over 5 years now and I do care about all of you a lot everyone who makes themselves known on this channel and you show your appreciation to me you show your support I care about all of you I really do and I think it's very important that we do show people just how much we care we need to promote and encourage this type of behaviour I mean this world is just getting worse there's a lot of things that are going on and it's becoming normalised things that just never would have existed 10 years ago, 20 years ago it just keeps getting worse and we need to be the change that we want to see in this world it all starts with us we can't change other people but we can change ourselves we can be the change that we want to see and hopefully that will help to influence other people I'm asked definitely a part of my motivation it's what makes me want to get on you every day and do this to just try and spread the love and positivity because there's not much love in this world anymore there really isn't the sad truth is most people are not willing to go live on YouTube and speak in front of tens of thousands of people around the world I just come out and say that yeah, the reality is most people in the world do not care about each other they don't feel what you feel but I think it's very important to speak on this and to accept it because with acceptance we can then change it we can then make it better as long as we deny it we reject it nothing is going to change and this world is never going to change completely I mean there's always going to be positive and negative there's always going to be good things and bad things and we do kind of need the bad and the negative as well that does have a place in the world but it's getting out of hand these days I mean there's not so much good in the world anymore just look at the things that people are going through look at all of the wars and it's just not nice to see especially as empaths we just take on all of this energy as if it's our own but what you need to do when people project this energy and emotion onto you use it use that fear, that anger the grief, that shame transmute it into something to where you can help other people instead of using it to become hateful like they do like they did to you let's focus more on connection rather than hating each other I mean I genuinely believe that all of us we all have good things about us inside there's something beautiful about everyone I do believe that that many of us who seem bad we seem no good evil, whatever it is maybe they were just around the wrong people in a bad environment maybe they just spent too much time alone I can do things to you as well this is why if you put those types of people in the right environment around the right types of people who are supportive and encouraging they're motivating and inspiring you would see a very different person after a certain amount of time so what I'm saying is we are all responsible when we see anything bad anything wrong, anything evil in this world we all have a duty we are all equally responsible for that and we need to accept it we need to be the change that we want to see in the world and I take it upon myself that's why I began this channel I started making these videos because I knew no one else is going to do it though not many people at least I have to take responsibility for the bad, the evil that I see in this world and begin by inspiring and motivating people so that's really the message that I want to finish with tonight and I hope that it motivates you to be the change that you want to see in this world it all starts with us so thank you all for joining me on another NARX5LI video and if this video was helpful to you you can show your support down below by giving it a thumbs up very important as it helps to get this message out there to other survivors other empaths as well and you can let me know your thoughts on this video in the comment section down below also hit that subscribe button and click all notifications to be notified of my future videos and if you would like to book a one-on-one with me you can do that on my website it is NARKSiveiver.co.uk and also you can follow me on Instagram it's NARKSiveiver YouTube as always thank you all for joining me and I look forward to talking with you in another live video very soon


Frisbee flying with Florian Haas and Tim Serong. And I guess you all know the real topic. Yeah, the title of the talk is and remains Royal Your Own Cloud, by the way. OK, so my name is Florian. The guy at the podium with the drawing pad is Tim. I work for Linbit. Tim works for Novell. And when we're not busy attending awesome technical conferences, what we primarily work on is, generally speaking, the Linux high availability cluster stack. Tim working mostly on pacemaker related stuff, myself working more on resource agent related stuff, and writing documentation, et cetera. And what we want to show you today, in the next about 30 minutes, is a brief introduction to, well, rolling your own cloud. And as we get into this, let's first talk a little bit about what are the common qualities and properties that we typically expect out of a cloud computing environment. So and what are the challenges and problems that we're trying to solve with clouds, right? So one of these things is typically utilization. If we take a particular workload that ran on a specific piece of iron five years ago, and that piece of iron is now beyond its effective life, if we wanted to just replace it with another piece of iron, we might as well put that thing on the beach and have it drink caipirines or whatever, because we're not going to utilize that hardware fully. And underutilized hardware is something that's typically not well-liked among controllers or CIOs, et cetera. So this issue with utilization is one thing that we're trying to solve with cloud computing, because it just makes so much more sense to take all of this iron that can actually do stuff for us and give it something to do, but just throwing more workloads at it than just one. A second major challenge that we're trying to solve with cloud environments is obviously as we're adding more workloads to, say, a specific set of hardware, the whole thing becomes, well, just a wee bit more complex than just having a single server. It just becomes a bit of a more challenging task to manage all of this from one central location or to be able to do that with a limited amount of manpower. So you can sort of compare that to, let's say, a person who previously tutored a guitar student to someone who now has to conduct an orchestra. And this manageability issue is another that we're trying to address with cloud computing. By the way, if you're running a cloud, whether it's your own or whether it's a public cloud, don't worry, you don't have to wear a suit and tie. But Tim does insist on the pompadour haircut. So that's a must. Another thing, and this is an issue particularly in large organizations, is as this workload becomes bigger and more complex, we tend to want to have specific experts managing specific subject matters. So we want to have, say, a storage expert working on storage, and we want to have an application expert working on specific applications. And we might want to have a virtualization expert working on the virtualization thing. And as the whole picture becomes a bit larger, obviously this turns into not one person, but into teams. And that creates challenges in terms of coordination. So what we really want is we don't want people to tread over each other's toes. And what we want here is fundamentally known as separation of concerns. By the way, meet Jane concern and Joe concern, which just got separated, which in this case is sort of a good thing. Because that's something that we typically want in large enterprises specifically, that we have the ability to do fundamental separation of concerns, so everyone can concentrate on what matters most for them without stepping on other people's toes. Another thing that we typically expect out of clouds is, well, in a conventional, all iron data center, one of the things that are typically among the rather daunting tasks is setting up and managing and running your backup infrastructure. And conventionally, this is done in a way that, I mean, in the old days, it used to be done in a way that we would have some sort of networked backup agent running on each and every one of those boxes, and they would report into a central backup server and that sort of thing. And then it would somehow go to tape. Now in a cloud, because we typically have access to some form of centralized storage, we can also centralize our backups. And we can have just like this one big vault, backup vault, so to speak, a storage vault, that we can pull data out of and toss it onto a backup. So centralized backups is another thing that's of crucial importance in these cloud environments. And with centralized backups and centralized storage and everything, we're getting to another important feature. And that's, of course, snapshots. Snapshots allow us to take a consistent point in time image of, say, a specific workload and be able to take a backup from that or be able to create sort of a checkpoint that we then can go back to if we need it. And finally, as I'm sure you're well aware, we typically design the resources that we allocate to our workloads according to sort of the peak requirements of these workloads. We always provision resources such that it can handle the workload under full load when it needs all the memory, when it needs all the CPU cycles, all the storage, et cetera. Now it turns out that that's not exactly super-efficient. We can do something slightly differently, and that is we can pretend we have more resources than we actually have and we give them out sort of more than is actually available, but pretend to the individual workloads that the total amount is much larger. So it works not unlike the fractional reserve system in banking. There's never as much money in circulation for all of us, literally all of us, to clear their bank accounts. But as long as not everyone does that, the system works surprisingly well. And of course, in this case, if it's indeed such that all of your virtual workloads at the same time use the max of computing and RAM and storage and whatever resources, then your beautiful thin provisioning scheme essentially turns to Lehman Brothers and calls a global meltdown. But while that's not happening, the system actually works very, very well, and it helps us to very, very efficiently use our resources. Now, what we've talked about up to this point is something that we're all very much familiar with when it comes to talking about the big public clouds, the Amazons, the rack spaces of the world. They make all of that happen for us. And they also come with another kind of a handy feature, and we, of course, have to design the workloads themselves as well, and that basically turns into or gets us to virtual appliances. So what the major cloud providers allow us to do is to create preconfigured images or templates of images that we can then configure with a few clicks. And we're, in essence, capable of mass producing these workloads as if they came off of an assembly line. And we have basically these pre-packaged virtual images that we can modify and change and redeploy and deploy a hundred times or a thousand times if we want to. And all of that becomes very, very easy. Now, here's the thing. What we talked about previously and what we have known for a long time from the major public cloud providers doesn't just apply to those. The same thing or the same requirements, the same challenges, the same problems apply to what's called the enterprise cloud, which is really just a fancy marketing way of running a modern data senate, because that's how we typically do it. And such enterprise clouds are comprised of certain building blocks to achieve what we've previously discussed, is achieve all this manageability, utilization, centralized backups, et cetera. So arguably the first and foremost that comes to mind, the first building block that we use here, is virtualization. Virtualization is an incredibly key feature which allows us to do a number of things. Obviously, it's very, very apparent that the first thing it does is improve utilization so we can just throw more workloads at the same piece of iron. It also enhances manageability, it enhances separation of concerns, it does a lot of things for us. Virtualization is a really, really key ingredient for this. A second key ingredient is some form of storage that is centralized in some way. Because number one, it creates separation of concerns because it decouples the workload from its storage. And if we have some form of centralized storage that is usable from all of these virtualization nodes that we're using in an enterprise cloud, then we can move our workloads freely between those machines. It's very easy to do. And the centralized storage enables us to do other things like, for example, centralizing our snapshots, taking our backups off of a central location. That sort of thing. So central storage is really, really helpful. Another thing that is really crucial, but perhaps not quite as apparent, is we also need some form of storage replication. Because we need to be resilient against things like natural disasters, power outages, et cetera, et cetera. If you live in Queensland, you probably agree that not everything that comes out of a cloud is necessarily good. And that may affect your data center uptime rather gravely. So we need some form of actually shipping data off-site. And off-site can be a different fire area, can be a different building on the same campus, can be a different city, just depending on our application requirements. And finally, we need a glue to tie all of this together and to, at the same time, meet the SLAs and other requirements that we're typically bound to. And that glue is obviously high availability. We need some form of high availability infrastructure to manage our storage, to manage our virtual workloads, to monitor them. We need these high availability building blocks also to simply and easily and effectively move resources across the data center, which is something that tremendously increases manageability. So this is something that we find pretty much all across the board. Now, there is a conventional approach or a conventional way of deploying the enterprise cloud in a data center. So for example, for the central storage and replicated storage part, you could buy a large and extremely energy hungry overpriced refrigerator, which in addition comes with really, really expensive firmware. And you basically spend in the six figures to just get your central storage. You could also be using a virtualization infrastructure that comes with a completely proprietary license and happens to be relatively important in the market. You could also use high availability software that is equally commercially licensed and lock you into that. So as a whole, the conventional way of doing this creates a fair amount of vendor lock-in. And what we are always wondering about, obviously when we see these things, being open source people, is is there a better way to do this? Can we do this with open source components? Can we do this with just open source components? Can we completely alleviate or remove any reliance on proprietary or commercially licensed or whatever stuff? And the answer is yes, we can. And we're going to show you what this looks like in a minute, but at first we're going to explain to you conceptually what this is made out of. So let's start with a very fundamental thing, and that's storage. Well, there's really not much need to buy something anymore where you actually completely lock yourself in with a certain storage vendor. Last I checked, and this was a few months ago, Intel architecture 64-bit boxes were readily available with 24 slots for hard drives. And you could stick 48 terabytes in them. That's a fair amount of storage. So there's no reason to not use an open system here. When it comes to the storage replication part, we have something for that in the Linux kernel. It's called DRBD. It's a fully synchronously or asynchronously replicating storage device, a block device, which we can use to simply duplicate our entire storage set of a full data center, if need be, to a completely different location. And again, this can be a different fire area, a different building, or it can be a different city. So we can have fully synchronous block level storage replication between two sites. If we now go ahead and cleverly stash one of the available iSCSI storage targets on top of DRBD and have this managed with the pacemaker cluster stack, lo and behold, we have a highly available replicated iSCSI SAM. And by the way, there's no limitation whatsoever to IET. The pacemaker cluster stack at this point supports three iSCSI storage targets, which are IET, TGT, and LIO. So all of a sudden, there's our iSCSI-based SAM. And by the way, we've completely removed any reliance on protocols such as Fiber Channel, which have their own issues. So that's sort of the lower half of the stack. And the upper half is the virtualization layer. And here, we have any number of virtualization hosts, which would typically connect to our iSCSI SAM using an iSCSI initiator. And we have a fine iSCSI initiator in software in Linux called Open iSCSI. And this is what we can use to just connect to any iSCSI target under the sun. But while we're at it, why not use a Linux open source iSCSI target? And then on top of that, we have a hypervisor. We use KVM, but the stack as such basically ties in with any hypervisor back end that Libvert supports. So we can use Zen, we can use OpenVZ if you're inclined, we can use Linux containers, whatever. The setup that we're going to be demonstrating in a moment is just happens to be using KVM. But that's by no means a must. And of course, we can, like I said, multiply these as we wish. We can have as many virtualization nodes as we want. So we can achieve really great scale out there. We can distribute, again, distribute those across different fire areas and different buildings, et cetera, et cetera. And it will just work. And what we have then is a highly available storage stack plus a highly available virtualization stack, basically all of which are managed with the same infrastructure, with the same building blocks. We're using pacemaker for high availability, top and bottom. We're using dbd for replication. We're using Open iSCSI to talk to an open source Linux iSCSI target. It's really a 100% open source. Stack. And what we're going to show you now is a demonstration of how this whole thing works. What we have here is a total of four machines. Like I said, we can have as many virtualization hosts as we want. For demonstration purposes here, we have two. And we have two storage nodes. We've labeled the tabs here with storage one, storage two, vert one, vert two. And currently, we've got a storage running here on that node on hex 13, which is our first storage node. If you're familiar with IET, there's nothing too surprising here. And we have existing iSCSI connections here. And you would have guessed, I suppose, that those are from our virtualization host. So let's take a look at that real quick. So we've got, these are our virtualization boxes. And that's our virtual machine. By the way, all of these are, so the servers happen to be Celeste 11 SB1 boxes. And the Superfrobnicator is an open Susie box. Very, very cool. And if you haven't tried it, do. The way that we set these up, they're just Susie studio images. Very, very easy. Point and click, and toss it into the iSCSI target, and start it and go. It's very, very cool. It will actually, Kiwi will boot up, and it will resize partitions according to whatever space it has available, and all other cool stuff. It's really, really neat. So we have the Superfrobnicator thing running here. And to give you a brief demo of how resilient this thing is, we're going to open a console on that thing. And because this is not a marketing presentation, we can do something like relatively mundane, like just hex-dumping our dev VDA device here. So we're just reading from this thing here. While we're moving our storage around, and that's something that we're going to do next. And like I said, we have all of this managed with the pacemaker cluster manager. Pacemaker comes with a handful of very, very handy configuration and administration tools. CRM Mon, for example, is a utility where we can monitor the cluster stack. So the iSCSI target thing that's important is at the top, currently running on hex-13. And what we're simulating now is hex-13, which is one of our storage nodes, has to undergo some form of scheduled maintenance, such as we have to reboot in order to do some sort of upgrade or install a new kernel or whatever it is. So what we're doing here is on hex-13, we're just going to send this node into what's called standby mode in pacemaker, which basically causes all the resources to migrate away. And this is just going to be a few seconds. Whoops. And there's the failover, and that's completed. And we can look back on our virtualization node, and that thing continues to run completely unimpeded. There's not much data on that thing, by the way. It's not falsely reading zeros. There are actually zeros on there.  Otherwise, your voice doesn't go on the AV, and nobody knows on the other side what you're asking. So you've moved the storage from one DRBD machine to the other? Precisely. Yeah. So you're still using the same DRBD device, but from a different machine? Well, yeah. What it's done is it's moved the ice-cozy target over on the other machine, and also reversed the direction of DRBD replication, rather obviously. OK, so that's that. And what we can also do, and this is also something that we can, of course, manage from pacemaker, is we can go ahead and migrate the virtual machine itself rather than just the storage target. This happens on the virtualization cluster. So our superfrobnicator is currently running on Hex 11. And now we're going to do something different. We're not going to send full node into standby mode. We're just going to migrate that resource over to the other node. And we have a question up there. Just a second. I'll get right to it. And as we migrate this, this is actually live migration. So pacemaker here completely ties in with the live migration capabilities that we have in Lipvert and KVM. So there is no interruption to the machine as such. There we go. Completed. And here the console obviously has died. But we can go to the other box, and we can continue to see this thing reading here. So it's a completely interruption-free live migration of the whole thing. You had a question? Yeah, just to do with the timings, like when you did that, how would the timings be affected for something that's highly available and requires intensive timings? Yeah. So that's entirely configurable. So there is a timeout that you can define that an iSCSI connection must always survive. And the parameter, if you're interested, is a default time to retain, which is something that's negotiated between the iSCSI initiator and the target. And you simply have to set that such that it always is longer than you expected failover time. As you can see, the failover time here that we effectively got was about 10 to 15 seconds max. And if you use a default time to retain of 60 seconds, then that means if the iSCSI target is gone for anything less than 60 seconds, the initiator will not even flag an IO error. It will just block. And that's very simple and easy to configure. So that's how this is done. Just as an aside, for those of you who were familiar with things like heartbeat 2 and having to manage XML directly and possibly hating it because you never quite could get rid of the feeling that you're currently running around with a cocked and loaded shotgun pressed firmly against your feet, which is absolutely true. By the way, all of that is gone. We have a nice and very, very effective command line user interface for pacemaker now, which we can use for managing the entire cluster. We have, lo and behold, things like online help and tap completion, all that sort of thing. And we even got syntax highlighting for the config. Yay. So this, by the way, may look sort of daunting if you see it for the first time. But that's really all you need for having a highly available virtualization infrastructure here. That's all. That's one screen full. That's it. Nothing to add. OK. So what do we get from this? Well, we get better utilization. We get pretty good manageability. We have pretty damn good separation of concerns because we can literally have different teams manage one manage the storage cluster and one manages the virtualization cluster. Centralized backups are a breeze because we have this completely centralized storage thing, which is also, of course, a snapshot cable because that's something that just happens to come with the Linux IOS stack. We have thin provisioning. We can do that as well. And we have a tool in order to get things just off the assembly line and pre-configure and use resources that way. If you want a PDF of this, these are the two addresses that you can contact. And we thank you very much for your attention. And we'll be happy to take any more questions. Thanks. You said you're supporting IET and a couple of other SCI SCSI Terminage. You didn't mention SCST, which seems to be the most popular at the moment, for a particular reason, or just you haven't got a round to it? Well, yeah. The reason is that no one has yet contributed a patch for the resource agents to do this. I have my own thoughts about SCST, which we can get to in just a second, because it would sort of... Yes, next question. Yeah, back to the failover again. So if you got replicated databases, how does that affect the timings on those? For example, like a clustered database, if you were to failover a node like that? That depends entirely on your application. So as far as the UBD and SCSI is concerned, it will not make a non-crashsafe application any crashsafe, and it will not make a crashsafe application any less crashsafe. So basically, if your application does the right thing, through the use of DirectIO and synchronous replication, UBD will guarantee that whatever you write is written on both nodes. And when you failover that same content is there exactly as it was on the original node. We had a question in the back. Mike is printing. How tolerant is the iSCSI DRVD stack of active, active replication? Can you use the same volume on both sides with that? Well, again, if you do it right, you can. You obviously would have to use some sort of distributed locking. So applications don't try on each other's toes. It's, for example, perfectly fine to export a DRVD-backed iSCSI volume to multiple nodes, and then run OCFS2 or GFS2 on them. That's perfectly fine. And yeah, you can actually, but this has nothing to do with virtualization, really. You can also run OCFS2 or GFS2 directly on dual primary DRVD. But the use cases for that are somewhat limited. The thing with exporting iSCSI and then sharing that across multiple nodes and then having a cluster file system on top of that, it's more frequently found. Yes? When I was first looking into this, somewhere I read that the heartbeat OCF resources were being deprecated? No. They're not? No, absolutely not. No, we had the ones that are deprecated are the heartbeat one compatible ones. So heartbeat two and pacemaker continue to support the resource agents that ran with the old heartbeat one stack for compatibility reasons. But the OCF, no, the OCF ones are definitely not being deprecated. They're not going anywhere. OCF, by the way, is the open cluster framework and the open cluster framework resource agent specification is the API that the pacemaker resource agents implement. You were talking about snapshot capability. Is that done through the file system? Well, what people are mostly doing is they would export, they would map a DRVD as a physical volume to an LVM volume group and then export LVM logical volumes as LVM logical units and then use LVM snapshot capability, which we know is limited but good enough for most. But arguably, you could also have a file backed ISCSI store where you're storing your ISCSI logical unit images in a file system. And then if that file system is snapshot capable, then you can snapshot that way. But what's most frequently found as of today is just people using LVM snapshots. Do we have more? Yes, one more here. Just a sec. Mike, right here. Non-technical question. Go ahead. Who's using it? If we want to implement this and we've got a customer that maybe is business-oriented, they're going to want to know who else out there is deployed to sort of infrastructure on how successful this is. Yeah, so there are some really small accounts like ModaPhone or HP or there's quite a few people out there using it. So our current estimate of actual installations out there. Kind of applications. What kind of applications did you run on top of this? What, on the virtualization stack? Yes. Well, we've had people basically running anything that they ran on Linux boxes, on Windows boxes, on what not, in the virtualization stack. So there's really not much limitations as to what you can run. There may be some where you have certain requirements that you can't fulfill in this stack, but there's typically not specifically performance-wise. But there's typically not any major showstoppers. They're completely keeping you from doing this. So for example, one example would be if you have to virtualize some age-old Windows version for which there are no pair of drivers available for KVM, then you might suffer a pretty bad performance hit, and then that might not be an excellent option for virtualization. But generally, there's no major showstoppers. Basically, for things that don't scale out, things that are very heavy in utilization, you probably wouldn't use this kind of system. Some, for example, a big mail server. Yeah, you might not. It clearly depends on what kind of hardware you have available and what kind of workload you have to manage. There's no one answer for that. You had another? First, I had an answer for this. I was asked this exact question by some colleagues just the other day. There's a really obvious answer if you think about this for five minutes. The database server, for example, because I know DBAs love that question, the database server that you've got that today is filling an entire hardware platform, one of your maxed out blades, for example, will fit into a little tiny corner of the blades that you buy next year. And do you want to rebuild it on the new blades next year? Or do you want to migrate it to a little tiny corner of the blades that you buy next year, live, without a reveal? But no, the question I had was, has DRBD improved at tolerance for cross-site, longer, flakier links? Yeah, via an add-on. It's called DRBD proxy. It's an add-on that's primarily designed for being extremely efficient in terms of asynchronous replication. Talk to me after the talk. I have plenty of info about that. Just want to re-talk back to the gentleman there who was talking about databases. Databases that max out your hardware now that will fit into a tiny corner of your blade next year, depending upon how they're set up, probably too busy to be virtualized because of the timing issues that exist. Like I said, there is no set in stone answer. VEV is not here. We run Oracle, and we can't virtualize it because the timing issues. Do you have online any recipe or how-to for building one of these? Well, right now I don't. But that's the number one item on my to-do list. But while you're at it, we do have one for building the storage half of this. So if you go to Linbit's website and look under Education, there's a section called Tech Guides. And there's one for setting up a highly available ice guisey storage cluster. There's also one for NFS. There's one for OpenVZ, if you're interested in that. And KVM1 is in the works. And like I said, that's number one on my list. There are no further questions. OK. Well, it's time come to thank these guys for the great, great job they've done. The hard part is here that being Solomon, I can't really break this gift into two because there's only supposed to be one of them up here. And I think every time the talker gets away with it. So this time, I'm going to give the talker the analog kit. Thank you. And I'm going to give the artist the macadamia nut bowl because an artist would appreciate this work of art, so put your hands together for them.