One of the three signs that you're gonna be in rich one day. We're gonna talk about those things in this video So stay tuned. This is the David Willshow. Let's go All right, welcome back. So one of the three signs that you will be rich one day. I'm here with Will Fan. He's a multi-millionaire Entrepreneur here in Vancouver, BC. He's been featured in the Horace magazine for the top 30 Multivariate Billionaires under the age of 30. He just recently turned 30. Congratulations. Awesome. Okay, and So let's talk about like it Okay, so what are the three signs I be rich one day number one? First one never give up. I remember When we all started you know, but we started with humble beginning humble background and the only thing that we had was really ourselves, right and Having that energy to work every day every morning every night. I think a lot of you would give up Generally, they would give up if it's something that they're not used to you know, they know I just have friends like, you know come out of school or Or they got embedded into the workforce most of the life and the problem that they were having was they can't go side comfort zone. So, you know, they provide, you know, they they actually They need the support of a job. They, you know, or People were studying now in school. They they follow a system is to hope that they can build a career or future there But then when they all come out, they almost literally give up, you know, it's not easy We're gonna have to work hard for it. We're building something for the future and that's not bad And most people they don't they can't perceive that it's not you know in the realm of Reality so what we have to do is, you know, take it step by step and explain why and how But I see most time when people give up is usually That one disbelief that they have they take it as like it's the whole thing, right?  So you need to get new skills You need to get new opportunities need to do new things that you haven't done before Okay, and the thing is that when people do that they give up because they don't not getting the result that they want They're not getting the success they want, right? Let's say they want Get a new skill like networking, right? We talked about networking in our last video how to overcome the fear of never came That's a skill. You need to learn it. You need to practice it So in order to get good at it you have to take it out of your comfort zone so that you can get successful So the problem that people do is that they They try it and then they fail and then they say it's not working and they get out. Yeah You look at the bees in the fly the bees in the fly so So the flies actually They're stuck in one place. You put them in a jar and If you put it up upright, right? There's the exit. It can actually leave They actually remain the jar for whatever reason Even if you turn it around They still don't know how to come out of it I didn't know that. Yeah, it's interesting. So flies actually they stay in one area So you ever walk in on the road or the farm you always see flies Flying that same area, but they keep flying anywhere they want, right? However, the bee inside Actually, you'll find this way out. It always will Right. So the little flies is low. No tiny flies. They actually they only know how to stay in one area, right? So you think about most people are like With flies, right? They either noisy distracting their focus at all like in their mind So we see that quite often and I think the bee is gonna go hunt. He's gonna go fly but this hive goes looking for, you know, honey and food, right? So he needs to get out and say I need to go produce. I have work to do, right? So that's different between the bee and the flyer.  Okay, so number one never give up That's the number one sign that you're gonna be rich one day. So number two number two sign of that you're gonna become very rich one day is You are very resourceful. What does it mean to be resourceful? It means that you can solve problems creatively I'm always gonna counter problems, but you can't solve the problems creatively and differently than what a lot of people solve them Then you're gonna have a tough time to be successful making any kind of money or making any progress in your business Yeah, so being resourceful is so important.  This means You know, don't go excessive on things that you might buy and you're not going to use it Use whatever you have now and maximize that you gotta be like you gotta be like Tony Stark Yeah, he was in a cave There's a bunch of scrap material and he built his mark one Iron max to get all the cave. Yeah, you build armor out of nothing If you even want to stay in the cave Right and if you want to get out of the cave, you need to be resourceful while you have that's right And then create something just like Tony Stark did exactly if you You don't you're gonna get stuck in the cave. No, we're gonna die.  Yeah, so so If you're gonna start in your business and you want to figure out a way to get to the next level, right? You know make a list of things that you have, right? What connection do you have? Who do you know? Um, write it down access to money Access to other people's money access to credit Access to products what products that products do you have what skills do you have? What did you study in school? What are your expertise? You don't have to be the world's best But let's say out of a hundred people are you in the top 10 people that know about certain topic, right? It could be anything right. So what's your expertise? Why are your strengths? So make a list of all that And then those are your materials. Those are your raw materials like Like 20 start in the in the cave. Those are in raw materials. So you have all raw materials Then you can create a plan make a blueprint and say how I'm gonna build my business So that I can get out this cave get out to the world to the next level the next level of lifestyle exactly Okay, so the third final and I we think the most important Sign that you're gonna be successful one day is Dream bake So what this really means is I I seem to play often And a lot of people in the society they don't Believe in being bigger. They think anything that's bigger.  So So I think the most important thing that was lacking this society now You know through bigger expanding your mindset expanding your financial wealth mindset like Like some people actually Have an experience, you know making more than $10,000 a day, right? So you haven't experienced even making $100 a day, right? So So it really comes down to a different level of thinking and back then I remember when You know $100 a day that was pretty good.  That's not, you know, you have to keep going for more, right? I remember one of the first books I ever read was the magic of thinking big If you have read that I suggest you get that book magic of thinking big And um, that really changed my perspective of how we think because it's bigger than we think Than the more that we can achieve. So for example today I refer now I make in one day what I used to make all year You probably you too. Exactly. Yeah. Yeah back then when I was younger It's like well 10 grand a year was about some really good But now you can be 10 grand a day. That's yeah way better. And now we do it easily. All right, and we never thought Back then that we could achieve what we have now. Exactly. Yeah, unless we thought and we unless we dream really big Right and now we have bigger goals, right? And so you have multiple Universes like multiple people with different ideas and thoughts Right and just imagine you're you're the only guy thinking this big while everyone's like thinking this is my new thing. So I always quite see often was there's always more people thinking very small and there are two guys who think There's only one guy that always think big. Everybody around me was very very small, but But you can see that there's so many people that actually think so small that It actually imprisoned the guy who's taking big, but he's one he's one of you more he was a built you know Multi-million or bill dollar enterprise, right? And you know, there's very few people that can think like that is very few Just imagine if nobody tried to make you wouldn't have anything today. There's no infrastructure. No technology No technology Like for example if Elon Musk never never dreamt about making an electric car You know so successful that we wouldn't have Exactly or Tesla himself, right? We wouldn't have the Like Nikola Tesla, right? He didn't dream big. We wouldn't have Basically 90% of the right technology have today everything from radio to these semi-conductors and like a soft You know mills gay. We didn't have them. We have no computer Your technology wouldn't exist, right? Right.   So dreaming big is like the opposite of fear. There's more fear you know Going back to small dreamers is because that's huge fear And when there's more fear than courage You only think to be this small, right? So So you have to get rid of that Most important thing is, you know, increase your confidence and actually just keep thinking about keep dreaming about it every day It's like going to your repetition.  So I hope that helps you out and if you like this video make sure you subscribe to this channel and click on the Notification bell to get all the latest videos coming out from the David and Will show Yeah, make sure you ask questions if you have any questions about your own business about your own, um You know, personal development and ask them in our comments and then we're gonna create a video just for you to answer your questions Yeah, right. So at least like our all our videos as well. Yeah, like all our videos. Share them comment comment And uh, yeah until next time David will show peace


I remind colleagues of the Covid-related measures in that face coverings should be worn while moving around the chamber and the wider Holyrood campus. The next item of business is general questions, as ever, that I'd appreciate succinct questions and answers to match. I start with question number one, Pam Duncan-Glancy. Thank you, Deputy Presiding Officer. To ask the Scottish Government whether it will provide an update on its plans for the United Nations Convention on the Rights of the Child Incorporation Scotland Bill. Minister Clare Haughey. We remain committed to the incorporation of the UNCRC into Scots law to the maximum extent possible as soon as practicable. While the Supreme Court judgment means that the bill cannot receive royal assent in its current form, we are urgently and carefully considering the most effective way forward for this legislation to ensure that incorporation can happen as quickly as possible with confidence that any amendment to the bill do not attract further challenge. Our preference is to address the Supreme Court's judgment by returning the bill to parliament via the consideration stage. In parallel with planning for that, we are also exploring options for extending our powers to incorporate the UNCRC beyond those available under the current devolution settlement. The Deputy First Minister has shared a copy of his exchange with the Secretary of State about that in an update to the Equalities, Human Rights and Civil Justice Committee. Next Wednesday will mark a year since the UNCRC bill was passed unanimously by the Scottish Parliament. I am concerned about the letter that the minister refers to, because it suggests that, in the reconsideration process, the Government is seeking further powers in order to make the bill competent, rather than working with the UK Government to make the bill competent within the powers that it currently holds. Political points going over the constitution by the Scottish Government has already delayed the bill, and it seems that it will continue to do so. In the meantime, young people have no idea what timescale the Government is working to, or how long the process will take. They need reassurance that the Government remains committed to their rights and to bringing the bill within the Parliament's competence as soon as possible, and they deserve to know when the UNCRC incorporation will be a reality. When will the Government bring back the bill? I thank Pam Duncan-Glancy for her further question. The Deputy First Minister has committed to keeping Parliament and the committee updated in the progress of progressing the bill. However, I think that it is really important to recognise that the majority of the work in relation to the implementation of the UNCRC is continuing at pace. Ruth Syllomantry, Stephanie Callaghan, will join us online. Make it right is the aim of a campaign led by the young people of North Lanarkshire that is encouraging other local children to better understand their rights. The young people have even created and started their own social media video. Can I ask what steps the Scottish Government is taking to involve young people themselves in raising awareness of children's rights ahead of incorporating the United Nations Convention on the Rights of the Child? The Scottish Government officials are due to meet North Lanarkshire Council later this month to learn more about the excellent project and how we can share good practice. The Scottish Government commissioned Young Scot and Children in Scotland to work with children and young people to develop resources to raise awareness of children's rights across all sectors. The materials were published in September 2020 to coincide with the introduction of the UNCRC-built Parliament. There is also good work under way in schools, uniseffs at rights respecting school awards, provides a framework to embed the UNCRC strategically and practically into schools, ensuring awareness of children's rights among children and young people. The children's Parliament has recently launched a complementary resource called Dignity in Schools, which aims to demonstrate ways in which primary schools can adopt a rights-based approach and help to make rights real for children. People get fed up when our two Governments can't just sort things out especially when it comes to an issue of rights for children. We heard last week about the number of children locked up in prison when they shouldn't be, so this matter is real. Rather than hunting for a never-ending battle with the Conservatives, when will the Government sort this out? We need a date. Presiding Officer, as I gave my answer to Pam Duncan-Glancy, the Scottish Government is working at pace at this. The Deputy First Minister will write to the relevant committee on that. 2. Richard Leonard on co-operative development in Scotland. Minister Ivan McKee I surprise agencies to support the growth of co-operative and other alternative business models, which we know can deliver strong outcomes on fair work and benefits to local communities. The Scottish Government is determined to significantly increase the number of co-operative, social enterprises and employee-owned businesses in Scotland while supporting regional regeneration and the wealth of local communities. A recently published 10-year national strategy for economic transformation sets out our commitment to undertake and publish a review of how best to do that. I thank the Minister for that answer. The Scottish Government has stated that its goal is the creation of 500 employee-owned businesses by 2030. In 2018, it set up a Scotland for Employee Ownership group to achieve that. Cooperatives and employee ownership were mentioned again in last week's economic strategy. The problem is that the co-operative development team in Scottish Enterprise was dismantled. Scottish Enterprise acts the budget for co-operative awareness raising. Grant support to cover 30 per cent of the cost of the transaction to convert to a worker-owned business has been withdrawn. The Scotland for Employee Ownership group has become little more than a ministerial photo opportunity. As a result, Scotland is not going up, it is going down the UK league table of worker ownership. When will the Scottish Government finally address that? Stop paying lip service, put in place a credible industrial strategy, back it with the resources that it needs and ensure that it really is serious about co-operative development? I thank the member for the question. Just to clarify, co-operative development Scotland is the arm of Scottish Enterprise that works in partnerships with Highlands and Islands Enterprise and South East Scotland to support company growth through collaborative, co-operative and employee ownership. In terms of the number of businesses in that regard, Scotland continues to be ahead of the rest of the UK with regard to that. Obviously, we are focused right across the whole alternative business model space, including social enterprises, and the member will be aware that social enterprises have doubled their funding to almost £2 million over three years to support the activity that is happening in that regard. The resources are in place, and co-operative development Scotland continues to work as part of Scottish Enterprise and the Scottish Government remains committed, as we have highlighted in the national economic strategy to developing co-operative social enterprises and other alternative business models. In Scotland, because it recognises the value that it brings to communities and to Scotland's employees and to Scotland's economies of work. Question 3, Willie Coffey. To ask the Scottish Government how its investment has built the wider Ayrshire economy. The transformational £103 million Scottish Government investment in the Ayrshire growth deal supports projects that identify the greatest potential for long-term inclusive growth. Regional partnerships have estimated that across Ayrshire the deal will create 7,000 new jobs and unlock an additional £300 million from the private sector. Projects included will transform the regional economy through high-value job creation, strong regional supply chains, tackling weak productivity and delivering skills across Ayrshire. Additionally, Ayrshire has benefited from investment from a range of regeneration programmes supporting development and delivery of local solutions to tackling poverty and disadvantage in communities across Ayrshire. Willie Coffey. Thank you Minister for that answer. Could you confirm that the investments made by the SNP Government are making a positive impact to the Ayrshire economy, in particular at Priswick airport, where the financial intervention has saved many jobs directly, saved even more jobs in the wider economy and will support the potential for thousands more jobs across Ayrshire in association with the Ayrshire growth deal? Without our intervention, in 2013, Priswick airport would have closed and the result of hundreds of jobs would have been lost and were otherwise saved by our action at that time. We were clear that closure would have a significant impact on the local economy, not just because of jobs lost in the airport but in the various other businesses that rely on the airport being operational. Our investment through the growth deal projects, including in and around Priswick airport, will be £30 million over the course of the deal. Four space and aerospace projects will benefit from this investment, and those projects will deliver significant economic benefits, playing a key role in signalling Priswick as a major inward investment destination to the international space market. Regional partners have estimated that across the deal over 7,000 new jobs will be created as a result of the deal. Within Priswick, that number is expected to be over 2,700 direct, indirect and construction jobs. In a letter to the Scottish Government, the Ayrshire Economic Joint Committee notes that projects in the Ayrshire growth deal have seen an increase in costs since the projects went to Tainter. They write that contingency funds may be required by councils for some projects, but budget pressures may make this impossible. What funding will the Scottish Government provide? Will it guarantee all projects that they are involved in will be fully funded and delivered on time? The member will be aware that the growth deal money is obviously—some of that money has obviously been agreed between the Governments and working with local partners to direct how those funds are spent. We are clearly well aware, as the whole economy is, of on-going cost pressures within the construction sector and other sectors. The Scottish Government procurement team is working hard to do what it can to mitigate that and to provide advice and support where possible to partners across the public sector that are facing those on-going cost challenges. I would like to ask the Scottish Government whether it will provide an update on any safety issues on the trunk road network in my constituency of Perthshire, South and Cynroshire. The trunk road network in Scotland is subject to an annual road safety review based on statistics recorded by Police Scotland and measures are then prioritised where they are expected to contribute to the Scottish Government's 2030 casualty reduction targets. That process has identified road safety investigations on the A85A9 and the M90, which are currently under way. Those considered trends have recorded injury accidents, as well as focusing on vulnerable groups such as motorcyclists. Any recommendations for improvement works such as signing, resurfacing and speed management emerging from those will be considered and prioritised for construction as appropriate. The minister may be aware that there is a growing community concern about the planned new junction on the A9 at Sianafut east of Ocdrardder. Residents have got concerns about road safety and speeding issues, the locations of the proposed junction and the fact that it appears to have been scaled back from a two-way to a one-way system. Will the minister agree to meet me on site to discuss the concerns with local representatives? I am aware that the Sianafut junction proposals are intended to support developments identified in the Perth and Cymru local development plan and that those addressed junction issues that would otherwise exist on the A9. The design of any new junction on the A9 would, of course, be subject to a safety audit and other design checks. I recognise that, as Mr Fairlie has outlined, concerns have been raised by the members of the community that he represents. As I understand it, those mainly relate to local roads access to existing communities and new developments. I would, of course, be more than happy to meet with Mr Fairlie on-site and with members of the community that he represents, because it is absolutely essential that we get road safety improvements right for the communities that they serve. 5. Brian Whittle To ask the Scottish Government what action it has taken to increase the number of jobs in offshore wind and the wider green economy. I thank the member for the question. The Scottish Government is determined to maximise the economic opportunity for the Scottish supply chain from our offshore wind potential. We will drive forward offshore wind skills development, working with stakeholders to focus on the opportunities for diversification and skills transfer from our oil and gas sector, in line with our commitment to adjust transition. The introduction of Scotland's supply chain development statements demonstrates how serious the Scottish Government is about holding developers to account if they do not honour their supply chain commitments and create green jobs. To make the most of opportunities in development of the green economy, we must ensure that our education system produces skills, a skilled workforce, appropriate for future jobs. We missed that boat with the development of offshore wind. We now have opportunities with offshore wind, hydrogen and electric power and all the associated servicing skills. I can ask what the Scottish Government is doing to embed the green economy into the education system. I thank the member for the question. It is not easy to overstate, it really cannot be overstated, what an enormous opportunity Scotland is. 25 gigawatt is a number that does not mean much to many people, but it is huge. The Scotland leasing round will provide a strong pipeline of projects through this decade and beyond, and to prepare our workforce for those skills, we have the future skills development plan and the climate emergency skills action plan. It is absolutely vital that we equip the people of Scotland, young people coming up the way, people transitioning from high carbon industries and people returning to work with the skills that they need. Our national strategy for economic transformation commits to lifelong learning for all people of Scotland and the appropriate skills development that we need for the green industrial revelation. Given the importance of transitioning from oil and gas to green energy such as offshore wind, this is a matter of great importance to businesses and residents in Aberdeen South and North Concord and my constituency and the wider north-east. Can I ask the minister how the Scottish Government is engaging with north-east employers in the energy sector, particularly with this week being Scottish apprenticeship week, to ensure that support is in place to train apprentices and re-skill existing employees? I thank the member for the question. Of course, the north-east has enormous potential not only in offshore wind but in green hydrogen. Looking at the Scottish port and the opportunities in Aberdeen in this respect in particular, the Scottish Government is completely committed to supporting the north-east through the energy transition and that includes with skills development. The Scottish Government previously promised to deliver 130,000 green jobs by 2020, but just a sixth of that number have been delivered. Latest figures reveal that the number of green jobs is now falling with a loss of nearly 3,000 since 2014. Will the Scottish Government support Labour's call for the £700 million from the Scotland licence and all annual income to be ring-fenced for investment in the creation of green jobs? Applicants to the Scotland leasing round were required to submit a supply chain development statement setting out the anticipated level and location of supply chain impact, including jobs. Developers can update their statements throughout the development phase. The introduction of the statement demonstrates how serious the Scottish Government is about holding developers to account if they do not support their supply chain commitments. We fully expect developers and OEMs to be engaging with the domestic supply chain to create green jobs and to fulfil their commitments. To ask the Scottish Government how the country's transport system can help to improve economic growth. We have a vision for a sustainable, inclusive, safe and accessible transport system helping to deliver a healthier, fairer and more prosperous Scotland for communities, businesses and visitors. All of our investment in transport aligns with that vision as we have set out in the second strategic transport projects review. By focusing investment on sustainable transport options and continuing to invest in green innovation, we are making Scotland more accessible for residents, visitors and businesses and supporting Scotland's workforce. Last week's so-called national strategy for economic transformation said of the country's transport network that there remains opportunities to improve connections within and between certain areas and it said of the trunk road network that it's a crucial facilitator for national and local delivery of goods. Given that, is the minister now prepared to break free of the shackles of the extremist greens and commit to properly funding improvements on the A77, A75, A83, A9 and A96? Minister Dinegal Roof. Thank you, Mr Simpson, for his supplementary question. I remind him that, since 2007, this Government has invested approximately £9.5 billion in managing, maintaining and improving Scotland's trunk road and motorway network. The £3 billion investment to dual, of course, the A9 between Perth and Inverness is one of the biggest transport infrastructure projects in Scotland's history. To his substantive point regarding the national strategy for economic transformation, it sets out the priorities for Scotland's economy as well as the actions that are needed to maximise the opportunities of the next decade to achieve our vision of a wellbeing economy. Scotland's transport network has been identified as a key driver to help to achieve the ambitions and vision outlined in the refreshed strategy. He will also be aware that the recently announced strategic transport project review 2 is highlighted as a means to improve connectivity and infrastructure that he touched upon. The SDPR2 is currently out for public consultation until April. I will be very welcome to meet Mr Simpson next week and perhaps we can discuss some of those matters in more detail. A very brief supplementary view on it. The minister will be aware of the reopening of the Bathgate air-tree line, which has brought great economic and social benefits to my constituency and right across central Scotland. Can the minister identify how many transport projects the Scottish Government has invested in? I am sure that Graham Simpson will want to welcome them all and the economic contribution that they have made. Minister, I bear in mind that you can write with further detail. Ms Hislop is absolutely right to highlight the huge investment that the SMP Government has made in transport infrastructure for rail alone. We have invested £1 billion, including £300 million on the AirJet to Bathgate rail link improvement, which brought three new stations and a 31 per cent increase in the number of passengers at existing stations, investments in electrification of all rail routes between Edgbrain Glasgow and to Stirling, Alwa and Dumblane. Thank you very much indeed minister. That concludes general questions and we will be a slight pause before we move to the next item of business.


Hello, DevConf. Welcome to the last day of the conference. I know it's late in the afternoon, well, early, but thank you for joining. I know some of you have flights to catch. So I appreciate you being over here, hungover or otherwise. Let's get started. Today we'll be doing a presentation on raw block persistent volumes in Kubernetes and how they help change the way the Rook project deploys in Kubernetes. A fair warning, these presentation may include opinion speculation and bad jokes on our part. They are strictly our fault, and Red Hat will not be held liable for anything we say here today. All right, introductions. My name is Jose. I've been at Red Hat for about 7 and 1 half years now. I've been working in storage the entire time. Currently, I work in OpenShift container storage with a focus on Rook and stuff. And I'm the project lead for the OCS operator, which will not be discussed in this presentation since that does not pertain to Rook directly. And I participate in SIG storage in upstream Kubernetes. And I like hitting things, mostly drums, though. Here we have Rohan Gupta. He graduated from college in 2018, did a Google Summer of Code with the CNCF to add the NFS operator into Rook. And we like that so much, we decided to keep him at Red Hat. So he's currently working on Rook inside of OCS as well. He likes watching anime and riding motorbikes. All right, here's what we're going to cover today. We're going to start by setting the stage and giving you an overview of storage in Kubernetes, what Roblox PVs are, and a quick rundown of Rook and Seth. We're going to go into the details of what Rook and Seth used to do in Kubernetes before Roblox PVs came in and afterwards, some hurdles we encountered in the implementation of it. And with luck and network permitting, we're going to host a live demo. Those are always fun. All right, to set the stage, storage in Kubernetes. When working in Kubernetes, you deal with resources, resources defined against an API. In this case, the core of what we work on are called persistent volumes, which are a representation of some sort of storage volume in a storage system somewhere. The actual API doesn't entirely care about the details of it. It simply presents enough information about it so that it can be consumed by applications running in Kubernetes. And of course, different storage backends define what a volume even means. Then you have persistent volume claims. These are the things that are actually created to be consumed by the application. So an application creates a persistent volume claim that then gets bound to a persistent volume in the back end. And these two bound together represent the connection from the storage to the application. And finally, we have storage classes, which are an object that allow you to request, to dynamically request, a volume of storage from some back end. So the way that usually works is that you create a PVC and then you create a PVC against a particular storage class, which will then, if appropriate, bind a PVC to your PVC for you without you having to specify a specific volume. This is generally the ideal way for cloud native applications to specify storage, since you don't want to necessarily tie your data to any particular locality or identity.  And here's the flow I just described. The developer, in this case, submits a persistent volume claim against a storage class. And the operations or administrator is what sets up the storage class, which then instructs the storage back end to create the persistent volume, which is then what's actually mounted by the application pods. This entire process I mentioned is called dynamic provisioning. The alternative is what's now known as static provisioning in which the administrator doesn't create a storage class, but instead manually creates individual persistent volumes and then those get bound to PVCs. So let me just say the whole dynamic provisioning thing was great. This came around in Kubernetes before I actually joined the project. And I cannot imagine a world without it. I'm sure Yann over here can remember those horrible dark ages. And I'm so glad I was not around for that. All right, Roblog PVs. All right, I keep talking about PVs. And notice that I said mounted by the application. All right, so the thing about PVs up to this point is that they were always presented to a pod as a file system. So regardless of what your back end was, whatever device you were given had a file system on it, and that's what was mounted into the pod. Roblog PVs are a new mode of PVs that allow Kubernetes to present storage to containers basically without a file system. So you get access to a block device rather than a file system. And there's a lot of applications, such as databases like MongoDB or Cassandra, that can leverage Roblog directly without the need of a file system just natively. And in particular, when you remove the layer of the file system, it allows certain storage providers to provide more consistent IO performance and lower latency, since you don't have to deal with the abstraction of the file system in between. And I've provided a link there to the concept documentation for Roblog PVs if you're interested. So to make this work, Kubernetes introduced a new field into the persistent volume spec called volume mode. It's been in beta since Kubernetes 1.13. Jan, this went GA, right? No, not yet. Still beta? All right. We're up to 118 now, right? Yep, still beta. But we'll get there. All right. So this field specifies whether it'll be in mode block or mode file. So both the PV and the PVC have this field, and they must match, right? So when you're doing dynamic provisioning, you create a PVC with volume mode block, and then that must go to a storage class that supports volume mode block, in which case it will provision a PV for you that is also volume mode block. And then, of course, the default is volume mode file, which was presented to preserve backwards compatibility. Here we see the YAML for the relevant specs. In this case, you have your persistent volume with volume mode file, which you can freely omit since it's just the default. And there's the field again. And then I'm going to stay here because the recording microphone is here. Over there, you see a pod spec where you define the volume at the bottom from that PVC, and then you mount it to a file system mount path. That's the way PVs and PVCs have worked before. With volume mode block, you change this field and that field to block. You still specify the PVC down there under volumes, but now there's a new field, volume devices, where you specify the name of the PVC, and then instead a device path where the device will be present. And that will be just a file, not a file system. Any questions on this so far? All right. Oh, got one. Doesn't matter. So the point here of the device path is that this guarantees a consistent name for the same device, regardless of how it shows up on the host. So I forget the specifics of implementation, but the device does get attached to the node, and then via sim links will get presented by a given name inside the container. Now, a quick note here. To those of you familiar at all with Kubernetes, volume mode and access mode are different things. There was a lot of confusion about this when volume mode was introduced. Access modes are things like RWX or RWO, and they define how many pods may attach a PVC at a given time and whether or not they can write to it. So for instance, read, write once means that only one pod can attach to a given PVC for read and write operations. RWX stands for read, write many, where multiple pods can attach to the same PVC and have read, write operations. Oh, what's the other one? ROX? No, what is it? What's the one where one reader and multiple writers? Is that ROX? Read only many. There we go, yes. So where multiple pods can read the same PVC. Real quick, any further questions on Kubernetes? We're about to jump into the application layer. Rook and RookSeth. All right. What is Rook? Rook is storage operators for Kubernetes. They help automate the deployment, bootstrapping, configuration, and upgrading of your storage, systems. And in particular, this follows the operator pattern for storage solution. If you've heard about operators during this weekend at all or beforehand, an operator is basically an application that implements operational knowledge, typically handled by a human administrator, and automates it in an application running natively in Kubernetes. And this follows the general Kubernetes model of desired state versus observed state. So your administrator defines a desired state of what they'd like their storage system to look like. And then the operator takes care of executing operations to make the actual state of the system match the desired state. This is what we call reconciliation of the actual state. And it's just a constant running while loop that will check the state. Does it match? Yes. Check the next part of the state. Does it match? No. Make a change. Restart the loop. Wait for it to change. Eventual consistency, I believe, is the term for that. Now Rook, as a project, hosts multiple operators. The one we care about today is RookSeph. RookSeph brings Seph into containers and in a cloud native fashion. Seph, for those of you that don't know, just like the five-second pitch, it provides resilient distributed storage in a self-healing manner. It is highly scalable and can run on commodity software. That's not to say it can't take advantage of more advanced storage systems, but it can technically run on a wider array of hardware. And it's fully open source. So the general outline of RookSeph is, or the main entity in a RookSeph cluster is the Seph cluster resource. This defines the desired configuration of a Seph cluster, which will then inform the RookSeph operator to configure a system that matches this spec. And there are a number of demons that are part of the Seph system that are running containerized in individual pods. I'm not going to go over all of them right now. All of the ones you really need to care about are up there in the upper right called OSDs. And OSD stands for Object Storage Demon. I believe it does. I use the acronym so much I don't entirely remember what it means. And those are the demons that represent the storage devices that Seph uses to store its data that it will then serve through Seph volumes. OSDs, then and now. So typically in Rook, you had local storage OSDs, meaning that Rook would just directly access the devices on the host. So you would have to rely on hosts that had storage already attached to them. And then you could either manually define which specific devices you wanted to use or have them automatically discovered by Rook and then consumed and then consumed by OSDs. In this case, your Seph cluster would have to define. You could define your storage nodes via names, labels, or just say use all nodes in the Kubernetes cluster. I mentioned you can define local devices either manually or through auto discovery. And then Rook takes care of formatting the devices appropriately, spinning up OSD pods and consuming the devices for use by Seph data. So the thing about this is that Rook was designed with this as its primary paradigm when it was first being brought up. So they made it really easy to configure. If you just want to use whatever you can find in the cluster, the Seph cluster definition is really short. And in particular, it's familiar to people who build traditional storage systems outside of Kubernetes. If you're not dealing with all this cloud native watching or who sits, you would typically have to statically define all the devices that you're using and import them into your storage cluster. And in particular, in doing so, this supports any type of device or appliance that is supported by Linux because all we need is a device to be present in the Linux file system. The downsides to this, especially when considering a cloud native environment, is that this relies on specialized nodes and there is a rigid coupling between your compute and your storage, which some people may not want, especially if they want to have a more robust storage solution. Income storage class device sets. Yes, that's a mouthful. Yes, I did design it. No, I don't entirely like the name, but community consensus is what it is. In this case, you still have to define your storage nodes via names, labels, or all, but instead of defining individual devices or selecting all devices, you define your desired amount of storage. So this is a different way to consider your storage system now because you no longer specify I want these x devices across these y nodes. You just say, I want x amount of storage across all nodes I have available, for instance. And in much the same way, Rook will take care of providing the actual automation to discover or acquire the devices, prepare them, and start an OSD pod for each device. This is a typical configuration of a storage class device set. They were designed to be a generic struct within Rook. So if you go look at the spec, you'll find some fields that don't entirely make sense for RookSef and are not used by RookSef. Currently, RookSef is the only operator that's making use of storage class device sets, but it's open for other operators to make use of if they so wish. Going down some of the individual fields, you need to specify a name so that you can have unique and consistent names for your PVCs that will get created. And then you have a count that specifies how many devices you want in this set. You need to mark whether your PVCs are allowed to move between nodes or not. You would think that this would not be necessary, but in order to make storage class device sets more flexible, allowing them to not be portable makes them a lot more useful in more environments rather than just the narrow scope of dynamically provisioning cloud environments. And then the volume claim templates is just a list of PVC specs. I say it's a list, but currently only one is supported in Rook. We made it a list to leave room for more advanced features down the line, but for now, you only need to spec. You can only specify one volume spec, and then this spec is used as the template for your count of devices that you want to use. As you can see here, we're using volume mode block. This is required, of course, because Seth wants to use a block device. We specify the amount of storage, the storage class name, and the access mode that I mentioned earlier. In this case, we want read write once because we only ever want one pod to use any given PVC at a time. The pros of this, naturally, since I designed this, I'm fairly biased towards this. You can offload device distribution to the Kubernetes scheduler, so you no longer have to worry about necessarily making sure that your storage devices are properly distributed in a balanced fashion or not. If you have the correct parameter specified, the Kubernetes scheduler will automatically distribute your OSD pods in a good enough distribution for redundancy. And in particular, this gives you the ability to migrate devices between nodes so that if your compute node goes down and you had any OSD pods on it, well, the OSD pod can now just migrate to any other node that's available and continue running, ideally, without any downtime to your data IO. And it works with any raw block PV, regardless of back end driver. And of course, it's shiny and new. The downside to this is that it requires predefined storage classes. So you must have appropriately configured storage classes ahead of time before configuring your storage class device sets. Device support is now obviously limited by whatever Kubernetes supports, so it's not quite as broad as what's available on a native Linux system. It's not as easy to configure since the spec is now longer than just three lines. And it's new and different, so that might put some people off. And now we're going to go over some of the issues we ran into in our implementation. So we as developers, when we write code, we just want it to work out of the box. We write a code, we run it, we expect it to work as it should. But that never happens, right? So this is the first issue which we faced. So OSD pods run as privileged pods. That means they require privileges. And in case of privileged pods, Slash Dev is mounted differently there. So Kubernetes prevents block storage to be shown inside the pod. So we came up with a solution. We had an empty directory. We mounted the block device to the init container. And that empty directory was also mounted on the privileged pod. So if we look here, so on the right hand bottom, we see there is an empty directory and there is a block device set to one dev zero. And in the left hand side, we see that on the privileged pod, we are having just one empty directory mounted. What the init container is doing, it's copying the block device to the empty directory. As in Linux, everything is a file, right? So it's copied there and we got the block device in the privileged container. Any questions on this one? So next problem which we faced was when we were running multiple OSDs on the same node. So Rook Chef uses LVM and what was happening that Slash Dev was mounted inside the privileged container. So we have all the devices Slash Dev Slash SDA, SDB and the block volume is presented as a loopback device into the pod. So what was happening that LVM was picking up the original device Slash Dev Slash SDA or SDB and also the loopback device. So it was confused which one to use. So for this, we found a solution that we were passing the LV name to the Chef OSD start command that is Slash Dev Slash name of the volume group and then the name of the LV. The third issue which we faced was proper distribution. So when we were running multiple OSDs, so all the OSDs were, suppose we had three nodes, OSDs were coming up only on one node. So if an OSD was having replicas, so if the three replicas are on the same node and if the node goes down, so we will lose our data, right? So for this, we found a solution that we were using basement affinities. So on the, here you can see this is a standard basement affinity which we apply to our pods. So now the pool is time. We are demo time and here is a cat peak and which is praying that hope it works. So we can work on a Kubernetes cluster but because of internal automations, we are using OpenShift cluster here. We can see that three OSDs are running. Two OSDs are on the same node and one is on triple two. So we are not applying any pod affinity. That's why two OSDs came up on the same node. And if you know about self status, so it is checking the health of the cluster. So we have three months, a manager and three OSDs which are up and in. So we are having four workers here, worker nodes. So in OpenShift, we are using something called machine sets. This makes it easier for us to add nodes or delete nodes. So what we are going to do is we are going to delete a node and we will see if the OSD is coming back up without any loss. In normal case, what will happen when the node is down and we will have to take the device out of that node and put it through the another one and several detectives as a new OSD and the data will be lost and there will be a lot of rebalancing and stuff like that. But in this case, when the OSD is back up is the exact same device running somewhere else. So we won't run into any issue. So I'm going to delete this 143 node, which is yes. So when I delete the node, so this OSDs are going to migrate to a different node. Now in this case, since we do have two OSDs, we did a really dirt symbol configurations. We don't have any affinities on there. So we have two OSDs on the same node. There will be disruption to the data readability or data writeability technically. The any volumes on those OSDs will go into read only mode until the other OSDs come back up. Can I just delete the node? Yeah, the machine. Oh, okay. Yeah, there we go. So we can see that the two OSDs went into a pending state and it came up on the two OSDs coming up on the other node that is triple two. I'm willing to bet the set tools pod we were using was also on that node. Yeah, so two OSDs are down and here the two OSDs are in pending state. And we will have to wait for these two OSDs to come back on the node. Check the pod status. Oh, the prepare is still running, okay. Oh, yeah. So in this case, because we're deleting a machine, OpenShift will also perform a drain of the node which will push all the pods off the node. When the pod gets removed, when the pod gets descheduled from the node, any PVC that it's running will get detached from the node. Now I specifically say the PVC gets detached from the node. What that means to the underlying storage system and the host itself depends on the storage driver that is managing that device. So on something like a GP2 volume in this case, it is in fact getting detached from the host in runtime. And then the GP2 volume just kind of sits in the ether for a while until a pod that is claiming it gets scheduled onto a new node. Ah, and they're running. Yep, they're running and the host is back to okay. That's all, it worked. And as we are not using any placement affinities, that's why that pods turned up on the same node as it would have been on the different nodes in different AZs. Yep, so one of the things we, one of the things we recommend, for instance, if you're running this in AWS, if you have access to more than one AZ, we recommend running your workers so that you at least have one storage node per AZ. That way you can isolate failures if an AZ goes down. Obviously if two of your AZs go down, well then you're kind of screwed, but hopefully Amazon service guarantees are such that you'll never have, you'll rarely have two, more than one go down at a time. Can you speak up a little? Wait, what do you advise us? So if anybody. So in this case, in this case what we're doing is using remote attach storage. So yeah, the storage lives separate from the actual host. In this case we were on AWS, so that was just a standard. So the machine is full? Well, the machines already exist, right? So the AWS node, the AWS instances already exist. So all that happens is that Kubernetes detaches the GP2 volume from one node and then will attach it to the next node that already exists. So the AWS EBS volume provisioner is coming into picture here. And is there a question more specific to your physical machines that you have in your labs on bare metals? So for bare metal machine and physical volumes we use locals, what was the name of it? Local PVs. No, local storage operator. So local storage operator is going to create PVs of that physical disks, and that PVs will be referenced here. Yeah. And that's how things will work on bare metal. So Kubernetes itself already has a notion of what it calls local PVs. This recent, Jan, keep me honest, did that go GA, local PVs, or is that still beta? Right, so local PVs are now GA generally available. And what those are is that it's still the same PV construct, but instead of defining storage against some external storage system, it defines storage directly attached to a node, right? So in our lovely cloud native case, AWS has a certain type of instance called I3EN, or just I3, I forget, that is just an instance that comes pre-attached with NVME devices, right? And those will just get presented as local volumes. So what you can do in raw Kubernetes is you can manually create a set of local PVs that reference those devices directly. Then when you create those PVs, you can specify an annotation that specifies a storage class name that those PVs will be associated with. So then you create this other storage class, and but then you set it to provisioner to a special value called no provisioner, meaning that there is no storage backend Kubernetes will engage with to provide that storage. So then what happens, you create a PV, the developer, as we like to say, creates a PVC as normal, and then references that static storage class. And then Kubernetes will take care of finding an appropriate PV, in this case what's known as a static PV since you create it manually, that meets the requirements of the PVC and binds now the static PV to the PVC. But it's also a local PV. So you now have a PVC that's referencing a local device, meaning that, and then with another feature that just came in called topology aware provisioning. The PV PVC entity can inform the Kubernetes scheduler to restrict where the pod that will consume the PVC will be scheduled. So you can literally just have a pod that references a PVC with a local PV underneath it. And then based on the restrictions of whatever PV it grabs, the pod will be scheduled, for instance, on your AWS instance that has your NVMe devices. Now I'm talking about AWS and cloud, et cetera, but this is directly what would apply to bare metal environments, right? You would simply create your local PVs for any devices you want to consume as storage devices in Kubernetes. And then you just specify enough devices, enough storage class devices at instances to consume all those devices. So a hand like that, right? So yeah, we can do that if it's a year. So we are running a Rookshift tool pod and here I am inside that pod and from here I can run all of the step commands. For example, we can accept status. We can, if you want to remove an OSD, we can do this. So we can mark an OSD out. So this is marked out and if you see the status, so two are in. So we can do all the operation which we can normally do in any step cluster. Yes. Yep.  Hold on, can you say that again louder? So whether the Rook operator will spin up a new OSD if it detects that there are only two running? Oh no, in this case, no. If my understanding is correct, the OSDs, can you, let's see, the OSD pods are still running, right? So from the Kubernetes API standpoint, the OSDs are still there, Rook is fine, right? This is now internal to stuff which is beneath the Kubernetes API and thus the two don't necessarily interact. So when the Rook operator is down, it doesn't matter because after the cluster is up, it will be fully functional. So we will need the operator in case we are upgrading the cluster or adding a new OSD as we don't need the operator. So it's just there to keep track of things and if they are working properly or not. And one thing I just, I remembered the thing I wanted to say, can you show the, what's it, the tree, the hierarchy of the OSD tree or whatever it is? Yep, so the main thing I wanted to point out here is that the way we got this, the way we got this whole portable OSDs things to work is that we have set the host here to the name of the PVC, right? So traditionally in Rook, this would just be the same as a standard stuff cluster where host is the host name of the machine that the device is attached to. In this case, we set host to be the name of the PVC that is representing the device which is how we keep the OSD's identity. So when the OSD leaves the cluster that is marked as out. And then when it rejoins the cluster, it sees that the IDs match, so it's marked as in again. So this is the case when we are specifying portable is true. So when we put the portable as false, here the host name will be the name of the node where the OSD is running. Probably the question was, can you have multiple devices across multiple nodes represented as a single device? Maybe? I have never heard a compelling use case as to why this would happen. In particular because like, all right, so you could argue that you could have, oh, now I have this fancy distributed redundant device which is double redundant since stuff already has built-in redundancy for your data, right? So if you configure stuff correctly, you don't need to have further redundancy in your devices. So you're saying you have Prometheus, so you have Prometheus data, right? And are you saying, do you want to be using a stuff volume? Is that what you're saying? Right, so in this case, you're using a stuff volume. The stuff volume is a separate entity from the storage devices, right, from the OSDs. So a stuff volume is part of a PG, right? Yeah, stuff volume is part of a PG. Is it like process group or something? Placement group, thank you. You can see how familiar I am with stuff. It's part of a placement group and then that placement group contains a set of OSDs, right? Which defines which OSDs that the data on that volume will be distributed to. And it all comes down to blocks, right? At the end of the day, your volume is just a chunk of blocks and where the blocks get replicated is irrelevant from the perspective of the volume. So if your node goes down, the stuff volume just, in this particular stuff detects, oh, that client went away, the volume is no longer attached to that. And then when, in this case, you would have a stuff CSI Kubernetes volume, Kubernetes PV, that will just get attached to a new node and then the stuff CSI storage driver in Kubernetes will spin up a stuff client on the host of the new machine and connect to the stuff, to the stuff cluster, access the same volume, the same data comes back up. Yeah, I mean, the default crush rules set the, what is, no, so it all comes down to when you just specify your block pool. So Rook has another entity called the stuff block pool, which specifies, you know, I'm gonna be using these OSDs and here is I'm gonna be distributing my data, right? Yep, yep, yep. So I believe that if you just have a really dirt simple stuff block pool, the default replication point will be the host, right? So even if you have like, you know, three OSDs on the same host, if you have your replication point be host, the crush algorithm will only select one OSD from that host and find two OSDs on two other hosts. So, you know, the main thing, the main takeaway here is that, yeah, I'm not sure either, but the main takeaway here is that stuff can do, yeah, so the main takeaway here is that there's a lot of things that stuff can do that can be relatively easily configured via Rook resources, right? So what we just showed you here is a very sort of low level type of redundancy here. That is in a way sort of irrelevant to the higher level SEP configuration that you can do, right? So the real trick to this is that the way we wrote it, SEP doesn't know what's going on, right? SEP doesn't know and more importantly, SEP doesn't care. So it doesn't know, yeah, thank you. It doesn't know that the devices are moving between hosts or not. All that SEP knows is, is it down? Yes. Oh, it's back up, good. And that's it. So from the standpoint of a traditional SEP mindset, that host just went down and now it's back, the end. I'll just show you the clusters here once. So this is the cluster.yaml that comes with Rook, with the Rook project. It is obviously a heavily detailed and heavily documented piece of YAML that tells you how to use each and every field in the SEP cluster. Or ideally, each and every field we want you to be using in the SEP cluster. There is a, okay, correct. How do we check this cloud instances of storage? That is beyond the scope of this presentation. So we don't have that much SEP knowledge. Yeah. Yeah, so the question was basically, how can you guarantee that any particular cloud instance that's storing your data is storing your data reliably? What does SEP do to mitigate that? And the answer to that is, that's beyond the scope of this presentation. We are not necessarily SEP experts. We are sort of cloud native developers pretending to know about SEP. So take that with a grain of salt. All right, I'm being told we've got five minutes, so I'm gonna call it there. Thank you everyone for joining us. Have a good rest of the day. Safe journey's home.