The Cube presents On the Ground. Welcome to Silicon Angles, The Cube On the Ground here at Oracle's headquarters. I'm John Furrier, the host of The Cube. I'm here with Neil Mendelsohn, the Vice President of Product Management for the Big Data team at Oracle. Welcome to On the Ground. Thanks for having us here at the headquarters. So Big Data, obviously a big focus. Oracle Local & Rural is right around the corner, but in general, the Big Data breadth of products from Oracle has been around for a while. What's your take on this? Because Oracle is doing very well with this new cloud story. When I interviewed Mark Herd, 100% of the code has been cloudified. Big Data now is a big part of the cloud dynamic. What are some of the things that you're seeing out in the marketplace around Big Data and what does Oracle fit? Well, when this whole Big Data thing started years ago, I mean, Hadoop just hit its 10th anniversary, right?  And now I think people have realized that to get the real value out of these new technologies, it's not a question of just the new technologies alone, but how do you integrate those with your existing estates?  But what's interesting is with Hadoop and some of these other ecosystems, what customers are looking for is to not just use Oracle databases, but to use whatever they might see as a feature of some use case. Hadoop for batch. So you guys have been connecting these systems. So can you just quickly explain for a minute how you guys look at this choice factor from a customer standpoint? Because there's a role for Hadoop, but Hadoop isn't going to take over the whole world as we see in the ecosystem. What's your role vis-a-vis the database choice? Yeah, so we very much believe, you know, when Oracle started, it was all about database and it was all about Sequel. And we believe now that the new normal is really one that includes both Hadoop, no Sequel and relational, right? Sequel is of course still a factor, but so are the ability to interface in via REST interfaces and scripting languages.  And we've been taking what we had done previously in database and really extending that to data management over Hadoop and no Sequel. We had a great chat at Oracle Open World last year and you told me about your history at Oracle before you did your run with startups. You've seen this movie go on early days with data warehousing. So I got to ask you, big data is not new to Oracle. Obviously the database business has been thriving and changing with the cloud around the corner and certainly here on the doorstep. But could you explain Oracle's database, I mean, a big data product offering? Sure. What was the first product? Take us through the lineage of where it is because you guys have product. We do. And a slew of stuff is coming. I can imagine, I'm sure you can't share much about that, but talk about the lineage right now. So we started about three years ago on the Hadoop side by making an appliance made for Hadoop and then in the future, which followed on with Spark. And that appliance has been doing well in the marketplace for a number of years. And we've obviously continued to enhance that. We then took what we perfected on-premises and we moved that up to the cloud. So we have a big data cloud service for customers that offer them high performance, access to Hadoop and Spark, and without then necessarily the need to actually manage security and all the things with it. At Open World, we'll be making a series of announcements. We'll be creating yet another big data cloud service. This one will be fully managed, fully elastic for customers who only want to take advantage of a Hadoop or Spark service as an example and don't want to deal with the ability to specifically tweak the environment. We also announced a little while ago our family of cloud machines. So you'll see a, the first cloud machine is one that provides Oracle, IAS and past services and then we'll add to the family- That's shipping already though. That's shipping already. And then we'll add to the family an exadata cloud machine and a big data cloud machine. And the cloud machines are really kind of a cool concept. They're cool because for a lot of customers from a regulatory point of view or otherwise they're just not ready for the public cloud but everybody wants to take advantage of what the cloud provides. So how do you do that behind your firewall? How do you provide IT as a service? So what Oracle has done essentially is to package up its cloud services and able to deliver that to customers behind the firewall and they get the exact same technology that they have on the public cloud. They build to one architecture and then deploy it wherever they choose. They get the advantages of the cloud. It's a subscription service, right? But they can deal with, but they can adhere to whatever data sovereignty or issues that they might have. So let's get to that regulatory dynamic in a second but I want to just back up.  Big data service. Cloud service, cloud, about a year ago. About a year, that's out there. Those are basically connect appliance that's on-prem with the cloud. And then now you have the cloud machine series of enhancements coming at Oracle Open World. Right, as well as a fully-managed cloud service that will add to the mix as well. Okay, so let's get down, so that's going to bring us fully cloud enabled, cloud on-premise, all that kind of dynamic flexibility and option for configuration and provisioning. Okay, back to the regulatory thing. So what's the big deal about that? Because you mentioned that. Most companies we talk to love the cloud. They love the economics but there's a lot of fud and fear internally amongst their own team about getting sued, losing data, certain industries that they might have to play. Is that a factor? Can you explain that for someone? That's important. Yeah, I mean, for some customers, it's a real concern, right? And the world is dynamically shifting. I mean, look at what happened a few months ago with the Brexit, right? I mean, all of a sudden it was okay to have the data as long as it was in the EU. Well, the EU is now shifting. So where does the data go, right? So from a regulatory point of view, we haven't fully settled, right? In terms of where customers, where customer data can be held, exactly how it's treated and those things are evolving. So for a number of companies, they want the advantages of the cloud but they don't necessarily want it on a public cloud, right? And that's why we're offering these new cloud machines because they can essentially have their cake and eat it too. So interesting, the dynamic advantage is that this whole regulatory thing is a moving train relative to the whole global landscape. Who knows what's going to happen with China and other things, right? Right, and I think that's what's really terrific is that, our history is, of course, we're a company that's been around for a while so we started on premises and we moved up to the cloud and our customers are ones that are going to have kind of this hybrid kind of a system, right? Other companies started much later and they're cloud-only. And while that's great for companies that want the public cloud, what do you do if you're in a regulatory environment that isn't ready to be public cloud? Now you have to have two architectures, one for on-premises and one for cloud. And then how do you deal with a moving landscape where a year from now, things that are on-premises can move to the cloud and other things that are in the cloud may have to move to back on-premises, right? How do you deal with that dynamic going forward and not get stuck? So is it fair to say that Oracle is a big data player in the cloud and on-premise? Absolutely, right? And not just for data management. I think that while we started at that core, that's our heritage. We've so much built out our portfolio, right? We have big data products in the data integration space and the machine learning space. We have big data products that connect up with our IoT strategy, with data visualization. We've really blossomed as the marketplace has matured, bringing additional technology for customers to utilize. Okay, so let's get down to the reality and get into the weeds with customer deployments. How do you guys compare vis-a-vis the competition? Now you've got the on-prem with the BDA, big data appliance with cloud service, cloud machines to create some provisioning, flexibility on other architectures that customers may choose for whatever reason that they would have. How does that compare to the competition? Well, on the on-premise side, if we start there, there was a recent forester wave that looked at various Hadoop appliances and we took the number one category or the number one position across all the three categories that they looked at. They looked at the strategy, they looked at the market presence and they looked at the capability of what we offered. And we ended up number one in that space. On the cloud side, of course, we're maturing in terms of that offering as well, but we're really the only company out there that can offer the same architecture, both on cloud and on-premises, where you don't necessarily have to go all in on one or the other. And for many companies, that's exactly what they need. They can't necessarily go all in one way or the other. So I got to ask you kind of put your historical, historian, tech historian hat on, as well as your Oracle executive hat on and talk about some of the technologies that have come and gone over the years and how does that relate to some of the things that are hyped up now? I mean, certainly Hadoop was supposed to be in this new industry, it's going to disrupt the database and Oracle's going to be put out of business and this is how people are going to store stuff. MapReduce, now people are saying, why even have Hadoop in the cloud when you got an object store? So, you know, things come and go. I'm not saying Hadoop is going to come and go, but it's good for batch. So what's your comments on that? Can you point to industry, technology, okay, that's going to be a feature of something else that's a real deal. What are some of the things that you look at that you can say? So, you know, we're seeing exactly as you described, a few years ago, you go to a conference and it was all about MapReduce, right? Now, a seminar on MapReduce, nobody goes, right? Everybody's going to Spark, right? And there's already things that potentially replace Spark, things like Flink, and we're going to see that continual change and a lot of what we focused on is to be able to provide some level of abstraction between the customer's architecture and these moving technologies. So I'll give an example. Our data integration technology, right? Historically, that was, you know, you're able to visually describe a set of transformations and then we generated code in SQL or PL SQL. Now, we generate code not only in SQL and PL SQL, but we generate that same code in Spark, right? If tomorrow, Spark gets replaced with Flink or something else, right? We simply replace the code generator underneath and all of what the customer's built gets preserved and moved into the future. I think a lot of people are now becoming concerned that as they take advantage of open source really, really at the very low levels, they have the potential to essentially get stuck, right? In a technology which has essentially become obsolete, right? As any new technology evolves, we move from people who just code, right? With all the lower-level stuff up to a set of tools. And, you know, we talk to companies now that have huge amounts of now legacy map-reduced code, right? You think only a few years ago- Kind of like COBOL. Yeah. So, it's going to be around, but not really pervasive. Right, so how can you, you know, take advantage of these technologies, right? Without necessarily having to get stuck to any one of them. So, we're going to answer the philosophical question. Obviously Oracle and database business has been the star over the years, since the founding. But even now, it seems to me that the role of the database becomes even more important as you connect subsystems, call it Hadoop, Spark, whatever technology's going to evolve, as a feature of an integrated system, if you will, software-based and or engineered system coming together. So, that seems to be obvious, that you can connect in an open way and give customers choice. But that's kind of different from the old Oracle. I have a database, everything runs on Oracle. Oracle on Oracle is great, certainly it runs well. But what's the philosophy internally? Obviously the database team sitting there, must be like, wow, big data is an opportunity for Oracle. That's right. Or do they go, no, the database business is different. How do you guys talk about that internally? And how do customers take away from that dynamic between the database crown jewel and the opening it up and being more big data-driven? I think it's ironic because externally, when you talk to people, they just assume that we're going to be like, oh my God, this is a threat and we're going to just, you know, double down on what we're doing on the database side and you know, we're just going to hunker down and I don't know, try to hide, right? But that's exactly the opposite of what we've really been doing internally, right? We really have embraced these technologies of Hadoop and Spark and NoSQL. And we're essentially seeing data management evolve, right? That is the new normal. So rather than looking at, you know, as we might, not only we might have said, we did say when we introduced Oracle in the data warehousing market back in 95, we said, put all your data in the Oracle database. We're not saying that anymore, right? Because there are reasons to put data in Hadoop. There are reasons to put data in graph databases and NoSQL databases. We need to be able to provide those choice, right? While still integrating that data management platform as one integrated entity. Would you say then it was fair to say that from a customer standpoint, by having that open approach gives more faster access to different data types in real time? Absolutely. Then isn't that the core value proposition of big data?  And for a lot of companies today, they still are faced with this conundrum which says in order to analyze data, I have to put it all in one place. So that means that you have to move your operational data, you know, into the one place. You have to move your data warehousing stuff into one place, right? But then at the same time, you mentioned real time. How do you get in the business of moving data from place A to place B on a constant basis while still being able to offer real time access and real time analytics? The answer is you can't. And the value of the data, the data capital as we've been talking about at Wikibon is an IoT piece of data from a turbine could have really big relevance to the system of record in another database and that has to be exposed and integrated quickly to surface some insight about the quality of the data. It's the thing that gives you context, right? Today what's going on is that we are getting all access to all these rich data sources and rich data types that we didn't have before, whether that's, you know, text information or information coming off sensors and the like. And the relevance of that information is when we combined it together with the corporate information, right? The stuff that we have in our existing systems to really reap the true benefit, right? How do you know, you know, when you get a log file, the log file doesn't have anything about the customer in it. The log file just has a, you know, a number associating itself to a customer. You have to tie that together with the customer profile, which data which might not exist in Hadoop, maybe it's in a NoSQL store. And certainly the open source is booming with Oracle. You guys are actively involved in all the different open source ecosystems.  Many people don't think of, you know, they're not even aware that Oracle's behind MySQL as an example, right?  You know, and I'm like, well, a little bit, right?  He's like, no, it's not. Right? You know, because- It's too cool to be Oracle. That's right. That's not a bad thing, right? I mean, the reality of it is, is that, you know, we've invested a whole lot of time and energy in these technologies and we're really looking to commercialize them, to mainstream them, to make them less scary for more people to be able to get value from. Well, your son's examples are great in the illustration of the new Oracle that's out there now, this whole new philosophy. Finally, I'll give you the last word real quick. For folks watching, what's one thing you'd want to share with them that they may or may not know about Oracle and its big data strategy? Give us a look, right? I mean, I think that, you know, when you think of big data and you think of these new technologies, you may not think of Oracle, right? You may think of the new companies that you're more familiar with in a light. The reality of it is that Oracle has an extraordinarily rich portfolio of technology and services on the cloud, as well as, you know, like cloud machines. So give us a look. I think you'll be surprised at how open we are, how much of the open source technology we've embedded in our products and how fast we're essentially evolving into, you know, what is the new normal. Neal, thanks so much for spending the time with me here on the ground. I'm John Furrier, you're watching exclusive on the ground covers here at Oracle headquarters. Thanks for watching. Thank you.


Alright guys, so around this time last year, actually around September last year, I entered my doctorate in Chinese medicine. So I jumped right from working a full-time job, or at that time I'd gone into my business full-time and now into full-time medical school while running a business. And the reason for me continuing to do both is because I already had a business and I wanted to graduate as close to debt-free as possible. So I came in gung-ho, guns-blazing, saying that I was going to pay off all my debt, in the very first year managed to burn myself out and have a lot of health issues from overwork and just from stress. And so this 100-day manifestation experiment was to see if I could bring back that excitement that I used to have, as well as still get even better results. What's up guys? Alex Hein here, author of the book Master of the Day. So I talked to a friend around this time when I was trying to figure out, you know, what do I do next? Do I start a new business? Do I just kind of run the current one kind of half-half? Probably that excited. And what she said to me was, just for 100 days, try only doing the things that excited you. And I want to focus on the four things I did and the four things I asked for that ended up happening so that you can get started with your own experiment on how your thoughts create your reality and you can decide for yourself. Because for me, I was very skeptical. I mean, I'm open to this kind of stuff, but the thing is like what got me to where I am in my life today was work. Like I know how to work. I published already four books. I wrote my last full-length book while I was in medical school here and then published it in the summer. Like, I know how to work, but just because you know how to work and you get results doesn't mean you enjoy it or that you're happy or that you can feel like you can do that forever. And I want to be working for the next 70 years. So I started with four things. So the very first thing was I would ask for whatever it is I wanted. Something for me was a formal, I wrote it down on paper, please help me with this. Find this. Do this. I did it in the morning and I did it in the evening. The second thing was I kept a little journal to keep track of my intuitions. And the way I knew something was intuition was number one, I felt excited by it. And number two, I felt it kind of in my stomach like, oh, that's a good idea. Not like up here like, yeah, I should do that. I should do that. It just popped into my head and I was like, oh, that would be cool. That's the way I knew that if something was intuition. And I tracked these little intuitions because they were very easy to miss. And I put them into a little journal, the one I've shared in a previous video. And I just would track these are all the intuitions. These are the things I need to act on. The third thing I did was I surrendered any negative emotion, stress, fear, anxiety, comparison, issues with self-worth. I surrendered, meaning I would say in my head, I surrender. I let go.  OK, maybe I will. That's it. Surrender, let go. I don't know what's going to happen. What if my business fails because I can't perform in medical school and in my business? OK, it could happen. Surrender, let it go. What if this girl says that? That person says that. Surrender, let it go. That was the third thing I did. So internally, I was clear, clear, clear, clear of any negative emotions day to day. And then the last thing was that I only did the things in my business. And as best I could in my life that excited me. That's it. The way you determine excitement was in my body. That's how I determined if it was exciting. Not, I should do this. It was, yeah, let's do it, all right? Everyone knows the feeling of excitement. Now, I wanted to do this not only to feel better, to feel more excited about my life, but also I wanted to see if I could get better results in my business and feel excited. So I wrote down four goals, four things that I was working on coming true. The first thing was that I wanted to make an extra thousand a month in my business to really provide more financial security and to pay off my medical debt. I wanted to start on Instagram even though I was really resistant and didn't really like it. I wanted a strategy that did not take a lot of time that I enjoyed that was basically not me doing more work. I wanted to get set up to do a TED Talk and I wanted to come out with a Master of the Day journal. All right, so the first thing was that as a business owner, the first thing I have to focus on is money. I don't like that fact. I don't want to focus on the only things I make money. I just want to write books and shoot videos. But the reality is those things don't make money by themselves. I have to study business. And so I thought every night, every morning, I would say, please help me make this without doing extra work, without working twice as hard, without doing something I hated. I want it to be easy and fun and exciting. So one of the things I was pondering was with my content on YouTube here. You know, most of my top content is personal development. But my brand, ModernHealthMonk.com, is about really about fitness and weight loss. That's where all my courses are related to in my coaching. So it felt like a mismatch. But what excited me the most is the personal development content. So I said, you know what? Fine. I'm just going to create the content I like that you guys like, the personal development stuff, for 100 days and see what happens. Well, the very first video I shot was super off-brand. And it was all about how to journal to be more successful, right? The journaling exercise that can change your life. Now, would you believe that very first video I shot, off-brand, whenever excited me, that video went viral. And it pulled up so many other videos. And I want you to look at my AdSense check here for the month and what it's been for several months now. And isn't that interesting that the very first thing I asked for was an extra 1,000 a month in income. Now, that's kind of crazy, right? And that's kind of humbling. And it made me just think, how? Like, how? I don't really know. I asked. I listened. And then I did. And that's what I think you're going to learn that I learned is the key thing. When you ask, an answer comes, the listening is the little intuition that's very easy to ignore, but then you have to do it. So the second thing is that I've been wanting to start using Instagram.  I hate what it does to people. I hate when you start uploading, it changes the way and the kind of content you upload. And so I was afraid that would happen to me. But I was kind of struck with the fact that I got these really meaningful messages, these really interesting direct messages on Instagram from you guys. And I thought that was really rewarding and really humbling. And so I wanted to figure out a way where I could start uploading content and build that platform over the next years, but without doing extra work because I wrecked my health the first year of medical school, from overwork, from stress. And so I asked what I should do.  So I put out this ad on Craigslist. I was looking for a photographer. And I mentioned that I was looking for people to do a shoot or potentially do some kind of long term deal where I can give them exposure. What happened was a bunch of people responded. But the last person that responded, just so happened, I put a Craigslist ad, remember that, just so happened to be a girl at my school who is now working with me to produce a lot of photography, a lot of the videos, and a lot of this other stuff. And that came out of nowhere. Like when you think about the chances of finding someone in a school of what, less than a thousand people, and you put a random Craigslist ad, it's kind of crazy. And so sure enough, we found a strategy that could work for uploading daily to Instagram that doesn't take a lot of time on my end and can still be impactful and still help a lot of people. The third thing I asked for is just some help on getting this Master of the Day journal created. The thing is, like I have to go through China to get a supplier and then get a distributor and have them do all this stuff. And there's the potential to lose a lot of money that I don't have to create a print journal. And I know journals are really popular now. And the one I want to create is very different in a very simple way. And so I've been working on creating it because I've used these exercises in my own life. And I know it'll help many of you. But I was getting caught up in all the barriers of I need to put money up front. I need to deal with the Chinese. I need to go through Alibaba.com. I need to deal. It's very complicated. And it's a long process. It's like one to two years from idea to inception. And one day, when I was asking for an easier way, a faster way to figure out how to do this, I got an email from Alibaba.com, which is a Chinese. It helps you connect with manufacturers. And I must have signed up and forgotten about it. Because I got this email, and it said to me, here are our best-selling hardcover journals. And I clicked that. And I set up a quote pretty effortlessly inside their system. So I went from reading like, all right, how do I do this journal? Do I create a journal? So the question was, do I create a journal? That very first video on journaling went viral. That was yes. And then it was, how can I do this faster? In my inbox comes the Alibaba email.  I'm not really super concerned with the time, whether it's now or in two years or five years. But one of the most common comments I get from you is, when are you giving a TED Talk? And part of it is what's massively on my plate right now. But it's also like, when is it? When do I have to coordinate? I have to coordinate three to six months to prepare. And so I mentioned it to that photographer that I mentioned. And she said, oh, you mentioned TED, right? Well, they're actually here next week in Portland, taking their applications. So TED was actually locally in Portland, doing their Dropboxes for people to apply to speak. And so I was like, OK, OK, it's kind of short notice. Next week is my finals week. I don't have two weeks to do a talk that's going to be on TED possibly, and also my finals. Not enough time. The next day, I'm working in a coffee shop. I come home. I'm driving on my regular route to home. There's a humongous billboard. It says, come speak at TED, April, whatever. And I just started laughing because I've been here already, and I've been here for almost two years. I haven't seen that. I haven't seen that last year. And so it's interesting, when you ask, you begin to see things aligned with whatever it is you're trying to have come to reality. So I think there's one final lesson here.  Certainly not in the last seven years, for sure. So if there's anything I would recommend, it would be to do those four things I mentioned. Ask the intuitive journal to follow your excitement. Don't resist. But really, don't resist because if you're not at your goals yet, there's going to be frustration and pain and discomfort. And if you do those things that are exciting you, it won't matter when you arrive because you're already having so much fun in the process. It won't even matter. It won't even matter. And that was a huge revelation for me. Listen, the intuition will be small. It'll be a little thing that's easy to ignore. And then you have to act on whatever is exciting you. And then don't resist if it hasn't happened yet or you're just wondering if it's going to happen. Just let it happen. So I hope that helps you guys. That was kind of an interesting 100 day experiment I did that I hoped would serve me for the rest of my life. And thus far it certainly has. So I'm making sure to do a lot of interesting documentation on what I did, what I would do differently, and how it impacted my life. Now before you go, I want you to leave a comment there below. Let me know for you what would be the coolest thing you can manifest. And be unrealistic. And I'm not saying like, oh, a billion dollars unless you think that's what you really want. But like, what do you really want? Tell me there below. Now, the best way to stay in touch is to come get my free personal development and weight loss challenge at modernhealthmonk.com forward slash YouTube. You can also see my latest two videos and related videos right here and right here. And otherwise, I will catch you guys soon.


This is absolutely my favourite day in the guide dogs calendar because it just brings together so much about what it is we do and it really joins all the dots in terms of all of the people that are involved in our work. I think this is a fantastic event. It just brings everyone together, all the different pieces. It's about 60 or 70 staff here helping out and how committed they are. This is fantastic. I guess it really sort of encapsulates why we do it, why we puppy raise. Well I was teary because it just made it all so worthwhile and I was so proud of him. Having Scotty graduate today is just the most amazing feeling. It's the end of an incredible journey and he's gone to the very, very best handler I could have wished for. Anyone who is a puppy raiser or was once a puppy raiser please stand so we can acknowledge him. Try when we give them back but then to know what they're going to do and to see them at graduation and just see the smiles on the client's face it's what it's all about. So I've had her for just over three months and she's changed my life and she makes it so much easier for me to get around and confidently and I can go out to bars with her even and just do everything that I need to do.  Oh gosh, she has changed my life. I now walk down the street, go to work without somebody having to pick me up. It's opened up a whole new world. It is great. We try to match people who like to get into the city with dogs that eat up that kind of environment and there's other people who just want to plod down to the local store and so we give them that laid back kind of dog. We gradually step back, let them do it on their own and gradually watch them walk off into the sunset without you. Please join me in congratulating the 2018 Guide Dogs Victoria graduates, their puppy raisers and our fabulous sponsors. Thanks very much for joining us today everyone.


Good morning everyone. Welcome to this topic. I'm William Wang and I'm the maintainer of the volcano community and from Kauai Cloud. This is my partner. My name is Li Mengxuan. I come from the false paradigm. Okay. Today our topic is cognitive batch computing with the volcano. We have some updates and the future roadmap sharing in this topic. So there are four parts. The first part is about the volcano introduction and then I will share the updates and the improvement of the volcano community. And the third part is about the GPU sharing and isolation from my partner. There will be a deep depth and finally I'm going to share the community and the roadmap. So here is the overall volcano architecture. From the picture we can see that volcano community have strong strong relationship with the upstream computing frameworks like Tencent Flow, PyTouch, Spark, Flink and so on. And also we can see that volcano is now just a scheduler. So it has a pod group and job group controller to provide the enhanced job management. And also it has the QCRD to help users to share their resources more efficiently. And also we have a lot of, we spent a lot of effort to work with the analyzing hardware to support different kind of heterogeneous devices such as GPU, S86, APU, TPU, something like that. And also we will work with the Kubernetes team to improve some performance especially for the AI workload and Spark workload. So this year we will add two more components in the architecture. First one is the volcano agent. The volcano agent is designed to provide the kills management for the workload. And also we will add a new component named the rescheduler. So this rescheduler will work with the permissions monitor system to balance the resources more efficiently in Kubernetes cluster. Here is the general of the volcano. So at the beginning we support, we designed a set of batch API to support the AI workload running on the Kubernetes. And then we supported the pod group, the QCRD and provide a set of scheduling policies for big data area like the FASHF scheduling the resource reservation to prevent the resource competition between the Spark driver and the workers. And then we have received a lot of feedback from the community users. So they said there are so many kind of training operators. So it's hard for them to maintain all these kind of operators. So we enhanced the job management to support the volcano job API to unify scheduling all kinds of training frameworks. And then today we are doing a lot of work to support the large language model training and inferences. So I will share some new features at this area. So the first feature is the job flow. As we all know, the workflow is very common requirement for some traditional batch workload and the AI pipeline. So there's a lot of projects like AGO, the air flow, the Qube flow. Still some users come to the community, said they need a lightweight workflow or workflow scheduling engine. And also they need the more complex workflow engine, just like this. So in the picture you can see that in their use case they need the workflow to support the AFL or switch case symmetric. So we designed this new project, it's lightweight workflow management. Here is the example. There's two CRD for this feature. The first one is the job template. Users can define their job using the job template. And the third one is the job flow. So users can define the dependency between the jobs in the job flow. The next new feature is the load awareness scheduling and the risk scheduling. So we can see that in your cluster we all know that the utilization for each node is very different. So we expect the usage on every node is balanced. So the basic reason is that currently the request and the node allocable resource is scheduled. And the next reason is that the pod goes away from time to time. And new pod is created from time to time. So it's hard to keep the resource balanced on each node. So in this feature we support the work channel scheduler to communicate with the monitor system like permissions and the ERK system. So the scheduler is awareness of the usage of each node and scheduler according to the usage. The next one is about the capacity scheduler. As we all know, we cannot support the proportion scheduling several years ago. The proportion scheduling is very flexible. So it can share the resources among different queues based on the weight. And also the scheduler reclaim resources by the user. And when the user scale more node into the cluster, the scheduler can keep the ratio all the time. It's very helpful and flexible. But also we have some use cases that the user said that they want to configure the resources and share resources by their configurations. So we support the capacity scheduler policy in the channel this year. As you can see, the policy is the policy is support user to configure the maximum resources in the queue and the capacity resources in the queue. And also we support the reserved resources in the queue. So these three kinds of definitions support the fine-grained scheduling and the reclaiming in the scheduler. Here is the use case from the LinkedIn. They have super large GPU clusters and they use the volcano and use the queue to share resources. But in their production environment, they have multiple organizations in their cluster. And for each organization, they have the different kind of GPU resources such as V100, 800, and T4, something like that. So how to use the scheduling and share resources mechanism to share the resources and achieve a great, perfect resource utilization is a great challenge. So they tried a lot of approaches. At the beginning, they use the solution one. They use the resource quota to do the acceleration. But the resources can't be reused efficiently. And then they use the proportion policy. They can fix the weight for each queue, mapping to their organization. But the resource utilization goes up. But still something can be resolved. Something like that. The resources about the V100 and 800 can be shared fully among their different organizations. So we supported the resources based on the each dimension of the GPU. Like the red part. After enable this feature, their production environment utilization goes up and achieved good result. For other part, this year and later last year, we improved a lot of manufacturers. The first one is about the we enhanced the scheduler to support micro services. And currently, user can move their default scheduler to volcano in the cluster and very smoothly. And also, we enhanced some other features such like keep the schedule work with the auto scheduler more efficiently. And also, we add some new small features to prevent the job from preempted in some conditions. Next one is about the colocation. There's more and more users. They deploy their different kind of workload into one cluster such as co-exist the Spark application and the non-running services. And also co-exist the training workload and the inference workload to achieve better resource utilization. So we developed this feature in the cloud. And we can support the tools management and the resource or commit to make full use of the resources and also prevent the interference between different containers. This feature hopefully will be released at quota two. And next, there's a big feature about GPU sharing. My partner will introduce this feature and show how to use this feature and some more use cases. Thanks, William and the volcano community for inviting me here. I am the contributor to this VGPU feature and I'm glad to give you a deep dive about this feature. The background of this feature is the growing requirement of the computing power. As you can see in these figures, the requirement of computing power grows quite accelerating. Especially with the emergence of the large language models, as you can see with the red line, it can be as great as 375 times per year. In the meantime, GPU manufacturers has released GPUs with more rapidly with more computing power to match the trends and of course with a higher price. Another challenge is the device utilization in volcano and Kubernetes cluster is quite low. Large amounts of computing resources are spoiled. The left two figures are the core and memory utilization of a single A100 GPU in a Kubernetes cluster. As you can see, the core utilization equals zero for the core and memory utilization is quite low as well. The major reason behind this scenario is the GPU-related resources can only be used in an exclusive manner. Our solution to these challenges is to design a device sharing mechanism called VGPU. And unlike other device sharing mechanism, volcano VGPU can provide device memory isolation among containers. In other words, we can enforce the upper limit of the device memory for each container to use. It is done by a component called Hami Core, which is another open source GPU sharing project. And you can see in this figure, at the scaling level, the volcano scheduler is responsible for grasping the usage of devices and assigning tasks to an appropriate node. Then it passes on to the device to mount the Hami Core and the proper devices into the container. And the Hami Core is responsible for the container resource control. And how do we achieve that? I mean, how do we contain the resources used in containers? We achieve that by using a component. It hijacks the level search between the CUDA runtime and CUDA driver. It recalls every memory allocation and retains an OOM error if it exceeds the limit. As you can see in this figure, if we allocate 3G device memory to this container, it can only get 3G when curated by NVIDIA SMI just as the figure shows. Besides memory isolation, it has the following other features as well, like call utilization limitation. It can guarantee fault isolation and it's transparent to GPU tasks. Volcano VGPU is quite easy to use. Simply specify the number of GPU you wish to mount into the container by using vocalo.shslice VGPU number. And you can specify the device memory available for each GPU mounted using vocalo.shslice VGPU memory. In this example, we create a container with two GPUs, each have an upper limit of 10G device memory. The result is showing in this figure. Volcano VGPU can improve the GPU utilization greatly in many scenarios. AB testing is one of them. Let's consider the following scenarios. This is a typical AB testing solution platform. It has the following features. The whole system consists of a base production model and several experimental variant models. Most of the input is processed by the production model and a small part flows into the variant model. Without the help of the visibility technology, each variant model requires an exclusive GPU which is a serious waste of computing power because the traffic flows into the variant model is quite low. With the help of Volcano VGPU, however, the two variant models can share on a single GPU and they can leave a GPU completely idle for other tasks to use. There are many concerns regarding the performance of Volcano VGPU and we can safely assure you that the overhead introduced by Volcano VGPU is below 1%. On the other hand, the throughput can be increased by 10 to 90% due to the kind of task it performs very good in the inference scenario. And Volcano VGPU is officially supported in Volcano 1.8. However, there is still much work to be done. We plan to add a monetary system in May 2024 and we plan to support multiple schedule policies in this summer. We also have a plan for next year which is to be compatible with CPU operator and other heterogeneous devices like Intel GPU and AMD GPUs. We are looking forward for you to try this feature and provide feedback to us. Thank you. It's a great feature in Volcano. We are here to visit Volcano to have a try. The next part is about the community. Here is overall. For the public users we have a lot of users adoption especially for the AI and big data and coding workload. Here is part of the adapters. For the code diversity in the last 12 months you can see that we got good diversity in the community development almost more than 50% computers are from the independent developers. Also, here is the future. The major features we are going to work this year. In addition to the algorithm in terms of AI engineering, the biggest challenges for AI training and inferences are the scale, the performance and the cost. This year Volcano will focus on this aspect. The first one is about the topology of various scheduling. As we all know in large model training scenarios, network topology is and the task topology is very important as the bottleneck has changed from the computing to the network. So the scheduler will be aware of the task topology like the data parallelism and the tensor parallelism and pipeline. Also, the scheduler needs to be aware of the network topology such as the RDMA, NV-switch, NV-link and something like that. How to use the scheduler to schedule a set of code with higher requirement for the network is very important to improve the training performance. Also, for the inference we are going to add more support for the resource sharing. Besides the GPU sharing we will support the ascent NPU device sharing and some heterogeneous devices more. The second one is about the AI training on multicaster. As we all know the model is becoming bigger and bigger. So the training of multicaster is a great training and we are exploring in this area. The third one is about the ascent NPU support. As we all know the ascent NPU is more popular in some areas besides the GPU. So we will support the NPU sharing and NPU topology scheduling something like that. The last one is the colocation. So it is found that more and more users are trying to deploy multiple workloads to one cluster as we just said. So we will still work a lot of effort on this area. Finally the community will provide the broader support and optimization for the other heterogeneous devices like AMD and Intel. So there are still a lot of requirements from the community. Feel free to get involved of the community. Here are some resources. Feel free to go to the community to share issues, share the requirements, all make contributions with us together. We also have the Slack channel where we can communicate there. So that's all from my side. Thank you. Any questions? Okay, thank you for the talk. I was wondering if you could explain a little bit about the colocation. So you were talking about you can have a mix of training and inference or a mix of Spark and let's say notebooks. How you're actually doing that colocation scheduling and how it's different from your traditional batch or fair scheduling. Pardon? Are you seeing the colocation? Yeah, between long running services and batch and batch ephemeral jobs. You're saying that you have a new strategy around colocation scheduling. Can you walk through what that is? So if you go back to the slide I think it's on colocation strategy that you're introducing. Sorry. Yeah, this one. Yeah, I was wondering if you can walk through that yellow section and how that enables both long running and ephemeral jobs to exist in the same nodes or same cluster. So for the yellow part the scheduler will be aware of the job priority like the online service priority and the offline job priority and scheduler zone with different priority and preempt the resources when the resources is sufficient. And also the scheduler is needed to aware of the oversubscription resources that reported by the volcano agent. The scheduler needs to schedule the oversubscription resources to the offline workload because the over committed resources is not stable resources. So we will allocate this kind of resources to the job with low priority. So do I as a user need to provide higher priority for my online jobs and lower priority for batch and that's my contract with you? Yes. So all through just priorities, right? That's the contract that we have. Like we say priority class and priority like zero. Yeah, yeah, different priority class. We will divide the job into five different kind of priority. So user can define their job just like the latency, intensive workload with the highest priority and then the normal batch, the free, there are about five classes. Okay. Okay. Thank you.