Hello, I am delighted to announce our next talk in the NEA channel, which is about parsers and compilers. Kishou from Google will be talking about the Learn from LL1 to pegparser the hard way. I believe this is a free recorded talk, so I will be just introducing that video should be started to play. Take it away. Hi, welcome to my talk. The topic is Learn from LL1 to pegparser the hard way. My name is Kerr, and before start this talk, I want to briefly go through this photo. So the photo is came from a famous show in Japan called Sasuke, and they are getting popular around the world called Ninja Warrior. And let's see how player play this stage. So as you can see, player need to jump from one bar to the other bar and eventually reach the goal, and player need to jump it twice. And this stage is one of the most difficult stage in the game. It's called cliffhanger, it knock down tons of greater players in this stage. Why I talk this thing in this topic? It's because I want to mention how I felt when I prepare this presentation. So I want to deliver such feeling for you and hope you can also experience it. Alright, let's talk about myself. I'm currently working for Google, and I used to work for Amazon. I'm born in Taiwan and now living in Tokyo in Japan. And I've represented at PyCon Taiwan in Japan since 2017. And you can find more about me in my Github page, or you can check today's talk related source code in the project called Parser Learning. As you can see I put a photo which is a movement similar to the movement in Sasuke. This is how I try to experience the feeling in the Sasuke show, how they do such movement and how difficult it is. And let's talk about agenda. Today, agenda will cover many, many points you can see listed here. The first one is my motivation of this talk, later what is parsing C-Python, and later dive deep into parser technologies from 101 to 102, and finally C-Python impact parser and take away. Let's talk about my motivation. My motivation came from the release article, what's new in Python 3.9. In that article, there's one line say PAP617, C-Python now uses a new parser based on PAP. And when I read this line, my impression was, if I remember correctly, I took a compiler class in school, so I supposed to know it. However, I check what I learned in school, I found out school taught us the brief concept of compiler's front end and back end, and scores parser assignment mostly use bison plus yuck. And I immediately figured out my brain was lost. Yeah, here's a list of questions I asked myself at that time, and this is also today's talk objectives. What is PAP parser? Why did Python use L01 parser before? And why did Gido choose PAP parser? And what other parsers do we have? What's the difference between those parsers? And finally, how to implement those parsers? So the first thing we need to learn is what is parser in C-Python? You can find this in the below tutorial called C-Python DevGuide, design of C-Python's compiler. Here is the Python compilation steps. Initially, we have the source code, and usually, you know, we just write wrong the Python command and generate the results. And what is under the hood? Here is the steps. The first steps is called lexer. So lexer will parse your source code into a bunch of tokens, at least time, token has no meaning. So parser, today's main character here, will basically parse those tokens and run some schematic analysis, and eventually generates a meaningful syntax tree. And we call it AST. I will explain this later. And now we have our tree. Tree will go through another compiler process and output Bico. Bico later can be executed by VM, virtual machine, and generated results. That's what you can see on the screen. And if virtual machine detects some other import module, it will go through a same process and yet again generate the Bico and the result. Let's take a deep look in each step. The first step, lexer, you can use a module called tokenize, try to present it. So tokenize, take your source code as an input, and then generate a bunch of tokens here. And remember the tokens here have no meaning. And so we have another module called AST, serve a purpose as parser. This AST stands for abstract syntax tree and contains two methods, one is called parse and the other called dump. And parse can parse your source code again and generate a tree. And a tree can be presented by dump function and generate kind of output like this. And this tree contains the mean. And now you got a tree, how to generate the final output. There are two building functions to help you. First is compile, the other one is avil. Compile your tree into Bico and avil your Bico, you can generate final results. And also, if you are interested in the detail of Bico, you can use a module called this and this disassemble function will take your Bico as an input and finally generate the step of detail execution. And today's focus will just be the parser. So I talk about the entire compilation steps. However, our focus will just be a parser. If you are interested in other steps, that's not the focus in today's talk. We learned CpyCen's parser. So we can start to talk about the compiler's technologies. And in parser 101, we will talk about the content in scores lecture. So if you are a computer science student, you may already know it, it's a good chance for you to review it. If you are not, that means you can just learn like a computer science student. And the first essential item called CFG. CFG stands for context-free grammar. Context-free grammar, the grammar can be represented by this gray block. And inside the grammar, usually we have a lot of rows. And each row presented by a yellow block. And in each row, we have an arrow. Arrow means derivation. Some paper may write it in the upside, but the meaning is the same.  So the upper case means non-terminal. The lower case means terminal. And non-terminal means it can derive to other thing. And terminal means it's a final token. And we have our bar here. The bar here in CFG means end. So it supports ambiguous syntax. Because in this case, we can interpret this grammar into both upper case B and the lower case A can be derived from upper case A. And why we talk about CFG here is because in the following talk or in the paper, you often will see a lot of grammar. And so please be sure to be familiar with the meaning of each block. And what is context-free? Context-free can be explained by following two examples. And simply speaking, context-free disallows any contacts in the left hand side. That means left hand side in all the rules can only contain one non-terminal. So in the second case, that's the invalid CFG because we put some contacts before or after the non-terminal. And we say that pattern can be, can derive to another pattern. That is not a valid case. The valid case, we can only say or a certain non-terminal can derive to another pattern. Now we learn context-free grammar. What's the usage behind that? So context-free grammar usually going through another process called semantic analysis. And in that process, it will use it plus the source code as an input and generate ultimate output called pastry. And pastry contains two types. One is concrete type, the other one is abstract type. In the upper part is a concrete type. Concrete type usually is pretty complicated and pretty raw and is not so human readable. It contains many non-terminal and intermediate process going through your grammar rules. And on the upside, abstract syntax tree is much more human readable. So for this example, if you have elementary education background, presumably you know the multiply operation is higher than the plus operation. Depends on their parser implementation. We need a treatment on the CFG and that's called CFG simplification. There are totally three types of simplification we should talk about. First one is ambiguous. Second one is non-deterministic. And finally, that's the recursion. Ambiguous definition is a grammar contains rules that can generate more than one tree. So for following example, as you can see, it generates two kind of tree if you try to put some practice here. And so this is problematic because as any elementary educated people should know, right hand size tree is the correct one and left hand size one is wrong. So to resolve this, we need to rewrite the grammar. So after rewrite the grammar, as you can see, we have not only one non-terminal, we use another two non-terminal here. Try to make this grammar able to represent by one identical tree. And the second simplification is non-deterministic. The meaning is a grammar contains rules that have common prefix. So for following example, a non-terminal A can derive to terminal AB and terminal AC. In this case, because we have the common prefix A, so we can rewrite the grammar with another non-terminal and eventually generate a grammar like this. And the third one, which is also the monster challenge one for the parser implementation is called left the recursion. The explanation of this is pretty complicated if you never experienced this before. If you have a grammar like this, let's say, you know, it's very human readable, right? E can derive to E plus T and the T and the T can derive to another T, multiply F and F, and F can derive to number. This is the kind of way we just explain the kind of calculator suppose plus and multiply. However, the problem is in the implementation, you may meet the problem is what if you always deal with E first? And you will meet a problem is E direct to another E and later that another E direct to another E and your program will recursively meet the E and eventually reach the memory limitation. So unable to parse the thing. That means left the recursion. So how to deal with that? Yet again, we need to rewrite the grammar. So after the grammar return, as you can see, it's pretty ugly, not so human readable, but the meaning is the same. I'm not going to explain the detail between left hand side and right hand side. I just let you know is if you want to implement the parser in a certain way, then you need to resolve the reference recursion. That means your grammar will be rewritten into a certain way, not so human readable. Let's have a recap here. We have three types of CFG simplification. Here's the before. Here's the after. We know CFG. And we also know different parser implementation may require certain simplification on your CFG. And we also know the output is kind of parse tree. So we can start to talk about the traditional parser implementation. Before that, we need to know they are actually two type of parser. First one is top down, the other one is button top. And top down parser, it starts from the root and eventually till the leaf and generate the tree. And button up on the outside is start from the leaf and go up till the time they can't go up anymore and eventually generate tree from leaf to root. So they are two type of parser based on the two type of classification. So the first parser called LL, second one called LR. LL parser and LR parser's first L stands for left to right. That means they all parse the input stream from left to right. And second L and R means derivation, timing. So in LL parser, it does the left most derivation. That means for the first example in left hand side, you can see it's a top down approach is the LL parser approach. And with this approach, it meets the plus operator first and data multiply operator. And on the upside, the right most derivation, that is the LL parser implementation. In the right hand side, you can see it starts from the leaf to root. And so it meets the operator in a different timing. It meets the multiply operator first, then plus operator second.  And following is a kind of example, right? So start from two, oh, I, if you parse, if you implement parser, you basically add two, I'm a token of a number. And if I perform one token look ahead and meet another token of plus, then what to do next? That is the meaning of the look ahead, right? So we learned LLK and RRK's definition. Now we can move on to take a look at the detail implementation. And the first one we will take a look is the top down recursive descent parser, which is also the simplest and the most straightforward one. And for this case, you can see we have three steps in total. The first step is you need to write function for each non-terminal in your grammar. So in this case, we have five non-terminal in our grammar. And remember, this grammar doesn't support left recursion. So it's pretty ugly. And now we have the corresponding function, we can start to parse the string from left to right. And during the time we parse the string, we always need to start from the root. So that means it's parse E. And start from parse E, based on the look ahead, we can decide what derivation we want to proceed and eventually reach the end of the input string. So look at the example code. Here is the aerial one example. We can see that for the grammar, we have five non-terminal and we have two examples here. One is parse E-pline and the parse T. And in parse T, it's pretty simple. We just have one possibility, one derivation. So there's nothing special. But in parse E-pline, because there are two possibilities, so we need to use a K look ahead approach. And based on, oh, we look at the symbol, next symbol is plus.  And if we didn't find the plus symbol in next symbol, then we decide to do another derivation. And that is the K look ahead approach in this case, and K equals to one. Another way to implement top down parser code non-recursive descent parser, Python's aerial one parser also implement in this approach. This way, we instead need to have a pre-generated parsing table based on your grammar. And this pre-generation require two steps. The first step is to have a first follow table for each non-terminal. That is based on non-terminal's previous symbol and next symbol. And we collect those information, later we can generate their parsing table. And the parsing table basically let you know when I am certain terminal, and I need to do based on the grammar what action I should do. And so these parsing table basically will need to implement in following step. We have a stack here. And yet again, the first error is left to right. And second error is from left derivation to right derivation. And from top to down. So as you can see in the beginning, we have their E as a first non-terminal. Later, we start to reduce, reduce, we start to run reduce to till the end. We reach the terminal and we can shift. And we will keep running this till the end. There is no more symbol in a stack. And we call it accept. That means the grammar is valid and we generate their tree. So in a implementation is also pretty simple. As you can see, we need to have a parsing table here. And then we have a stack. And we also have a pointer to point to the current place. And based on the symbol we meet, we perform shift action or reduce action. And if it's done, we eventually return a root of the tree. Let's take a look at the final traditional parsing implementation called LRParser. And LRParser in fact is more popular than LRParser, mainly because it accepts left the recursion grammar. Let's take a look at the detail. So LRParser like LRParser, we also need to pre-generate some parsing table for the step-by-step action. And the thing here is we don't use the first follow table, but we use a thing called DFA, deterministic finite automata. So using this DFA based on the state to generate the parsing table. And so in each state, we have the kind of point. And in the beginning, every point is in the start point. And we will take a look at the following symbol and perform the shift action to different states. And if the point reached the end, it will reduce based on the rule. So in this case, we will say, oh, this is the rule five. So we will reduce based on rule five. And in this case, if we are in the beginning, we meet E, then we will shift to state two. And so it's like a kind of workflow. We will keep shifting till the end and reduce based on that rule. And so the parsing table looks similar. As you can see, you have the states here and meet different symbol, perform shift to different state. And if you meet the end, then you will run the reduce action based on that grammar rule. And so the implementation state by step is also like this. We have the left to right parsing. And then the second thing is the based on the parsing table, we run the derivation from right to left instead. So this is the bottom upper approach. And look at the example code would be very, very similar to previous non recursive one. We also use a stack. And then based on this parsing table here, we run the stack. And when we meet a certain state, we will do the following action, either shift or reduce. And then get the constructor final tree. Great. We learned the textbook content. Let's start something outside of textbook. In the parser one or two, we will initially talk about pack and pack parser. So pack stands for parsing expression grammar. It was initially introduced in 2002, pack rate parsing. So the difference between pack and CFG, there are two difference. First is the rule supports regular expression impact. And the second thing, the most important thing impact is this bar. The bar means over now. That means pack no longer supports ambiguous grammar. If the first grammar success, it will not go to the second grammar. So let's take a look at the example. If for grammar one and two, CFG has no difference. CFG will say, oh, A can derive to lowercase AB and the lowercase A. But for pack, it's different. For pack, we'll try for the grammar one pack, we'll try, oh, A can derive to lowercase AB. And if it success, then I will go for it. I will not go for the second one. Or only if it failed, I will go for a second one. So grammar one and two is meaning become different now. And pack parser. Pack parser means parser generated based on pack. And pack parser can be a pack rate parser or other traditional parser with K look ahead limitation. However, mostly when we talk about pack parser, we mean pack rate parser. What is pack rate parser? Let's take a look. We need to recall one question I mentioned in the beginning. Why Gido choose from LL1 implementation to pack rate implementation? The answer will gradually shows up. First thing is the type of pack rate parser is also top down type. That means the implementation is also similar. There are only two difference in between the LL1 parser recursive documentation versus pack rate implementation. The first thing is the rule now become pack rule instead of CLG. The second difference is when we recursively parse the input string, we perform infinite look ahead instead of K look ahead. So we look at everything, every possibility. And why is allow is because we use pack rule, we no longer accept ambiguous syntax. And how do we implement that? We use an approach called memorization. So the implementation is like this. Another important fact why we shift to pack rate is because we can get super good benefit. Our grammar accept left the recursion now. It's like LL parser. And the implementation is also similar. We have a recursive way, but we address a decorator there for memorization. And then we still run the derivation based on recursive logic here. To talk about memorization, we need to slightly shift to algorithm 101. So one of the iconic problem called Fibonacci number, you can also find it in code. The implementation can be exponential tan approach by some recursive way. So in this recursive implementation, you can see we put a number in this function. And this function will call itself with a smaller number. And till the number is smaller than two, it will return the result. This approach is very slow, because when the given number n is super large, then below generated tree will be super big. And the number will, you know, the same number will be calculated all the time because you don't recreate it anywhere. So the time complexity in this case will be exponential. How to resolve this? We use an approach called LRU catch in Python, just two lines of code. It actually makes you to record the calculation result function somewhere in the memory. And then you no longer need to calculate the number you have calculated. And the time complexity in this case will from exponential tan to linear tan. And the space complexity instead will from all one to all m because you need to store all the calculation result. Right, you understand memorization now. How about left the recursion impact rate parser? Even with the memorization approach, we still need to handle this. So this approach wasn't introduced initially impact rate parsing techniques. Instead, it was developed later in the following papers. Guido's post talk about this specifically. So I will follow Guido's post to talk about his idea in C Python. And he used the second approach called reverse the cost stack. So as previous example give, we know that we need to write the parse function for each non terminal. And if we meet this case, right, the first non terminal will need to parse another same non terminal in either way recursively call itself and never stopped. And so to resolve these approaches, we need to have this magic function called Oracle parse E. So in parse E, we actually will not directly call itself. Instead, we will call this magic function. And magic function will help to fill the calculation result in the memory. And then till the end after finish all the filling, it will return data back. And then when parse E meet the same input again, it will retrieve the result from memory instead of call Oracle parse E. That is the approach. So the normal memory relation is pretty simple. If the calculated result is in memory, we return it. If we haven't calculated it, then we will just calculate it. But for left the recursion one, we will additionally do one more step. We will write the one for loop inside. And this loop will try to go through all the possibility from the current index till the end. And after we fill all the data, all the calculated result into memory, we can break. That is the way we handle left the recursion in CPython. So we have the knowledge of both traditional parser and the pack rate parser now. Let's compare the difference between them. Traditional parser and the pack rate parser both scan from left to right. And pack rate parser additionally need to run their right to left memorization. And for the left recursion handling, pack rate and LR parser support that. For ambiguous side, pack rate parser disallow that. And so they are shadowed between pack rate parser and the traditional parser. The space complexity, traditional parser is better in general because pack rate parser's space complexity depends on the code size. And traditional parser just depends on the looker has three depths. But the time complexity side, pack rate parser, it's better because it's memorize the result in the memory. So it's separate linear time, depends on the state. But traditional parser in this case will be exponential time and also depends on the k. And the capability side, because pack rate parser support the infinite looker head. So supposedly it has more potential than traditional parser. So because of that, we have several new rules in Python 3.10 and later based on pack. And you can find a corresponding talk this year presumably somewhere. So I'm not going to talk about detail here. Let's take a look at Cpycern's pack parser. And again, pack parser here means pack rate parser. Cpycern parser before 3.8, it uses LL1 parser written by Guido 30 years ago. And a parser requires a fuller step called CST. So you need to convert CST to ASD. And after Cpycern 3.9, it starts to use pack parser. So there are three main reasons why Guido choose pack. Firstly, it's infinite looker head. That means it may support more grammar in the future. And secondly, the most important thing, Guido complained multiple times in his media and posts about the horrible part of left recursion grammar. And now pack rules supports left recursion. That means it's much easier human readable. And the final benefit is there is no more CST to ASD step. So in fact, it's somehow save the memory in this case. And after Cpycern 3.10, it drops LL1 parser support. The best way to learn Cpycern parser is to play it. You can find it in tools folder. There's a thing called packer generator. And this is exact the thing we talk today, pack parser. It consumes the grammar and generates the parser. So it supports both Python-based pack rate parser and C-based pack rate parser. The input is slightly different. For Python's one, it only requires a meta grammar. But for C's one, it also needs a token because the parser is written in Python, so Python doesn't require a token for itself. And here's an example of the meta grammar. You can see there's a subheader. It will be added in the top of the generated parser file and the sum rule. And the rule contains, as you can see, a non-terminal. And the return type of that rule and the sum pack rule divider. And that means or in this case. And also the pack rule. And because this is SDT, so you need to provide some action. And the action will be written in Python code. And output. After you input this thing to the pack generator, it will generate the parser out. The parser, as you can see, this is fully generated code. And it's well written, as you probably can see in some of my previous examples is pretty similar. But it's auto generated. And it serves the purpose we mentioned before. Let's review the benefit and take a look at the performance result. The benefit of pack parser, firstly, obviously is more flexible because of the infinity to look ahead. And secondly, because hardware now is cheaper, so memory is cheaper. And we don't need to worry about the pack raise memory consumption. And finally, the CST construction step was removed. So in the newer implementation, it simplified the workflow a bit. And for the performance result from PAP 617, it mentioned the output was less than 10% difference before and after. They captured tons of popular open source program, tried to compare the speed and the memory. So overall, we can say that the benefits of pack parser is much important. And the performance is acceptable. So they decided to do this transition. Finally, we can talk about the tech away. Let's recap first. We learned what is parser in Python. We learned the parser 101, including CFG and traditional parser. And later, parser 102, including pack and the pack rate parser. And finally, in C Python, how do we implement the pack parser there and use it? So people may ask me, after this talk, what does that mean for me? And I will say, I verified my understanding. So I kind of hope that if people are interested in the same topic, you should try to get your hands dirty. And the way I did this before is to finish all kinds of parser implementation on the Lico question called basic calculator. So if you want to learn the thing in the same way, I would recommend you to implement them as I did. And if you need answer, you can check out my parser learning repository. It contains all my implementation for Lico question. And with this approach, you will not just learn it like academically read some paper only, but you will get your hands dirty really important result for real world problem. Finally, let's take a look at appendix. Here are a few related articles, including, of course, Guido's pack parsing series overview. It contains all sort of topic we talked today, such as the initiative, the penpoint of original parser and the detail implementation, including left the recursion handling of the new parser. Second important person is Brian Ford. He is the author of pack rate parsing and the pack. So if you want to read a kind of mathematic proof, then his publication would be the thing was to read. And related talks, if you want to know more about those Guido's posts, he personally gave the talk in couple of places, topic is called writing a pack parser for fun and profit. That would be the thing was to watch. And if you want to know more about PAP 617, the other two author, actually, there was interviewed in podcast that I need, their topic is the journey to replace Python's parser and what it means for the future. The content is rich and the contents all sort of detail in PAP 617. And remember, I talk about where is parser in Cpython, but I only focus on parser in today's talk. So if you are interested in that diagram in the completion step, other steps, then here are two talks I recommend to watch. First is Emily's, the AST and me. And second is Alex. So you want to write an interpreter. They both are related to the entire picture. All right. I think that's the end of today's talk. Hope you enjoy the content. Thank you so much for that, Kier. Are you available to do a live Q&A or would you prefer to do it in the Matrix chat room? Let me do it in the chat room maybe, because I didn't see any question there. Yes, I don't think we've had any so far, but they do often come in right at the end. But if your internet connection will prefer to do it via text, then of course we'll do that. Yes, let's wait for a while maybe, if anyone has any question. Yeah. Well, for me, I'd like to say thank you so much for such a comprehensive talk. And it's an area I know very little about. So I'm really glad to have seen you talk so eloquently about it. Yeah, it's also my first experience outside of Asia. So I'm really happy. Congratulations. Well, I think you knocked it out the park. So we're done. Someone's saying can you share your slides? And there will be a way for you to do that via your Python. Seeing mostly just applause in the Matrix chat room. No questions at all yet. I think above the slide, you can find it in the talk page. There is a button to download the slides and it will link to the slideshare. I already uploaded there. Yes, someone's just shared the link. Thank you.


Yeah, okay. Thank you for coming. Okay, we're gonna present dpdk collectee and cell meter stuff. It's written to the NFV and the subtitle is the missing link between my telco cloud and NFV instruction. So we're gonna present with Mariam and Emma can make to come here but and Carol's and me, Ryota, Mibu, from NEC and It's It's about the cloud. So Crowd is very nice. You can get servers and you can run on the your services on top of it without the notion of the back-end technology or the physical servers availability whatever. So it's very nice but You might notice someone eating your cloud. Your resource will may stop or disrupt it with someone else. Maybe it's the noisy neighbor or yourself or the physical failure of the servers whatever and And then you might realize that your cloud will be showing or damaged and also your service will be damaged and That don't mean it's very bad thing to you and also for your end users So we need more enhancement or your ability to monitor or see your cloud what to identify what's happening and In the most telco case as you may know there's a small and medium size of the deployment and we'll be connected in one big place with large scale cloud so You can you may have the hundreds or hundreds of thousand infrastructure systems including servers or the modern services as well and For the future and IOTs you may have more devices and client and increase the subs demand and also the network traffic and in these situations you still may be asked to remain the same level of service quality and Expectation from the user might be the same so you have to make your service available when whenever and wherever the Service or the Service type or the network was grossed so and the monitoring plan is is very important for us to Identify the malfunction of your cloud and overall system performance and So we think it's very important for us to Expanding amount of data availability And also the improving alarming functions in our cloud So we need telemetry telemetry is a cornerstone for billing benchmarking intelligent orchestration and for management So this is very important place for all of us So let's Pass to the Mariam sorry cameras to use case and more technical details So as we are the mention now everything is on the cloud and there is IOT coming to the cloud There is a lot of devices connected a lot of broadband everything and we need telemetry to Be able to charge properly our customers do some benchmarking and do some intelligence orchestration and fault management So in this session we are going to focus a bit more on fault management as one use case and specific to in telcos so one use case in telcos is to Deploy the services in an active standby manner, so we can have One VNF It will come later, but in an open-sock deployment. We have the controller nodes. We have the many compute nodes And also networking nodes and all that And what we want to do is to monitor the network So and in this case we can monitor The the link of the network so physical resources and also virtual resources using dpdk and that's the example that we are going to to give here and and we will Collect quite a lot of metrics out of samples and push it to the to the cloud to the to the controller and based on that We can do some intelligence decisions We can have performance metrics we can we can we can detect Failures happening on the network side, etc so in this case we will have The control node and two compute nodes. We have cilometer We have collect the collect these a monitoring system and also obvious with dpdk So obviously with the dpdk will be monitoring the the the links It will report to collect the collect the will push all of these these these metrics to cilometer and then later If there is something wrong, for instance, we can notify the user users through a AODH as you want to call it so At first we have everything the call deployed everything is is ready and and running so the link status is Okay, thumbs up on that Perfect So again in the telco we have Many cases where we have a Service virtual network function DNF deployed in one compute node and the other one in Sun by mode in another compute node And so at first everything is working But if we detect something On the physical or virtual virtual infrastructure that is going wrong some Someone just was stupid enough to plug out the cable without doing some Some actions before Then it's important to detect this So it's a big problem. Okay What should we do in this case in the telcos? we switch from the active VNF to the standby one and The problem is It's not easy to to to detect this with obvious TPTK With a collect the and now with the alarming functionality in in open stack That's a bit more convenience If we simplify that and we will show a couple of blueprints in in open stack in different projects where we were able to monitor do some orchestration Alarm the the user and the user can then Quickly react to this event So I will pass to Mariam To explain TPTK stuff collect the and all that Hi folks, my name is Mariam to Han. I'm a network software engineer at Intel And I'm also the project tech lead for software fast path service quality metrics in opnfe By the way, I'm only going to say that once so it's going to be SFQM for the rest of the presentation For those of you who aren't Aware of what DPDK is it's the data plane development kit and that is an open source project That provides the utilities libraries and drivers to enable fast packet processing in user space And as Carlos mentioned collectee is a system statistics collection demon today I'm going to give you a quick overview of SFQM We're going to look at some of the features that we've been implementing in DPDK and collectee and how you can use those features to pull statistics from DPDK and relay them all the way back to So let's dive straight into it The ability to measure and enforce telco KPIs in the data plane will be mandatory for any telco grade NFEI Implementation to meet this requirement SFQM has been developing the utilities and libraries in DPDK for three things Firstly the ability to measure telco traffic and performance KPIs such as packet delay packet delay variation and packed loss Secondly the ability to monitor the performance and the status of your DPDK interfaces and thirdly The ability to detect and report violations that can be consumed by higher level fault management systems Today's talk focuses on the monitoring and the performance Monitoring the performance and status of the DPDK interfaces So what are the SFQM features that are available to allow you to do that? Well, actually the features fall into two categories firstly DPDK features and secondly collectee features on the DPDK side We implemented an extended statistics API. This was a predefined API in DPDK We simply implemented it for the drivers that were available the 1 gig the 10 gig and the 40 gig Nick drivers and What this extended stats API does is that it augments the generic statistics API with the generic stats API today You get the aggregated stats right across your lower level registers So if you're looking at your error registers in the through the generic stats API You're not necessarily aware of which of the underlying error registers actually causing your aggregate count to increment And this is where the extended stats API comes in it allows you to look at the hardware level statistics registers essentially On your Nick and expose that information all the way back up to the DPDK application So now you can see exactly what the cause of the errors is is it undersized packets coming in your system? Is it CRC errors and you get a much more detailed view about what's going on there on? the collectee side we implemented two plugins firstly read plugin called DPDK stats which pulls the statistics and link status from from DPDK using the extended Nick statistics API for the stats and generic link status functionality from DPDK for link status and The second plugin we developed was a salameter plugin and this salameter plugin is capable of relaying any of the read Statistics that are pulled by collectee off your system to salameter So it looks after the transformation into salameter samples and post them to salameter. So How do these actually work together? Well, the idea is that you would have something some DPDK application Running on your compute node in the case of our example. We have OBS with DPDK It has a VM attached to it and it's doing some sort of packet processing on that compute node collectee then runs as a service on that compute node and collectee is usually configured to run a particular interval to pull statistics off the system so When the interval triggers the collectee process initiates a read on all enabled read plugins in this example DPDK status enabled it goes off it retrieves the statistics from OBS with DPDK And it also retrieves the link status for any of the interfaces that are bound to DPDK DPDK stat then dispatches those values to the main collectee process Which then pushes those values to all enabled write plugins in the case of our example the salameter plugin The salameter plugin then transforms those samples into salameter samples and post those values to salameter So let's dive into the details of the two plugins and how they little bit and how they work. I Apologize about the coloring. I should have gone for something a little bit more visible, but nonetheless, I'll walk through it How the DPDK staff plugin works as I mentioned, it's an input plug-in so it reads statistics off your system But what we had to do was actually fork off DPDK secondary process and that's the DPDK helper process you can see on the diagram there Why we need to fork off this process is Because we don't want to have to restart the daemon whenever your DPDK application, which is usually your primary process dies now the difference between a DPDK primary process and a secondary process Is that a primary process can initialize the shared memory and the resources for DPDK? Whereas the secondary process simply attaches to that primary process shared memory can't do any initialization But There is no process. There's no functionality within DPDK at the moment that allows you to release all of the resources once your To release all of the resources as a function call to the DPDK API so for that reason we have to kill The DPDK helper process in the background when a primary process dies So allow me to just step through it step by step When your DPDK application starts your DPDK primary process it actually initializes a configuration file It holds a right lock on this configuration file when your DPDK secondary process starts It actually holds a read lock on this configuration file So when your primary process dies In order to be able to restart another primary process on your compute node You need your help your secondary process to release that read lock that it still holds on the configuration file So as I mentioned, there's no graceful way to allow us to release this from the DPDK side so what we do is we Kill the process we let the kernel look that look after the cleanup and releasing that lock for us in the background and now Voila a new DPDK primary process can start up on your system and you don't have to kill the collectee service because this the secondary helper process was already killed and you can just spawn off a new one to go off and retrieve the stats for You again, so you might be wondering How do we actually share this the statistics between the DPDK helper process and DPDK stat So the answer is actually through a POSIX shared memory object When DPDK stat is initialized it initializes a POSIX shared memory object and it also initializes two semaphores It looks for a DPDK primary process and once one is detected to have been kicked off or started up So you have some sort of DPDK application running on your platform It forks off DPDK the DPDK helper process now the DPDK helper helper process Mem maps that shared memory and it's capable of writing to that shared memory But it only writes to that shared memory when DPDK stat kicks a semaphore Indicating that hey, you need to go read some stats for me the DPDK helper process reads the statistics into shared memory and it also retrieves the link status and Then it kicks the second semaphore Indicating to the DPDK stat plugin as hard to the DPDK stat process that okay now I have the stats I have link status you can go and dispatch these values to the main collectee process so now we have Retrieved statistics at this point from DPDK as far as the main collectee process So we want to write these these samples that we've collected to Cilometer so what we did was we implemented a Cilometer plugin that takes advantage of two things firstly Cilometers restful API that allows you to add custom meters custom meters to Cilometer and Secondly the Python plugin the Python bindings that are available to you within collectee so now the when when collectee starts and the Cilometer plugin is enabled it registers a right Callback with the main collectee process Once the main collectee process has retrieved all the relevant samples from the read plugins it Kicks or I suppose it calls all of the callbacks In this case the collectee Cilometer plugin callback for writing and as part of that process the plugin transforms the samples into Cilometer samples Edit does a HTTP post directly to the Cilometer API So now we have passed Link status and statistics low-level statistics all the way from our physical nick to Cilometer at this point Maybe not this point shortly. I would hand you over to Carlos who's gonna talk about what happens once those stats are passed on so for us There are some limitations to the plugins that we've enabled today So yeah, we have a bit of work to do on the link status side What we would like to do is take advantage of the notification plug in architecture and collectee To simply post an event when link status goes down Rather than passing it as a generic statistic to Cilometer today And on top of that then adding an alarm separately. So we'd like to be able to post notifications directly to AODH the other thing we'd like to do is Relevant to feature we call dpdk keep alive and Dpdk keep alive is a heartbeat mechanism for packet processing cores in dpdk. It protects against stalls or sorry, it detects stalls or application threat failure within your dpdk application So we'd like to be able to expose the core liveliness or the core deadliness in this case all the way to AODH. So again, we'd like to take advantage of the notification Plugin architecture in collectee to allow us to do that Some things you'd like to do in the future are performance scalability and aggregation analysis For what we have we'd like to do some knocky integration as well and in the future even Look at developing a plugin for open vSwitch With or without dpdk. So hopefully it will be an agnostic open vSwitch plugin that is capable of retrieving not just port stats, but flow level statistics in the future and this is the end of my presentation So we are now going to talk about doctor it's an open fee project and later on we will show a quick demo of How these two projects SFQM and doctor are collaborating and how we are learning from each other and fitting back each other So in the beginning of the presentation We said that telemetry is important. It can be used for benchmarking billing and also fault management and doctor is addressing these these these other case fault management, so It's fault management and maintenance framework We are working in open FV as part of the telco space and the main For actions that we do is identifying requirements so we work with operators and also vendors to understand what are the Major requirements that we have to address for fault management and maintenance Also, then we do some gap analysis. So we study open stock the different Projects in open stock. We also do gap analysis on some other projects outside of open stock And then based on these gap analysis We identify some something some some actions that are needed to be implemented some some some work So we implement and we push everything upstream so to open stock To the BDK to collect the so everything is open source and and upstream first and Later on we as part of open FV projects. We take all these projects With our code base we integrate and we test So that in the end we have an open FV release we can you can just download install and you have everything Installed integrated tested everything will work. Hopefully The four key requirements for doctor is consistent resource data awareness This one is about having the the exact states of our cloud So that when there is some some failure happening in our infrastructure, we are aware of these these states This is important because So far there are some some some Components in open stock that do not Detect some some failure so that they they are saying that their state is up Where they are not because they're the thing that there is something causing a malfunction there The other one and very important is immediate notification so as part of our gap analysis we detected that Solometer was not able to quickly notify the the user and here the user can be the cloud operator the code administrator can be In Etsy terminologies can be the VNF manager can be the NFV orchestrator or anything sitting on top of open stack So we are talking about toco But it can be used for anything enterprise whatever And with our with this item immediate notification that we have also addressed We could notify The the user within one second whereas before it was a couple of minutes, but I will talk about that later on the third Key requirement is extensible monitoring. So we understood from talking with different people that there are different people using different monitors monitoring systems and Many of the time they have even not just one single monitoring Platform running matching the the the visual and physical infrastructure, but also two three four five ten whatever So it is important to have these This platform this framework that can support multiple monitor solutions and also Create this self-bound interface so that we have a consistent and And standardized API there The last one is fault correlation is also very important so that when we have a lot of alarms coming a lot of triggering We have to be able to detect and to know from where what was the root cause of these failures so Sometimes there is one link that fails in the network sites and there is a lot of more links failing because of that One is specific. So if we get a thousands of alarms because of network failures event, we want to narrow down to the exact problem. What was the root cause? so we do root cause analysis and also we have a programmable dynamic framework that can accept policies and so that if there is an event We can immediately call enforce a policy So These four key Requirements can map to the doctor Framework architecture on the right-hand side. So we have the controller part Which are the normal open-stock services that we know Nova Neutron Cinder, etc. We also have the notifier that is responsible for notifying the the user and that's in open-stock Iodh It's part or it was part of the salameter of the alarming functionality the monitor and the inspector a Use case is this one. So we have a an event in our NFV infrastructure Let's assume it's a OBS the PDK interface That that there is some some problem with that we can notify we can detect we can notify and so we pass We can notify through the monitor the inspector of this failure Based on policies we can Find what are the affected? Resources and based on that we can also update the their state so if For because of a network failure We cannot access any any longer a compute node for instance then all the virtual machines running on that compute node Have to be marked as in an error state for instance So we do that in sub three and in sub four When there is this event The the user is notified that Is or her virtual machines are now in an error state and this is an immediate action an immediate notification now so mapping these four Generic Platform agnostic let's say Components to the open cycle ecosystem. We can see that in the controller side We can be talking about Nova Neutron and Cinder on the notifier a ODH Inspector we can have a couple of them depends. We are studying so we can have Congress We can have vitreyes. We can we can have monaska on the monitor. We can have a couple of them as well Zavik's collect the Nagios or we have plenty of of them available On the open source world and also proprietary ones our work in the doctor projects for the Liberty and Mitaka release Was mostly focused on the on the first two key Requirements consists resource state awareness and immediate notification Two important blueprints that we proposed and were accepted and merged in Liberty and the doctor releases Were these two ones the event alarm and the state correction? So for the event alarm We During our gap analysis, we noticed that It can take seconds to minutes to be alert of A an event something that is wrong or we exceeded a threshold or there is a server down things like this Why because there were two pulling mechanisms in place And if we are out of sync it's even worse so it they can take couple of minutes So what we proposed was a events driven alarming functionality where Components like Nova for instance could immediately notify send this event to the message bus and adh is listening and Taking this message and immediately alerting the user. So We went from Several seconds to minutes to within one second and that was a really important requirements for the operators Because of the app if standby switch fail over fail over so we don't want our service to be done for couple of seconds or minutes The other second blueprints we implemented Was in Nova the state correction or you must probably you you have heard of the Nova Marcos down API In this case if we have a an external monitoring solution monitoring the the compute host We can Probably detect a lot faster or with a lot more flexibility and we can monitor more things Then just the Nova compute service itself and Is so if we detect something is wrong We can now through the this new API mark these compute nodes as down and that will trigger an Event to the message bus adh can take it and if the user configures can Get these alarming From the project creation to the Brahma puter who is released in March 2016 We have worked on adh Nova and we are also working on a couple of more projects Namely neutrons in there and also Congress with rage We have we have also integrated Zabix and the PDK in the open FV platform And the important part here to mention is that in between March 2015 and September 2015 we worked upstream in OpenStack To by proposing these blueprints and also have them accepted and implemented and we managed to do that and Later on after September 2015 to March It was integration phase testing and and all that and just waiting for open FV to be released to release the next version These are three blueprints implemented in liberty Important here to mention Is that we are a Projects that consists of multiple companies vendors and operators So we welcome everyone to join this case and you see Intel and Nokia Participated in this effort from open FV Brahma puter release and the Liberty and Mitaka Cycles to the next one We extended our focus of contributions We are now also working on the monitoring parts and the inspector parts So that's where the PDK called the And so long there comes in in the monitoring part. We also had some two other presentations on the inspector during this open OpenStack summit and as so Yeah, Zabix collect the Nagios we can support all of these Congress with rage etc And updated table so far of implemented completed the blueprints are these ones as you can see again Multiple companies jumping in and helping each other and Putting everything together as FQM plus doctor. We can see that or we can get a Very good view of our platform in terms of Metrics and the current state of the network and Based on that we can modify the user in case of any events affect affecting their Their resource virtual machine storage networking So we have a quick demo just the video Where we will show a Link status check so the PDK is checking the The PDK interfaces we have the folded action propagation. So from the PDK to collect the solometer the inspector component We will update the Innova compute service because we are saying okay this interface is is is done. So there is no connectivity There is no no needs to have these compute nodes In this in the schedule in the scheduler. So let's just mark it as down and update the the current state of all the virtual machines running on that compute node and alerting the user So that's resource it correction alarm and then later on Do the IP stand by service switching? So we will start It's pretty small So in this window We are just preparing our platform. So we are Booting some virtual machines in compute node one and compute node two in compute node one We have the active stand the active VNF running. It's just an HTTP server and on compute node two. We have the the second VNF so if I skip it Yeah, just starting On the top left side We have we are monitoring we are getting these these the current state of the interface link from a selector The the its value is one. So it's good On the next one we have the current state of the Nova compute nodes So compute nodes one is up and On the second on the third one. We have two virtual machines server one in compute node two one and Server two in compute node two and they are all active on the top rights and sides We have just a client Refreshing the page is being served by being served by server one. So everything is okay And on the other Window the application manager blog. It's the the consumer that the operator. So you will get notified of events And then do the switch over So there was the event Coming so basically here dpdk detected a link down So it reported through collect the two cylinder. So it's only now is zero the the current state of the compute node is now down and The virtual machine is an error state. So based on these Events we notified the user and he switched to the To the standby node Sorry, just to summarize. I wanted to share with you a bit of an after analogy About the work that we're trying to do really so trying to manage a complex cloud solution without proper telemetry And without proper telemetry infrastructure in place is like trying to walk across a busy highway with blind eyes and deaf ears You have little to no idea of where the issues can come from and no chance to take any smart move without getting in trouble so what we're trying to do by combining the solution between dpdk collect the open stack and doctor is Is essentially painting the pedestrian crossing so putting the building blocks in place to allow us to Not cross the highway blinded that So I really encourage you folks to come and join us in opnfb in the doctor project and SFQM if you're interested In contributing there. Thank you very much for your time


It is a very big challenge because we can talk about landscape in terms of a physical landscape and in terms of crops and forests and livestock and so on. But there's also people's landscape. There are settlements and communities are not homogeneous. Communities are unequal. They are divided by class, by rich and poor, by gender. And those are the communities who will then make the landscape approach a success or not. So although we talk about community involvement, the question is how do we ensure that the voices of the most disadvantaged who have the most stake in local landscapes in fact get heard and influence the dialogue process. I think there are many aspects. One is people being, you know, increasing solidarity among a local community of the disadvantaged. Now we find many examples of that happening. So for instance, self-help groups in South Asia and there's also equivalent groups in Africa. We have small groups who come together for credit and sometimes for local enterprises. We are not having a discussion necessarily with them about a more integrated approach. These are the groups through which you have greater voices of the poor being heard. And these are the groups with whom we provide the foundation in some senses for taking this forward. Now this dialogue has to take place between communities with the poorest being empowered, with policymakers, and also with academia, with scientists. And I think it is possible if we recognize that we shouldn't just talk to a few but we should empower the many to speak for themselves. That's right and recognizing that people don't have equal access today. So if there are institutions on the ground which have been built over the last 10, 15 years which enable greater voice of the disadvantaged, they could provide another basis, an integrated basis to have a discussion and see how they relate to this approach strategically and how you take it forward. Well they do have increasing negotiating power as a result of this within communities. It's because there was a recognition that they don't have voice that we begin to think about an idea of groups rather than individuals alone. And now we have to take that idea further by saying people are responsible not just for what they do but how it affects others. So for instance what you do in your own field as a farmer affects others. If you draw too much of groundwater it's going to lead to less drinking water and less water in your neighbor's fields. And that requires people to have a discussion saying look what you're doing is affecting me and how can we move forward in a more cooperative spirit. By that I mean that the term landscape seems to be politically neutral. A political economy approach will mean that you recognize that there are serious inequalities within communities. There are class inequalities, there are gender inequalities, there could be other forms of inequalities based on ethnicity. And these inequalities are not just social, they're also economic in terms of the distribution of resources and access to resources. So if you're talking about landscapes as resources then we also have to recognize these inequalities and see how both in terms of their voice and their choice and the access to resources increases. And if you match the two then you'll actually come up with a much more realistic possibilities of sustainable landscape integrated approaches. We need to recognize that we can't just eulogize traditional systems. We need to enable traditional systems to be such that they also have enabled people to have better livelihoods, higher incomes and all of that. And that requires a scientific, if you like, a two-way system where what farmers need and what they provide in terms of innovations but also how the scientists then take it forward one more step and bring it back to the farmers.


Okay, we're back. This is Dave Vellante of Wikibon.org. I'm here with Jeff Frick, my colleague and co-host for the week, Jeff. It's been an excellent week. This is day three. This is theCUBE, Silicon Angles, and Wikibon's coverage of ServiceNow Knowledge. It's been a really eye-opening event. We're here in Vegas. We've been here all week. What did we learn this week? So we learned that IT has a toothache and ServiceNow has a remedy. And we heard from a lot of practitioners about how ServiceNow is helping automate IT, run IT like a business. We heard some challenges in terms of how people actually quantify the business case and things of that nature, but probably the best one that we heard is, look, just talk to your CFO about going from a CAPEX to a variable expense. That's going to probably be the most useful. The other thing that we heard is generally, they're very consistent. People are starting with incident management, change management, problem management, and then moving out to other parts of the IT organization, developing new applications. And we heard about the importance of starting with a CMDB, a configuration management database, a single source of record. And while that's best practice, it's not always the easiest from an organizational standpoint. For you to succeed organizationally in doing that, you're going to have to bring together application heads, business process heads, the various, you know, department PMO folks, IT infrastructure people, and the like, and the best chance you're going to have of affecting that change is to really do it from a top down. You're going to have to have top management support. The other thing we heard in terms of advice was, start small, start with incident, start with change, and problem management, and grow out from there. We saw some innovations this week from ServiceNow. We saw some mobile apps. We saw also some announcements, specifically around what's called app creator, the ability to really simplify the development of applications. We saw ServiceNow as both, we're seeing them as both a SaaS player for IT service management and increasingly becoming a platform as a service on which to build new applications. And Jeff, I got to say, I was impressed with the quality of the customers, with the quality of the management. Fred Luddy is a rock star. He's a super alpha geek visionary. Frank Slutman, you guys who follow this show know we always love Frank Slutman. The guy really knows how to grow companies and manage companies and inspire people. And as well, the people throughout the organization have done certainly a fantastic job with this event. Great job obviously supporting this Cube. We really appreciate that support and the great guests that we had on. And overall, I'd say home run for ServiceNow. Yeah, it was really a great event. And it's fun to get the executives on and Fred and Frank were terrific, but it's even more fun. And I think one of the special things, opportunities for the Cube is for people to hear from the practitioners. You know, let's hear the real story. And I think we did a great job of not only getting the story from them, which I thought was the best part really is that they summed it up, getting a seat at the business table. You know, really being participants in the business, growing the business, changing the business through transformative IT. I thought that was terrific. And really hearing the individual stories of really disparate set of businesses, disparate set of business problems, but how do they attack the problem? Again, as you said, very consistently start small. They all got in. They're all here looking for ways to expand the platform.  And clearly it sounds like Fred did a really phenomenal job in that basic platform structure before he built the application. That's enabling them to really go. That said, we heard time and time and time again, but these guys have a laser focus on IT and helping IT do their job better. So just because somebody from an adjacent department wants to come in and get a little assist, these guys are really laser focused on giving IT the tools to get that seat at the table to be business partners with the rest of the business and no longer just worrying about provisioning things and shipping things and counting things. Another thing that came through time and time again was the CMDB that you talked about enabled people to really look at their business with a degree of analytics and cost and information that then they could go to the management and really have hard numbers for where things were and how much things cost and where resources are allocated. And we heard that time and time and time again. So in terms of kind of being a data-driven organization and a data-driven business decision process, it sounds like this tool is really enabling that to happen. Yeah, and I think we also heard a lot about self-service. We heard Petra from KPN, CIO along with Martin tell us about how they've achieved that self-service capability. These are capabilities that you commonly don't associate with IT, I mean we associate with Amazon but you don't necessarily associate with internal IT. The impressive thing is that the attendees at this event who are service now customers, their attitude is phenomenal. They seem to, this is like a reawakening. I mean IT people are, generally speaking, incredibly smart, talented, really hardworking. They've got some technical background obviously and they're very customer-driven but they've been up over the years. They've been told that IT is a cost center. You guys spend too much. You're not responding fast enough, blah, blah, blah, blah. And it's been like a reawakening of renaissance of the IT professionals. So the people here are very excited about that. That's why when Fred Lutty walks through the party, he's surrounded by people, shaking his hand, slapping him on the back, thanking him. And I guess my observation is that while incident management and change management and problem management, all this boring but important stuff is the tip of the spear, there's some really exciting developments going on outside of that core IT operation. Yeah and as we heard from Doug from Sequoia, Fred was smart enough to know where he was strong and where he wasn't strong and he really seems to have a symbol to rock star team around him, both on the business side as well as the technical side, the services side, to build this company out. They've, you know, I just love the whole thing having spent those time at OpenStack and AWS and Shadow IT and people doing their own thing and how they're really arming the internal IT people to combat that. But even like with the new cloud provisioning application to bring what was happening outside of the control of IT back into the visibility in IT and embracing that component of an option for their customers to get stuff done. So let's break it down a little bit. Let's play a little analysis of the things that I think that observers should be watching in terms of the progress of service.  We had Mike Scarpeleon, CFO. And so, you know, of course watch the financials. I think, you know, they're going to meet or beat those expectations, you know, this quarter we'll see, you know, going forward there's a lot of things that might be outside their control, but let's watch the degree to which they can protect their turf, the IT service management turf, both protect it and continue to knock down the BMCs of the world, the HPs of the world, the legacy vendors that really from all accounts, everybody we're talking to, this platform is superior. So let's watch that. The operations management is very interesting. The whole cloud ops, we've had, we had service mesh on a couple of times in the last few shows, they're sort of going after that same space. This is a new territory for service now. They're in a way, certainly in the cat bird seat with their 1600 customers, but there's a lot of competition. A lot of people want to be that sort of cloud broker, that heterogeneous, you know, agnostic cloud broker. So let's see the degree to which they can leverage the tip of the spear and span out from there. And then also the business, whole business process management, the compliance, and also, you know, as a part of that, the metrics management, the business value piece. That's kind of a new layer. We saw some of the folks within the ecosystem that are actually building, you know, layers on top of service now, building applications to actually affect that. So that's the other thing I want to watch for, how well that service now can cultivate and grow its ecosystem. I think that's a critical metric and an observation point milestone, series of milestones for this company. And then finally, the whole notion of service now as a past player, as a platform, as a service player, the degree to which they can successfully get their customers to build new applications, and particularly mobile applications. We heard that theme that mobile is key in a way. I think service now is a little late to that mobile game, so now they got, I think they got to move fast. And I think if they do, they're going to reap a lot of rewards there. Jeff, what's your analysis and takeaways from these two and a half days? Yeah, it's just, it's a phenomenal story. It's really a fun story because of the personality of Fran, I think. And again, now there's new challenges as a public company. How are they going to grow? I like the ecosystem. I think when you talked about it, I like to watch the ecosystem and how that grows. I think that's a fundamental part of building a company today, both to get within your own culture, but also to get the extended culture and partners to build on. I think that's going to be good. Curious to see how they handle the growth. This is meteoric growth inside the company for the people and clearly the culture is a really big thing. How are they going to manage that? How are they going to keep the spirit that they've got with these folks and manage that growth? Those are probably the biggest. And then as you said, and I think we explored it a lot today with some of the customers is, what is the real ROI? Because we talked to the guy last night, he would like to replace a legacy system with some of this newer stuff, but the thing is old, paid for and depreciated. And so his ROI on keeping that system as it's defined right now is infinite, at least until the one or two guys that know how to operate it get hit by the bus. So I think, as you said, being able to better quantify the soft benefits and the ROI to expand the footprint from their basic entry point and grow inside their existing customers. All right, Jeff, well, listen, once again, it's been a pleasure. This is our second show together.  It was fun, thank you, Dave. Looking forward to do more events with you guys. I want to thank the crew, Alex, Andrew, Anthony, guys did a great job this week. Really appreciate all your support, Kenny, who had to split to meet Mark Hopkins and do Google IO. The whole team that was down doing SAP Sapphire now, you know, great job. We'd love to do multiple events, theCUBE. We try to go to the events, bring you the best guests that we can find. Also thanks to Kristen Nicole, who runs the news desk and the editorial and Stu, appreciate your help back home as well, tweeting out. And so also thank you for watching. You're always a great audience. We appreciate the tweets. We appreciate the interaction. Go to siliconangle.com. You'll see all the news. You'll see the news from this event and other events. You'll see write-ups of the interviews that we did. Go to youtube.com slash siliconangle. And you will see all of our videos, all the playlists, and check out wikibon.org, free research, open source, peers helping peers, no paywalls, no firewalls. Really appreciate all your help and all your attention. Thanks for watching everybody. This is theCUBE. This is Dave Vellante with Jeff Frick. We owe one quick word. So we got a number of events upcoming. We've got an O'Reilly media event. Fluent is coming up in a week or so. We'll also be at IBM Edge. We'll be at HP Discover. That's in early June. So watch for those events. We'll be back out here in Las Vegas yet again. So thanks for watching everybody. This is Dave Vellante with Jeff Frick. Signing off for now. We'll see you next time. Thanks everybody.


Hello to you. A Workaround is a method for overcoming a problem or limitation in a program or system. For example, my background was made using the Visualize app. The problem is that Visualize only makes images that are tall. The Workaround was to rotate text 90 degrees, then save to Photos. In the Photos app the image was rotated and saved before being imported into Telogami, the app that made me. Telogami only supports one background per project. There's a Workaround to use multiple backgrounds or scenes. That is to export a series of videos and piece them together using an app like iMovie or Splice. When you come across a problem or limitation, go ahead and see if you can overcome it with a Workaround.


Today guys, I bring you a new world video showcasing all known Legendary Spears in the game. I will mention a lot of these haven't been obtained yet so a couple of these we don't actually know where they drop. Also some of the images you will see are placeholder images so the weapon may look different. But in terms of the weapon stats and what they offer, it's all here. So let's go. How's it going guys? My name's DPJ and if you enjoyed the video leaving a like really helps out and if you like what you see and want to see more, be sure to subscribe. Okay so within this video I will showcase to you all known Legendary Spears in the game and how they are obtained. Now like I said some of the images are placeholder images and also I think I added this list here, there's three that haven't been found yet, I'm hoping you guys can help me out with those. So let's get into it. Firstly, we have the Spear of the Outer Isles. So this Legendary is obtained from the Blighted Gatekeeper who is a dungeon boss within the Garden of Genesis Expedition. So you'll want this Spear of the Outer Isles, this is where you get it from. Next up guys we have the Sub-Zero Spear. So this Legendary drops from three bosses within the Garden of Genesis Expedition. The bosses are Taxidius, Volde and Farkas. So you want the Sub-Zero Spear, that's where you get it. Next up guys we have the Angry Earth Exterminator. So this is a Legendary that drops from the Endgame Expeditions of the Lazarus Instrumentality, the Depths and the Garden of Genesis. So if you want a chance again this amazing sanding Angry Earth Exterminator, you have to enter into those Endgame Expeditions. Next up guys we have the Chaos Spear. So this Legendary is a world drop across the board. Obviously I feel you have to be a certain level before we start seeing these Legendaries drop, I think that could be around that level of 40 but don't quote me on that but it's definitely between 40 and 50, that's when you'll see a lot of these world drop Legendaries starting to appear for you. So yes the Chaos Spear is a world drop. Next up guys we have the Violent Descent. This is a Legendary you can craft. Obviously using the right materials and levels in engineering are required. So it requires a blacksmith tier of a level 4. You need to be an engineering level of 141. Trade Skill XP is around 3750 and Territory Standing is to be 100. Next up guys we have the Soul Forged Spear. This is a Legendary you can craft. Again with the right materials. It requires a station of a blacksmith tier 5. Requires an engineering level of 200. Trade Skill XP needs to be 7,440 and Territory Standing of 144. Next up guys we have the Floralus Grandmaster Spear. This is a Legendary you can also craft with the right materials. Trade Skill needs to be engineering. Requires a station of a blacksmith tier 5. Requires an engineering level of 186. Trade Skill XP needs to be 7,285 and Territory Standing 141. The next three weapons are unknown at this time. We don't know where they are obtained. They are the Traveling Storm, the Conqueror Spear and the Spear of the Corrupted Assault. Now there's a few clues of where these could be obtained from but nothing concrete yet. So if you've had any of these dropped please let me know and I'll edit the video. Change up the video description so others do know. But there we have it guys. All known Legendary Spears in the game so far. If you guys enjoyed the video leaving a like really helps out. If you're new around here and want to see more New World be sure to subscribe and if you never want to miss a video or upload you can turn notifications on by hitting that bell button. But guys thanks as always for stopping by and hopefully I will see you on that next one.


We're here at the ITU studio in Geneva and I'm very pleased to be joined by Ms Sylvia Pole who is deputy permanent representative of the permanent mission of Costa Rica to the United Nations in Geneva and she's also vice chairman of the Council Working Group for Child Online Protection. Ms. Pole, thank you very much indeed for being with us today. Thank you very much for inviting me, it's a big honour. Child Online Protection has received overwhelming support globally and in Costa Rica. President Lara Chinchilla has joined hands with ITU as patron of this initiative. Why is it so important for Costa Rica? This is a very important initiative for Costa Rica. Since 2010 the president was invited to be the patron of this worldwide initiative. She felt first very honoured but also felt a very big responsibility. Costa Rica has always been a country which has placed a lot of importance in issues related to human rights, to peace, to security and to social issues. And one very important project for her is to give accessibility to telecommunications to all the population of Costa Rica and that includes also children. She always has thought that the importance of having accessibility to telecommunications to ICTs will bring a lot of opportunities to our population, to the country and she has placed a lot of efforts on that. But at the same time she also recognised that in the case of children there's also a lot of challenges related to the problems that can come up of the use of internet and all the things related to children online. And for her first at a national level she made a big effort to establish a commission, it was a commission where the government with civil society, with the private sector have been working together to try to push these issues but from an angle related to education to creating awareness about the problems that can come up with the use and the dangers that are in the internet for children. So this she has also tried to put as an example for the rest of the world and she has tried to work diligently with Dr. Tré and ITU to try to create awareness and promote the importance of on one side all the opportunities that the online world can give to young people but on the other side also be aware of the problems and the dangers for children. Now your vice chairman of the council working group here at the ITU at an international level what steps have been taken by the council to strengthen child online protection? Well the latest steps have been very important since last year we re-established the COOP partners group and it's a group of a lot of members from governments, from civil society, from private sector. We had last year a big meeting during the WISIS forum and this year we had another meeting during the WISIS forum which was even more successful and that is a big positive science about the interest in the issues. And this has given a platform for a lot of actors and stakeholders related to the COOP to be able to exchange experiences and establish guidelines where we want to go with the COOP initiative and this is one of the big projects that have been really working really well. And finally Costa Rica is hosting the Global Youth Summit in September. What do you hope will be achieved by the summit? Well we're first of all very honored that we're hosting the Global Youth Summit. It's the first ITU Global Youth Summit and the premise to this event is to try to get over 500 youths to come to Costa Rica and discuss around ICTs the big issues related to what is important to them in relation to jobs, in relation to the environment, in relation to safety online, in relation to health and also accessibility. And after three days we want to have a very dynamic event. We want to break a little bit about from the traditional way of doing meetings because we will be surrounded by young people who want to have, probably do things a little different so we will have workshops, we will even do some off-site activities and not everything will be in the hotel where the summit will be held. And at the end what we want to have is very specific outcomes. We want to have for example a permanent exhibition about the COP in Costa Rica that will hopefully stay in Costa Rica. And we will also want to have a final document about the biggest issues and the discussions related to all these young people that the President of Costa Rica will be taking to the General Assembly only two weeks later in New York and presenting it to the whole UN system and telling them what the young people. And it's also very oriented to the post-millennium development goals, the post-2015 what they want beyond that, what they are going to be the future leaders, the people who will be leading our countries, our businesses, our NGOs and we want to know what they want and how they see the world but always at the core ICTs and how ICTs can serve as a tool to get to discuss other issues related to the post-MDGs. Silvipoel, thank you very much for being with us today. Thank you very much.