All right great. Thanks for coming and I hope you all can hear me. Luckily we've got this wonderful mic My name is Lee Shang. I'll be talking about code doffing with Python So a little bit about the company I work for it's Yelp We connect people with great local businesses So we have an app and a mobile website to help you find interesting local businesses to work with and a little bit About myself is I'm an engineer at Yelp building distributed systems primarily with Kafka And previously I've been at Amazon a startup called we be data and Dropbox and I intern at Starfleet a little bit That's a data scientist. You'll get the pun in a moment if you think about it So a little few caveats. I'm not actually a golf player. So my analogies may fall a little bit flat I'm not talking about traditional code doffing in the way of Maximizing every single character that you use to produce a particular result I find that that tends to generate unreadable code the way that I'm going to try to express code doffing is a way That generates more readable code while being more concise And finally because we're talking about code. I use a lot of monospace types. So if that hurts your eyes I'm sorry. That's just kind of what we're dealing with in this presentation So for me, I'm defining code doffing as minimizing the number of strokes in each block of code And I'll define what a stroke in terms of what I'm saying is later and my thesis is that Concise code requires less cognitive load to understand because we spend a lot more time reading code than we actually do writing it And so spending a little bit of time to be more concise Enables the people that look at the code later to and especially yourselves when you look at your own code to understand What it is and I find that a great exercise when you're looking at concise code is to say can I read this out loud using? using English or whatever my native language is very simply And oops and finally like to quote William Shakespeare Right like gravity is the soul of wit to kind of like continue along the lines of shorter things is better And I sound a bit like a snake oil salesman and what I'm trying to say is when you shorten your code you'll increase your clarity and To of course because this is snake oil. I have harm no pythons in the making of this talk So what are strokes? I kind of define them as plus one for any keywords variable names and operators so I want to promote the idea of using longer variable names rather than single character ones and to be more expressive using the language and then I don't count Whitespace dots commas parentheses quotes colons and I'll give more examples of this visually later But I just want to kind of explain where I'm coming from And these I believe are all the kind of like structural things that kind of separate out the individual blocks of information So kind of ultimately what we're talking about is counting the units of information present in a particular block of code So why even bother doing something like this? Python has this wonderful Easter egg called import this and it will tell you about the philosophy of Python So I've selected a few lines here that I thought are particularly relevant to this this topic Beautiful is better than ugly simple is better than complex and if the implementation is easy to explain it may be a good idea So if you all seen code like this It's kind of a little thing where you pull out the address field from this my contact stick You you set an unknown as your default and then that way if it actually exists in there You can replace the unknown with a particular thing That took me a long time to explain what this actually does So Python has this wonderful feature where you can just say I want to get this Attribute out of a dict and give it a pass it a default value if it's not there So this is way shorter. I can describe exactly what it is in like eight words And to go back to my strokes metaphor, I can count them all up So two male as a variable is one though my my equal sign is an operator. That's the second one if you work that out, that's about 12 strokes and then my kind of get using a default thing is Only six strokes at the end of the day so that like numerically I can think about exactly how much I'm making my code shorter and have a framework for doing this So for the visual learners to kind of like see where this information is actually going The final set the final line is kind of really where the the nuts and bolts is going on like I have two male I'm setting it to the address field of the dict of my contact this if statement is kind of extraneous like that Actually just goes away entirely and the default in that first line that unknown like that's what's get pulled in gets pulled into the final result So if visually this kind of shows like where those those lines are going away or those those key words And so along the lines of a simple get like we can also just initialize a dict in the same way like we've kind of seen code like this this little block of code takes this item in this count variable which I'm trying to in this count dick and Kind of every time you run into a particular item you want to increment the count by one And so you have to initialize it by zero if you haven't seen it before So you might wrap this little block of code inside of a loop or something like that And this is also similarly like a giant block of code hard to describe Python has this Collection called a default dict which lets you set your initial value for a dictionary You import this default dict and then you can just you tell it what you want the default value to be in this particular case I want it to be an end switch defaults to zero But you can give it you can give it a more complex object if that's what you need in your particular use case And then you can just use the object as though it were automatically initialized with the thing that you want So I can just start incrementing up by one whenever I encounter an item so this is much much shorter and You see like I'm moving from 18 strokes down to 13 and that's including this import at the top Which you probably only see once per file. So if you do this multiple times it actually saves you a lot of work So another thing I want to talk about is cleaning up resources So in a lot of times we'll open up network sockets will open files We'll be doing a lot of a lot of things with resources and when we open in this particular case We're opening files reading them out line by line printing them out and then finally we have to close it at the end Why even bother closing this up like Python has a memory manager that Deals with the variables and stuff and memory that we allocate closes them up when they fall out of scope Why shouldn't we do this with resources? Well, it turns out that we actually can Python has this feature called a context manager that lets you Kind of define the particular thing that you want to do using this with keyword Set it to the variable and once it exits that block Python will close the thing automatically So this saves you one stroke But at the end of the day if you look that closed statement is really the only thing that you're saving And I've got a room right here because that's the automatic cleanup tool Now that didn't seem like it saved you that much But I think where it really starts to come into play is when you have to deal with exceptions So in this kind of contrived example, I'm opening up my in file and raising exception Just to do some stuff and I have to wrap it in a try finally block to make sure that my code my my file gets closed With the context manager it just happens automatically like it knows how to catch the exception Run the run the closing portion of the piece of code and then re-raise the exception so that something upstream can actually handle it So I don't have to do anything else It's already handled automatically and this is much more natural because this is how I'm thinking about the code and the business logic That I'm trying to run So to do one of these context managers so simple right like you've got this wonderful arcane block of code You implement these double underscore enter and the double underscore exit methods And so this particular example I took off of the Python Example where you kind of poorly add an HTML tag to a print statement So in this sort of thing we've got the enter method which prints out the opening tag with the name that you pass into this thing And then on the exit when you exit the block it prints out the closing tag So you can print out a bunch of stuff in the middle and I'll be wrapped in this HTML tags This is a lot of boilerplate right I think we can do a little bit better Python has this built-in decorator called a context manager with the idea that a lot of these things follow the path of a function execution You run a few you you run a few lines of code in the beginning You give control back to the like thing that's within the block that you want to run and then you'd run some stuff afterwards So in this particular example if I decorate this this method called tag with a context manager I can print out the header tag Yield control back to the main function that I'm like or the block that I'm running this from and then print out my closing tag So this is way way shorter And I think way easier to understand compared to this like enter exit thing that I had before and Then this even gives me enough space to print out an example at the end that shows how this can be used I have no I tagged my h1 header tag with the foo stuff printed out And when the result when I run this the result will be that it prints out foo wrapped in those tags. That's really great So next I want to talk a little bit about functions so functions I'm sure we've all heard of them. They are really that scary What they do is they take something as input and then they do some processing on it and then they produce some output So in this particular example, I want to say that like I have this cook function I'm taking in some corn and then when I run the cook function on it. I can produce popcorn This particular construct example doesn't do any of the error checking to make sure that I actually passed in corn to Produce the result, but it kind of shows like this important output sort of thing Python has this feature called a lambda which is another way of defining a function without giving it a name And this will come in to play in a little bit When I talk about some of the other functional programming aspects, but it's an important concept to to understand So in this particular case, I've got lambda food Which is the thing that's being passed in and the result is this popcorn So it's equivalent to the function above except I don't give it a name But it's a it's an object of Python can deal with and you can pass this around just like anything else like any other Variable that's set to an inch or something like that So this can be very power a very powerful concept to utilize with functions that act on other functions So when people talk about functional programming, they're talking about three major methods, which I'm going to go through real quick We have the map filter and reduce so the first one is map Which takes one of those functions like the cook one that I mentioned before as well as a list of items So you take map you run the list of items and you have the function and what map returns is a list of Items where that function has been run on every single element of that of that list So as in this particular example if I run map with a list of corn a cow and an egg With the cook function, I'm returning the output with cook run on all of them So the result is a list of Popcorn a hamburger and a fried egg pretty simple, right? So the next kind of functional programming thing that there is is called filter and what this does is it also takes a list of items and It takes a function that returns a Boolean and it returns a list of items for which that Boolean about evaluates to true So in this particular case, I've got the pop the list of popcorn hamburger and fried egg before and the is vegetarian function And then my output list is a popcorn and a fried egg Assuming that you think that eggs are vegetarian. That's a matter for debate and outside of the scope of this talk And finally, we have the reduce function which takes a list of items again And it takes a aggregator function, which is a function that takes in two inputs and produces one output So in this particular case, we have popcorn and a fried egg as the list for reduce You pass to eat function and the result is of course poop And I'm not really not that clever I stole this tweet from someone else, but I thought it was a good way to kind of describe the three main functions of functional programming So that seemed a little academic Let's get into some examples and I find that examples are very well done with Goofus and Galant which are part of my childhood in terms of Kind of describing a good kid and a bad kid and how they interact with the world Goofus is the bad kid that does poor reckless things and in this example He runs around with scissors pointed up so that when he trips and falls he stabs one of his eyes out and Galant is the good kid Who walks around very carefully with scissors pointed down so that when he falls he doesn't hurt himself too badly So Goofus and Galant explore functions Goofus thinks very iteratively he focuses on how to compute the result He thinks about the computations that he's trying to make and ultimately he loops over data to do the things that he's trying to do Galant on the other hand thinks functionally about the end result that he's trying to produce and he focuses on what the result is And so using kind of this mathematical function oriented thing He composes the functions in order to produce the result that he's looking for So let's talk about how how they can use map Goofus in this example is trying to compute a Given a given this nums array to compute a new list that has a double version of all of these numbers And so what happens is Goofus iterates over this array and has this result called double nums and appends Number times two to every single when you encounter every single number This is pretty straightforward reasonable block of code. It's actually very clear. I would say Now with functional programming what you can do is you have this map function and you want to pass it in a lambda which takes this What did I even type here? Okay, that's a typo on the slide It should be x and I don't know why this got turned into xbox But imagine that there is an x there. Maybe I was playing video games or something while making this slide So So I'm passing it a function with lambda of x which is that thing that we saw before And this function let this lambda transforms x to x times two and I pass it in the same numbers living I run the map function which will produce the double of the results and I have to tell Python that I want a list at the end And that's an implementation detail that I might not go into So this is a little bit shorter fits on one line Possibly a little bit clearer if you're starting to think in that sort of functional transformational form And now we've got reduce so we've got this like let's say I want to compute the total sum of all of the numbers in the list I have my total equals zero I'm iterating continuously over the list or at least Goofis does And then kind of adds this thing to the total and this takes 10 strokes as well So Galant has this accumulator function which is this lambda x comma y taking in two variables And then the result is x plus y and he continues to run this over the nums thing He runs reduce and he gets the total in the same sort of way So this is also 10 strokes it's about the same thing but it's kind of shorter compact fits on one line Which is great because I have a widescreen monitor and this is a widescreen And finally filters Goofis iterates over the numbers he runs this for loop again We've got this result accumulator thing which is a list and we say if this number mod two equals zero We append this to the result 16 strokes Galant can just run this filter method using a lambda that says that do lambda x mod two equals zero Which is that same little if statement returns true when it's even runs it over the nums list And then runs the filter thing over it and that produces that same over only even thing with 12 strokes So that's a bit quite a bit shorter. So this is I think a distinct advantage here for functional programming Yay But wait there's more So now that functional programming seemed a little bit arcane in some ways I would say like took a little bit to introduce and a little bit to explain Python has this concept called a comprehension which can be used in a lot of these cases And it's a much more natural way of constructing these lists and dicks as well So we have this kind of result We've seen this pattern in many of the examples that I was just talking about before we have this kind of results list that we're trying to produce We want to iterate over everything in a collection of things We run a condition on it and then we kind of append some sort of transformation on this particular thing This is kind of the basic pattern that like the previous three examples that I was talking about was doing And this is kind of 14 strokes at the end of the day Now not all of the examples use all of these features, but they're all kind of there So list comprehension is a way to define a list using this kind of bracket notation Which is what we say with a list And we say I want to do this transformation on the item for all the items in the thing If this condition is true like that was very natural for me to read out loud right Result is the list of the transformed item for the item in things If the condition is true on that thing Very very clean and easy to articulate Whereas the previous one if I were trying to describe it, it's a little bit harder I couldn't quite do it on stage And this is actually a little bit shorter right, this is 12 strokes rather than 14 So back to that like visual sort of thing that I think is kind of shows you where this information is going is We still are setting this result list And we're kind of creating a list at the end We want to have this for item in things that kind of moves to the middle We still have that if statement which is the filter thing and that goes to the end And then this transformation moves towards the beginning So this is like we're removing this result dot append thing in our in our list sort of thing And this is where those like two strokes go to So let's kind of go through those functional examples again And like let's see if we can do this with a list comprehension So before when I was talking about producing the doubled version of every element in a particular list We've got this map of lambda with and this one actually has the right thing without the Xbox This takes 10 strokes Now with Billy Maze's list comprehension We just say double nums is the list of x times 2 for every x and nums Very simple to articulate in plain language And this is super easy to understand right Filtering works in much the same way Galant has this thing where I'm going to filter for only even numbers within this list of nums 12 strokes And Billy Maze uses the comprehension We have only evens x for x and nums if x mod 2 is equal to 0 Now this is actually a little bit longer So my metaphor kind of breaks down a little bit But like which one is easier to understand I would say the second one is much easier to understand So don't listen to everything that I say But think about what you're trying to do And what your goals are And better reduces We've got this total accumulator And we have this like lambda x and we have mod this to do But then think about it like all of these functions When I'm saying a list comprehension my result is a list This total thing is really just producing a single number And the Shamwell guy who's the Billy Maze's chief competitor Realizes that there is in fact just a sum function in Python And you can just call that on a list And it will produce the result that you want to do So why not use that instead Because it really just does the thing that you want So think about these built-ins And I teased a little bit about with dicks And dick comprehensions work in the same way That list comprehensions do So instead of doing x for x whatever whatever You do for x and y The two like the key and the value for the particular list So here's a new example of Goofus Trying to create a mapping of integer values From 0 to 26 with the letters that they correspond to So this has, you know, character 97 Is the ASCII value for a So I add 0 to a and I'll get that like a value So my result will be a dick with 0 to a 1 to b and so on and so forth all the way up to 25 to z And this is 17 strokes So when I use the dick comprehension I have nums to letters which is the same variable And I have x which is the number Numeric index that I have And then the care 97 plus x function Which turns that number into a particular string And basically this works in the same sort of way That the list comprehension does You can use the if filter as well So if you understand list comprehensions You can do this with a dick as well And this is 14 strokes so it's much shorter And I like it So where can conciseness help? I would say they're helpful in presentations like this The more code that I can fit onto a slide The easier it is for it to read And with widescreen monitors like you can fit a lot more And also on the screen when I'm actually coding Like I can type a lot more I can see a lot and get a better picture For the code that I'm interacting with At any given point in time And finally like if you're ever interviewing I think that on a whiteboard Being able to write code very concisely Will give you a lot more ability To fit more code on your whiteboard As an example Quick whiteboarding tip If you start coding in the middle of a whiteboard Like naturally your eyes are drawn there You're essentially dealing with this font size A 26 by 6 screen If you're coding on a screen You would start at the upper left So why don't you do that with a whiteboard You start from there And your screen is way, way bigger So that lets you do a lot more I'm sure that we've all encountered that instance Where we've been coding on a whiteboard And then we're trying to juggle around And we run into the edge And we draw all kinds of arrows Really convoluted and hard to follow So start at the top left on a whiteboard The same way that you start on a piece of paper And a piece of code So final takeaways I'd say that stroke reduction Which is making code more concise Reduces the cognitive load to understand code We read code a lot more than we write it And therefore like the less we have to The less energy we have to spend To understand the code that we've written The better we are and like Better able to understand like The broader picture of how things are And Python has a lot of great features For enabling us to do more with less And I think this is a big part of learning And discovering all the different techniques And kind of mastering the art of Python And finally with some operation That I talked about For a lot of common things If you think that this should be really, really obvious There actually is probably a built-in Or a library So you should search and try to find what that is Because it will do exactly what You would expect it to do And hopefully cover a lot of those edge cases That you don't want to deal with So let somebody else have dealt with them first And like not have to worry about them later So finally a quote by Blaise Pascal I apologize for the length of this presentation But I didn't have enough time to make it any shorter And a little bit about Yelp We're hiring, so come see us at our booth And here's our wonderful social media links We've got a Facebook thing With our engineering stuff We've got Twitter Where we advertise our engineering blog Where we talk about a lot of the cool stuff That we do with Python As well as other things And we've got GitHub Which of course is a social network these days And that's the end of my talk Okay, we have five minutes So I could take some questions If someone has a question, please Raise your hand No Ah, there are two So sometimes it's nice to Observe a mechanism For making some code shorter And more concise And you're really impressed with it And you show somebody and they say I don't understand that It's not clear I preferred it the long way Because it's easy for my mind To read and comprehend You sort of want to keep it Because you're so proud of it But you sort of sometimes have to let go Do you have anything to say about that? I think it's always a delicate balancing act Between clarity and conciseness It's definitely something that I encounter Especially when I'm reviewing code Written by other people If it takes me longer to think about it It's something... For example, the first time That I saw this comprehension I was actually like This is really complicated This is really confusing And this is very arcane But as I kind of grew And learned about it And I asked those questions I'm like, oh, this is actually more clear As a particular thing And I've kind of promoted On the teams that I'm on This is a much more concise way Of doing this And because it's such a common thing to do For less common things Like producing lists Or I don't know I'd struggle to think of a less common Like if you're trying to open up A network soccer or something Where there's a lot more Like non-straightforward kinds of things The explicitness and explaining Exactly what you want to do Comments are kind of The best way of getting around it And I think it's really about Understanding what the team That you're working on is comfortable with And which kind of conciseness techniques Are best adopted there Does that kind of make sense? Thank you very much for your talk I really like the golf metaphor Do you know what the 19th hole Stands for? Hole 19? Not so much What's that? It's the place where we get food So it's doubly perfect That you have to talk right before lunch I should have thought about that When I was creating these slides But I didn't know why Slot was right before lunch So everybody off to the 19th hole Right? Okay, any more questions? Oh, yeah Great So one of the downsides Of doing these sorts of things Is that sometimes They make debugging harder Because there are cases Where you want to actually have the array Which doesn't actually materialize When you do all these list comprehensions So do you have any recommendations And how to deal with that? Not often I think sometimes If I'm debugging it's Yeah, I think the thing That I kind of wave my hands around In the functional programming example Where they produce this Where you have to wrap it in a list It actually produces a generator Which lets you produce Like the individual items In that particular thing So if you're kind of like Iterating over that You can produce the element Look at each element within there And kind of use that As a debugging tool When you generate these comprehensions Or something like that Straight off the bat Sometimes it's worth just Kind of outputting them Or inspecting them in a debugger Sorry, if that doesn't Quite answer your question Debugging is a harder Sort of concept That really depends on The environment And the editor Stuff that you're doing So maybe go to some of the PyCharm talks And I'm sure they'll Give you some better advice on that Alright, one last one Anyone? Nope, well Then give them a hand


Welcome back to our live programming in our Palo Alto studios. Wendy Whitmore is live remote, but she's in the house sorta. Wendy is the senior vice president of Palo Alto's Unit 42, the world-class threat intelligence operation. Wendy, great to see you again. Thanks so much for taking some time out for theCUBE. Oh, you're so welcome. Always great to see you, Dave. Hey, so, you know, some folks might not be familiar with Unit 42. So why don't you set up our discussion with a little background? Great, I'd love to. So you're probably familiar with Palo Alto Networks, right? We're the largest cybersecurity company at this point in the world. And Unit 42 is really the eyes and ears on the ground for Palo Alto. So we are the special forces team that jumps in and solves investigations on behalf of our clients. We're also made up of threat researchers and analysts, as well as threat hunters who are mining data sets throughout the globe to really identify what's new, what's latest and greatest when it comes to attacker techniques. So we first had you on just about a year ago, almost exactly a year ago. So what's changed in terms of the threat landscape over the past 12 months? Yeah, so three items I would highlight for you. So first, and there are all three assets. So scale, speed and sophistication. I would say from a scale perspective, this is the first year in, I mean, I've been doing this now over 20 years, which is hard to believe some days, but from a scale perspective, for the first time ever, attackers now are targeting mass vulnerabilities as they're inroads into getting into these organizations versus spearfishing. So that's really the first time that we've seen any type of avenue into an environment be larger than spearfishing. So attackers are looking for scale. How can they compromise more systems in a shorter timeframe? And that gets us to speed, right? How can they move quickly? So when we talk about cyber criminals or nation state actors, they were really looking at the ability to exfiltrate data within hours instead of days or weeks as it used to take and in some cases used to take months. And then from a sophistication perspective, we've always seen sophisticated nation state actors where they have an objective and they execute against it. But what we're seeing when it comes to cyber criminals is that they are more organized than they have ever been. They understand how businesses operate more effectively than they have. And they pair that with technical acumen as well as the ability to conduct social engineering attacks which are now fueled by AI. Okay, so you've got the nation states, Russia, North Korea, et cetera, and you've got cyber criminals. Can you sort of lay out the sophistication of each? Are they comparable? I know, I think they have different objectives, but how can you help us understand what that looks like? Yeah, I would say they're in a bit of an ARPs race, right? So from the perspective of causing the most damage and destruction to operational impact, I would rank cyber criminals today higher, right? These ransomware cases, we're seeing so many cases now, 37% this year versus 20% last year when we look at cases where there's extortion and so extortion, data theft, and then actually causing just disruption to individuals. So not only attacking an organization through their means of, say, a CEO's business email, but perhaps doing something, reaching out to a spouse over social media, sending flower arrangements to their house to prove that, hey, I have a lot of understanding of where you live. We're seeing just a tremendous amount of increase there. When we look at nation state actors, they're not typically, and we can circle back to this because some of this has changed, but they are not typically going after a return on investment from a financial lens immediately, right? They're looking to steal data and then to provide it to some backend group that's going to conduct the analytics on it and then make some decisions based on that data. So the means and the objectives are often different, but the capabilities, I think, are really growing increasingly pretty similar. Have you seen any evidence from, say for instance, Russia or Iran, given the war in Israel and everything that's going on over there, the chaos, have you seen any either escalated attacks or new threats from them or new signatures? Well, I think the real point of your question is the fact that these geopolitical wars throughout the world right now are causing the ability for nation state actors in particular to really operate behind the scenes in a way that's incredibly effective. So we have absolutely seen increased activity in particular with Chinese nation state actor related to activity within the Pacific Rim region, as well as the interest and intent to compromise more organizations throughout the globe as well. When we point back to Russia, still very active from a cyber landscape, right? So the war there certainly is very ongoing in a physical sphere, but in terms of just disruptive attacks in the cyber landscape, we're seeing those more and more, including attacks today. And with respect to Russia specifically, you're seeing obviously nation state activity. I suspect you're also seeing a lot of cyber criminals. Are they coordinated? Are they generally sort of isolated? What are you seeing there? That's a great question. I think with Russia specifically, it's sometimes hard to tell, right? The waters are often muddied and we see that on very intentionally from these nation state actors. And so, generally we've been able to see different objectives coming from a nation state that are largely based on right ceiling data, which I mentioned, but now you've started to see North Korea in particular and China in particular have organizations that are offshoots of part of threat actors that are tied to nation state actor behavior that are actually responsible now for conducting financial attacks and ransomware attacks. So I expect that we are going to see more of that in the future from countries throughout the world. I remember last year, last December at Palo Alto Ignite, near Zooks had made a statement, something the effect of within five years, every SecOps organization is gonna be AI-powered. Has that actually happened in 2023? I bet you we're much closer to that than maybe that five year forecast, but what have you seen in that regard? Well, it's wild that that was 12 months ago, right? Because I think so much has happened in that timeframe in terms of AI-powering activities in a great way for defenders and a great way for security operation centers, but also then causing certainly enhanced capabilities on the side of the attackers. So many SOCs throughout the world today are using AI for accelerated automation. They've been able to really offload some of the workflows so that they can focus their limited people resources on the highest criticality tasks and those that really require human analytics. So I think in those senses, we've seen a lot of positivity. Certainly on the side of the attackers, we're seeing them leverage AI today. It's less to effectively generate new types of malware and it's more to enhance the human side of social engineering, right? So the human side of these attacks, we've seen that ranging and everything from, eliminating language barrier in written communications to them being able to successfully conduct voice scams where they're replicating for identity verification, they're replicating employees' voices and being able to get through identity verification systems that way. So I think a year from now, we're going to be having an even more different conversation, right, based on the capabilities growth at that time. Yeah, when I asked you a similar question in last year, even at RSA, at the time it wasn't as obvious in just a few short months, now you're seeing very clear indications and patterns emerge. That's amazing. I think it really speaks to the growth of AI, right? Of generative AI in particular, right? Machine learning's been around a long time and different types of precision AI have been embedded in these technologies for a long time but generative AI has really changed the game and the landscape in terms of how we work and the ability to accelerate some of the workflow. So I think it was hard to predict at the time when it was just emerging and I think likely similarly, it's going to be hard for us to predict what 24 months looks like, right? These voice fakes are kind of scary, right? You'll get these robo calls or maybe not robo calls but you'll get these calls from, you don't know who it is, you're kind of afraid now to actually use your voice. You'd rather just, you know how you, some, Lena Smart from MongoDB says, just don't click on links. Just do not click on links. Now you shouldn't answer the phone unless you know who's calling. Right, right. It's just something that, you know, from an increased education perspective and awareness we all need to be aware of but for people, you know, like you and I that are out there and it's easy to find video clips, right? And voice clips certainly concerning.  How is this changing sort of the need for how organizations approach things like backup and recovery and how that fits into the whole equation? You know, I think backup and recovery have become foundational and table stakes for so many organizations. And that's great news but that's specifically then why we've seen attackers shift towards the extortion campaigns, right? So, hey, okay, we're going to steal your information and then we're going to threaten you with the public release event whether that's publicly whether that's to specific clients of yours whether it's, you know, coming back to the SEC, for example, afterwards and saying, you know, hey, we attack this organization we saw that they didn't disclose it. There's so many different angles of that now. So I think we'll continue to see, you know, the forced kind of really human connectivity in these attacks and those becoming very human. I think the good news for organizations is it so much of that information then still falls back to security best practices. And it doesn't mean that, you know, you have to buy the latest and greatest brand new technology. It means you still have to do the fundamentals very well. The thing I would say that's changed that we hear CISOs and I talked to CISOs nearly every day of the week and they say, it's really hard for me to prioritize what's next, what I defend when a new application comes out with a vulnerability that I've never even heard of that wasn't, you know, where that this was in my environment. So the ability to continue to identify where you're exposed in your external attack surface, I think that's more important now than it's ever been. But the good news is that's still really a security fundamental that so many organizations can do well. The thing about organizations, and there aren't a lot of them that have sort of unit 42s world classness, if I can call it that, it looks like the anatomy of complex hacks is really evolving. You talked about mass vulnerabilities. I mean, we're just, we were familiar with, we were just in June at the, at, you know, MGM, one of the MGM facilities and you read about how MGM and Caesars got hacked with a spearfishing incident. And now you're talking about, just again, a few short months later, you're seeing just a completely different, you know, approach and the anatomy of these breaches is changing. Can you add some color in terms of what you're seeing with your incident response cases? Yeah, well, I would say one where our team's busier than we have ever been.  And so I think that really speaks to the volume of attacks and breaches that are going on. You know, you mentioned the Las Vegas attacks. And so you have an attacker, in that case, Scattered Spider, you know, who you're, what I think you're really seeing, and part of the reason that these get so much attention in the press is they cause actual consumer impact. And I think that these attackers are, are quite bright, certainly in the way that they're approaching these types of attacks and we're likely to see more of that. Because when you get that consumer impact, it starts that they'd see, hey, I caused a lot of business disruption. There's not only, you know, the financial reporting that goes on behind the scenes, but there was actual, you know, trades and bets that couldn't be made. There were hotel rooms that couldn't be checked into. There were purchases that, you know, on the casino floor that could not be made. And that actually impacts people's livelihood when it comes to, you know, tourism and hospitality industry. So I think that we're likely to see more of those types of attacks where they're disruption focused and in particular are going to really challenge businesses to operate through these types of attacks, which becomes, you know, more now than norm than the exception to the role. So amazing. We were out there that same week of the MGM hack. We were staying at the Cosmopolitan and you've been to Vegas many times as have I. You know, the MGM is always lit up and it's green and you can see it from outer space probably. It was completely dark. There were maybe a couple of lights on and we were just astounded. They had to revert as you I'm sure know to sort of old techniques of actually escorting guests to their room to make sure that they could open the door. It was interesting the following week we're out at Caesars. Now it was a little more time to recover but they seem to recover a lot better.  Do we pay? Do we not pay? It's sometimes illegal to pay. What do you tell people who ask you should we pay the ransom? I think the intent 100% of the time is always to never have to pay a ransom. The reality is there are some cases where that simply is not practical but whether it's national security related whether it's simply business impacts related and there was data stolen that the organization simply can't get access to. So what we recommend is that organizations are as prepared as possible on the front end of that and that is not just the CISO. That's the CEO, that's the board of directors having very close relationships with the CISO, with the CFO making sure that they understand what are the potential scenarios that we need to be prepared for? What are their scenarios within the realm of possibility that we would potentially need to pay? And if so, how do we do that effectively? How do we make sure that we protect our consumer and our clients data as much as possible and do the right thing? And then certainly coordinating all of that activity with the correct law enforcement entities and regulatory bodies. When I was in awe of people like yourself I think I wonder sometimes if you sleep with one eye open but after we get through the holidays obviously RSA is coming up, you're speaking there I'm sure again you probably have a lot going on there but what's next for unit 42 generally and you in the world of cyber? You know 42 in terms of not resting, right? We certainly, that's what we get paid to do is not rest. So holiday season unfortunately is often also known in our industry as breach season. So I think we've got our eyes and ears wide open as to what comes next. If there's some sort of major global attack or new exploit that's released that we all need to be prepared to defend against but we're really looking forward to moving into the new year and helping secure as many clients as possible. So do hackers take vacation? Do we have any evidence that they ever? You know they do but they don't seem to coincide with that major holidays, right? Just like anyone, they see that as an opportunity to potentially take advantage when you know people are out or their schedules are different. So we need to all be prepared for that. Wendy, thanks so much. I hope we see you at RSA 100% we want to get you back in the program. You've always been a friend of theCUBE and a great guest. I really appreciate your time. Awesome, thank you Dave. Look forward to seeing you again. All right, you bet. Okay, up next we wrap with analyst angle myself John Furrier and the two MDs of theCUBE research Kelly Kramer and Rob Scratchey. We're going to talk about what we learned today and we'll set up the next section of the program. We're going to hear from the ecosystem. Keep it right there.


So, the first question is why HTML5 stock charts? So, how many of you are really interested in stocks in general? And you have been following stock charts I believe so. So, most of the stock charting libraries that are available are typically flex or java based and there was really no alternative. There was SVG, but there was little limited in capabilities. Then the HTML5 canvas element has got most of the things that are required to do 2D plotting and with WebGL it is also for 3D plotting, but that is not something we are concerned about. Good thing about it is it is dynamic, it is scriptable. So, we can make use of JavaScript for doing all our charting. There is another thing that there are some really excellent JavaScript plotting libraries some of you might have used those. For example, if lot is something I have used and jqplot is another one that I have used, but they are more for plotting typical more like scientific data or other type of data and not very specifically mean for stock charting. When I am when I say what specifically mean by stock plotting what it means is there are certain requirements that a typical stock plotting application will require. So, you have something called as an open, high, low, close charts and you also have something called as overlays or indicators typically like exponential moving averages and so on. And you also need some interactivity as in you want to be able to add indicators to the plot you want to see on different time scales. So, those kind of that kind of availability is not available in some of those libraries. So, that is something that I also decided to do. I was mainly interested in stocks for the stock charts for Indian markets and they were not really good charting applications that I could come across and a lot of it was data problem. So, I had to do solve that as well, but that is something that we are not going to talk about it. So, there was clearly a case for doing something with HTML5. I knew already a bit about JavaScript because I had used jQuery in the past and Flex was not kind of intuitive to me. So, I decided this looks like a good something to try in HTML5. So, this is a library that I have developed which is essentially for plotting the chart data. It is still early and it does not have a well defined API. So, if somebody has to use it, you have to dig a little bit into code and use it. And there is an application that I have developed that demonstrates the use of this library. So, with all the straight away going to the demonstration of that application. But before we go, let us look at what we want from a stock plotting library. So, typically if you see any stock data is a time series data of open, high, low, close and volume data. So, you need an ability to plot at least one of these types of plots which is like line plot, open high, low plot and there is something called as candlesticks. Then also you need ability to zoom into some data, zoom out of that data and pan across the time scales on that data. Then you need some more ability like to be able to add trend lines, retracement lines. These are little specific technical terms to stock charting. Let us not worry about it right now. And you also need some kind of support for indicators or overlays as they are called which are typically like exponential move. How many people are aware of these things exponential moving averages, right? So, it is basically I will just quickly explain what this is or we will just explain as we have a demo. And this is something that I was particularly interested in in doing something that could care ideally from tick data to all the web to monthly data. So, you have a tick data with you, you have a library that takes care that makes use of the tick data and you could just plot it from a from a minute chart to a 5 minute chart to a to all the web to a monthly chart. So, let us see, let us get, let us see about this. So, we of course do not have an access to internet, but I have this application running on my local news. So, some of the data that you might see might be old, but that is still ok. So, essentially what this is, this is an application that that I am discussing about. So, the UI is very primitive because as I am not so much concerned about the UI, but it should still show all the kind of interactivity that is possible with this kind of an application. So, most of this application is written in Python Django. So, this is not node.js, there is something in node.js that we will talk about little later. Basically we have this input for ticker symbol and most of the DOM manipulation is taken care of using the jQuery. So, basically here we select a ticker symbol we might be interested in. This seems to be my favorite. So, what we have now is a plot of the data of the state bank of India. Now, if we really look if we look at it what is happening really on the network what we get here is essentially we get a JSON object that is nothing but a series of timestamp open high, low close, a series of open high, low close. Then if we really look at what is happening little inside, then we will see there is an object called as plot, there is an object is it visible? Not quite right, never mind there is an object called plot. So, this is basically when we open this URL we get this object called plot initialize. This object if we try to inspect a little bit about this object. So, this object is basically what it does it does an initialization of an HTML canvas page. I think that should be visible right? Yeah, that is what is more important. Then we have something called a chart parameters object, then which we have something called as a context object. So, these are the internal browser objects the canvas and the context object. Then we define the define certain objects that are mean for manipulating data. That are mean for manipulating data those objects are called as current object. So, the current object essentially if you look at it holds what is called as indicators we will worry about those indicators. Then there is this object called open high, low close which is the actual data that we got just now from the from making a network call and then there are these objects. So, what essentially happens is when we make a network call this is an object that gets initialized and it has all of these sub objects that get initialized as well. Now what we want to do here is this is a daily plot of the daily plot of state bank of India chart. What we want to do here is we want to have a weekly plot. We just change the time scales from daily to weekly and that is it all we have to do it does not do any network call because all the all the data all the all the computation that is required to be happening happens at the client side. There is no network call involved in this and you could simply as well scale it to a monthly data and then you could again change back to daily data as well. So, the interesting thing that happens about this is let us say let us from daily data when we move to weekly data and if we again inspect the object what happens is the current object is only changed to monthly data and everything else just remains the same. Then there are basically one thing that needs to be kept in mind. So, the only network call that we make in this is when we get the JSON data everything else happens on the client side. So, there is there is no other network call that happens. Then you could change that chart to a bar chart or a candlestick chart. Sometimes you want to be able to print this chart. So, you want a better background. So, these are just bells and whistles. Let us not worry about this. So, there is something interesting I want to talk about. Then here you find something called as mode. So, the mode that we support is one of the two modes one is called as either a pan or zoom mode. So, in the pan and zoom mode what we can do is we can just pan across the time scale and then there is something called as a trend line mode. So, this is an elementary support for what I call it as trend line. So, you would be able to add trend lines to the chart in this mode. This support is quite elementary. It is not as good as I would want it. So, this is something I am working on. So, these are the modes. Another important thing of any charting application is typically if you see the distance between this is a linear chart. This is a linear support. The distance between 1000 and 1250 is same as 1250 and 1500. So, if you want to change the distance from linear to logarithmic all you have to do is this. Again there is no network called involve all the computation happens on the client side. Typically, logarithmic charts are used for data on a very large time scale. So, weekly may not be a right, but on a monthly time scale this logarithmic data makes lot of sense. And then we have the support for what is called as zoom in and zoom out. So, basically you can focus on more specific area of the chart. This is a kind of interactivity that is not typically available with most of the normal plotting libraries that you get, like jqplot or flot that I discussed just now. Then there is something more that a typical charting application will have support for something what is called as an indicator or an overlay. So, the indicators are something like an EMA close. So, basically it is an exponential moving average. So, what is an exponential moving average? This essentially says that this is an exponential moving average of close prices for last 20 periods. So, the thin line that you see is essential. So, what it basically means there is a moving window and the thing about the exponential average is the exponential moving average is more sensitive to the latest prices. So, there are different kind of overlays that are supposed a simple moving average is just some of closing prices divided by 20. So, this is how you can add indicators and delete indicators from the chart. So, this is the kind of interactivity that I was discussing about when I said that this was not readily available. So, this is something that I went ahead and did. I am going to talk little bit about what is inside the library and why certain design decisions were made. So, the data is supported is in the JSON format. So, everything here is essentially a timestamp. An array of timestamp open, high, low, close and an optional volume. So, volumes are not, volumes sometimes are not available. Then the timestamps are essentially unix timestamps. I am sure many of you are aware of what are unix timestamps. So, one good advantage of having unix timestamp is you just have to deal in one format. For example, data could be 12 January 2011. It could be 12 January 2011, 5 p.m. or anything like that. So, we convert all of that into a unix timestamp and deal with it and these being numbers they are very easy to deal with. So, that you can easily take percent hundred, you can easily take percent thousand, percent eighty six, four hundred. So, that is the idea. And these being numbers also helps in scaling across the same. So, for example, if you have let us say a minute data or you have this data from minutes and you want to scale it to an hourly. So, what you really have to do is really take collective 60 candles and take their data. So, that is how you can easily scale using this unix timestamp. So, that makes it much more easy. So, how do you really use it? So, we define the object that we described has got what is called as a read function. So, basically this tries to read data and then there is a function that is called as plot function. So, this plot function is typically called whenever any parameters or anything changes. So, excuse me, we want to make sure that plot function is really lightweight. We do not want it to be very heavy. So, basically it initializes the canvas element. If the canvas element was not already there, then all the margins that you saw in this are needed to be initialized like the margin at the top and then the margin for the volume etcetera. So, all these margins are initialized in the plot function and sets the basic objects and its properties that we saw like the current object, timestamp etcetera. So, and we have to make sure that this is a lightweight function because every time any parameter or any data changes we are making use of plot. Even when we are panning or zooming, we are essentially what we are doing is repetitive calls to plot on an event listener. So, we have to make sure that this is not a very expensive operation. So, what we do for that is we define something called as a plot window. So, a plot window typically if you see in a typical use case, you would rarely see more than a couple of hundred candles or a couple of hundred time series data. Even that is pretty rare, but typical use case is like 50. And what we want to do for these 50 is we need to know the x y axis bounds for this. That should also include the indicators or overlays that we have added. We also need to scale to log or linear depending upon user interactivity. Also remember log is a very expensive operation, it is not a simple mathematical operation or easily available. So, this is an expensive operation. So, we have to make sure that we do not do a log operation on a 10 k data size array, we have to do it on minimum as minimum as possible. So, we typically define a window, do all these operations inside this log, inside this window and then just plot the window. So, all the all this heavy work is done for not more than 50 or 100. That we define that determines the bound depending upon where our x axis bounds are. So, these are some of the design choices that I have made that I have made doing this. This is something else that I wanted to talk about. So, I will just keep a brief background. So, I was just having a chat with a guy and he said that this library is cool, can I do it for real time plotting. I said yes, it should theoretically be possible, it is not really difficult. And then we were just discussing he had some data, but that was a very static data and only 50, so 50 or so kind of data. So, I said oh that is not fun. So, there is something called as a random walk. So, basically what they say is the stock prices typically follow what is called as a random walk. So, essentially what it means is the movement of stock prices is random and the current price is only a function of last price. So, I said that could be a good idea. So, I just quickly googled for a random walk something that could give me a random walk on a javascript. So, I found out and put together using node.js and something this was really fast that I could put together using node.js. I actually intended to show it live and unfortunately I did not have it ready on my laptop, but I will just show you how the app works essentially and that makes use of node.js and socket.io. Okay, so what we are basically going to do in that application is we are going to generate a stock chart data using a random walk process. We will just show quickly see what the code is like and use that data on the client side javascript to generate real-time chart using the DICP library. So, I am not sure whether we will be able to see that live, but I am just trying to show what we are going to what how the app looks like. Okay, this is so is it visible? So, basically we have heard enough about node and socket.io. So, I am not going to talk more about it. This first line that you are looking at is just to tell the node that where to look for some of these library if not if they are not present in the current directory. Okay, then this is the random walk function. So, what it basically does is there is a there is this normal random that is defined in my library and wall is for the volatility. So, it just finds a normal random, it finds a multiplier and the next price is the multiplier of the previous price which is which we call it as seed. Okay, and somebody is going to talk in detail about express. So, I do not want to get into the details and frankly I do not know much about it myself. So, this is something I just could quickly hack up in like half an hour. So, essentially this is just a configuration for our create so for an express server and what we are here going to do is basically we define a function called delay here. So, every 100 milliseconds what we are going to do is we are going to call this emit price function. We are going to call this emit price function that is going to change a value of a global variable to some value using a normal random walk and then we are essentially going to what we are going to essentially do is there is there is going to be an event called take on a socket on a socket socket that we create and we are just going to throw that throw that price to the client. Okay, and this is just the initialization stuff and the application listens on port number 8002. All right, and this is how the client function looks like some basic initialization. What is what is of interest to us here is this is a function called tick. Okay, so this is a function that gets called on on every tick event that happens on socket IO. So, what it is essentially trying to do is build the candles on the fly of one candle will be of typically of 25 data points. It's accumulating those candles of 25 data points into this and just plotting. I wish I could show you you would be able to appreciate this really well after that. I don't have the node packet manager with me so as a consequence I'll have to figure out how to add those sockets that are your parts in my so I wasn't quite prepared for that so I can show you. So, this is essentially what I wanted to show on node.js application for real-time talk charts. Yeah, so this is pretty much what I had to talk about in this library. So, yeah, if you have any questions or something. Well, you don't really create Canvas. Yes, unfortunately for this kind of application there is no other choice. Okay, because what's happening is when you panel zoom here it's not just you know that one area of Canvas that gets changed. Sometimes even the access gets changed. Okay, sometimes the whole data set gets changed. So, this is still expensive, yes, but this is something we can't get rid of. Yes, yeah, I am not really aware of. I'll have to look at it. But it will be, I did a benchmarking long-term back but it wasn't the Canvas painting that was the most expensive from what I remember. I need to do a little bit of benchmarking again, but it wasn't the one. So, what we want to do, what we try to do here is, yes, that is expensive and that will always be somewhat expensive. What we don't want to do is we don't want to add computational overheads to that. For example, let's say typically this data is typically is like 1k or 2k data points and sometimes very easily 10k is a very normal use case. So, in that kind of data points we don't want to go and take log and we are anyway essentially going to throw away 9, 9, 50 of those. So, first what we do is we define the x-axis bound that in turn determines the y-axis bound and then we re-plot the Canvas. So, that's that's how we do it. I've tried it using path. There is something there is a there is a way to do it.  I think I can show you how just a minute. Yes, something like this, you want the value at these points. So, the way to do it is basically find your offset in the canvas of that data point of your mouse position. Let's say your x-axis is let's say 200 and your mouse position is like at 150 and your entire x-axis range is whatever let's say 10 on some scale or some function. So, this is going to be 7.5.  It doesn't matter. So, the way to do it is okay the way to do it I would say is let's say you get your x and y coordinates and get your actual x and y values okay.  This is very little else that you can do because you know the granularity that you have a label is one pixel and nothing nothing less right? Yeah, okay, okay, get an equation for a path that's like completely outside the scope of this. I mean you have a path and you want to get an equation for the path is that no no that's I don't think I have never tried that and I don't think that's it that's really that simple all right okay thank you