It's time to, yesterday if you were in here, we saw a Teddy Ruxpin get hacked and today we're going to do something a little bit different. We're going to do power plants. So this is going to be very exciting. Let's give Joss and Marina a big round of applause. Have a good time. Good morning. Thank you so much to all of you for coming here. So today we will be like, we will present you like, what is an effort, what it takes to design embedded exploits to actually cause physical damage in industrial facility from the beginning to the end. So briefly about ourselves, I will let Joss to present. Yeah, so my name is Joss Wetzels. I'm an independent security researcher with Midnight Blue, where I mainly focus on different kinds of embedded systems, industrial control systems, automotive, IoT. And I previously worked as a researcher at the University of Critical Infrastructure Security. So my name is Marina Krotofil. I've been doing, my major specialization is physical damage and I've been doing it for over eight years. I presented a lot of works at Black Hat and DevCon. And so how this combination of us came together is, on one hand, Joss can reverse engineer and develop like exploits and implants for any embedded system in the world. However, like what exactly do you want to do on that device? And I am my specialization, hey, here's this power plant, chemical plant or traffic light system, robotic system. How exactly you want to cause physical damage? So I will be designing engineer this exploitation scenario to cause physical damage. And then at the end, I will come up with a set of algorithms which need to be implemented and with a long list of tasks which needs to be executed on that embedded system. So with that wish list, I would go to a guy like Joss and say, hey, this is what I need to do on that device. And maybe like somewhere we will also meet in between when I look at the design of the device like, hey, the specific design features of the device can be probably used to accomplish like simultaneously several tasks. So this is where we would be working together. So this is how the combination came into place. And so this is how we will be presenting the presentation. So after the introduction and industrial control system and safety systems, by the way, how many of you have heard Triton? And I hope that everybody, good, because that was our motivation. We also wanted to show like in the mass media, this Triton exploit was like described as like so sophisticated, like only state sponsor, only a few people, and we will show you if it's really so. So after the introduction, we will go into the device exploitation because at the end, you need to obtain code execution on the device. So after you have code execution, then you start already developing, for example, implant like back doors like and payloads which actually targeted like design for physical damage. And of course, conclusions in the end. As you already probably noticed, I'm speaking very fast and there is reason for that because the topic which we're presenting to you extremely complex, but we still wanted to show you the process from the beginning to the end. And still, of course, with our deep details and it's already a lot. So it will be fast-paced talk. However, again, the motivation was to show you the full process so that you could see it from the beginning to end. And we will be posting even longer version of the slides after this talk because we have to cut out certain information. And if you're interested, you can go and review it again. So introduction. So what are the industrial control systems? Well, you probably all know, but it's important to set up a vocabulary so that all of us operate for this talk, operate on the same terms. So industrial control systems, those are computerized systems in network which control physical process. So typically, if you talk about industrial organization, there will be corporate network, there will be another network which is called SCADA or control network. And even though this is the same looking computers, one will be called information technology and another will be called operational technology or OT. Nevertheless, that is still all computing devices and the guys who deal with those systems are studied typically something like computer science. And as long as we are moving towards physical process, that is already engineering science and it's completely different. So the laws that we come into the physical process, the computing devices which actually monitor and control physical process, they look already like this and they typically know them as embedded systems. So they don't have operating system as we know it. So operating system which actually runs the device and executes the programs is called firmware. And since these systems are real time systems, the entire firmware must be loaded and executed in the memory at all times. So on one hand, the attacker needs to execute code so the attack will be executed on this embedded system. So it's still cyber domain in order to cause physical damage and basically damage, for example, the machinery and operate the machinery as needed. And so when, for example, when the attacker is going into the penetrating financial sector, his goal is to steal money in some form. So when the attacker is penetrating the industrial network, his goal is to cause some sort of form of physical damage in the physical domain. It could be also economic damage like spoiling the product. However, in the mass media, typically this physical damage is depicted as a big explosion. So I see a threat landscape has changed in the past eight years. So I started doing this type of security before it was fashionable and nobody knew about it. Well, like in IT domain, there was like crazy amount of hacking happening all the time. Like industrial control systems lived in bubbles, nobody knew about them. And then Stuxnet has happened. So it seemed to be like Stuxnet was kind of a trigger and a tipping point because we started seeing more and more publicly known as pionage attacks. And somewhere like we started seeing first publicly known like activities related also ready to recon of the operational technology system. And from 2015 it started to happen. And so there was two power grid attacks in 15 and 16 and in 17 basically it was announced at the end of 2017. Triton has happened. What it was, the attacker tried to install backdoor or remote access trojan on a safety controller. And it was a big thing because it was picked up by pretty much every mass media including Wall Street journals which said that that was very, very, very alarming situation. And why was Triton so special? Because it was targeted safety instrumented system. So what does it mean exactly for us? Physical processes are inherently hazardous. So it means that there is like toxic flammable liquids, fires and explosions, electrical hazard, moving parts in the machinery. So there are a lot of layers of safety protections to prevent harm or maybe even casualties to humans first of all. And secondly to prevent environmental damage and also machinery because it's expensive to kill machinery. So there are layers of protection starting with the design of the process. Then we have a control loops. Then we have a human operator who reacts to the alarm. But as soon as like control system and human operator are no longer able to control the process in a safety manner, we have so-called safety instrumented system. That is independent control system which reacts on the hazardous condition and tries to prevent it. So as you can see this is the last lane line of defense before hazardous accident is already happening. So and the attacker was apparently was trying to disable the system. So that is actually attack on the civilians which is not good. So previously safety instrumented system are software systems and because it is a line of defense, typically it is recommended that they would be run on the isolated and segmented network. But for ease of design and for ease of maintenance, they often are connected together to the main control system. And that allows the attacker to obtain remote access to safety instrumented system. So that was a triconix from Schneider and it is a very critical safety instrumented system of safety integrated level 3 which is only 2% of all hazardous situation for example in oil ease of that severity. So it is very critical safety systems. Triconix is everywhere for example you could find it on the swimming ships and so on. So the attacker obtain remote access to the engineering station which was connected to the safety controller so they got the ability to communicate to safety controllers. And what an attacker attempted to inject a passive backdoor or some people refer to it as a remote access Trojan which would allow attacker to read arbitrary memory right into the memory for example shell code for the attack and then they execute that code. So now even if you have the backdoor it really means nothing to you because unless you know what you want to achieve in the plant the backdoor is actually absolutely useless and harmless. So attack scenario depends on attacker goal and sometimes it means explosion but in most cases it does not because you don't need to over engineer if you just want to send a warning sign to your enemy or what not. And like some you can achieve some simple like attacks like do not press there are certain buttons on the HMI which are so descriptive like stop start you can press it and you achieve some effect but you know that is not a long lasting effect. So like yes the first cyber physical attacks were happening at the HMI then the attackers moved their exploits for example in destroyer crash override they were executing the attack already at the level of the industrial protocol but now we have Triton which is already on the embedded system. So what's going on why the attack is already moving their exploits into the embedded systems and we will explain. So industrial processes are very complicated and they actually build inherently in design to be robust and recoverable. So if the attacker wants to achieve any significant long lasting damage they actually need to obtain very specific process like very detailed process comprehension the design the dynamic behavior of the process physics and so on. So for example like what causes the pipe to explode well what causes the right pipe to explode and then what causes the right pipeline to explode at the right time and suddenly the complexity is increasing. So industrial control systems operate on the control loop principle so control systems and human operators they use sensors to observe the state on the process and then control system compute the commands to instruct actuators to control the process and bring it into the right state. So in a nutshell when the attacker is start designing or engineering damage scenario there are huge number of tasks which he has to accomplish like starting from of course manipulating the process but then the attacker needs also to obtain a feedback loop to know whether the process is moving into the right state and he is successful. I've been doing this for many many years and I've designed a lot of damaged scenarios and I can tell you that obtaining feedback loop was one of the most hardest tasks to achieve because mostly sensors which are installed there in the plant they're useful because the attack is something always weird and the plant was not designed to measure something weird going on so you have to do it indirectly and of course because you are trying to bring the process into the wrong state in the harmful state the control system and human operators and safety system they will try to fight you back and to bring process back into the normal state so the attacker also have to prevent the response and try to falls into this category preventing response because they wanted to prevent response from the safety systems so in a nutshell the cyber physical attack or damaging attack that will be a collection of clandestine control loops because the attacker is becoming a control system and you have to have the cycle of observation and manipulations to achieve a safe state so attack timing is crucial because process is not vulnerable at all times so you have to find that vulnerable time and when the attack has to be executed the attack coordination is critical as you have seen there have been a lot of tasks and for example observation is of state 8 and component B needs to trigger payload X, Y, Z so this requires very granular control across the entire process and to manage tasks quantity and timing so there was a very nice presentation even though I've been presenting on this topic a lot of times like many times so there have been a nice presentation from Jason Larsen who decided to compare different damage scenarios and he came up with this timing and state diagram where he divided all the control damage tasks into damage scenarios into different tasks where he also tried to map like hey this is a set of tasks which I need to execute how do I map them now to the different device and to different implants and so basically as soon as you as amount of tasks which you need to execute to achieve damage is becoming large you need to have an implants because like you're getting separate tasks you will not be able to coordinate them so and this also allows you actually to coordinate so you can easily then install the coordination between the implant via the communication links or you can implement the routine to detect specific state so and it's also much more stealthy because you don't have like anomalous you could not create anomalous network traffic so but before like in order to actually achieve this to implement an implant you first have to exploit the device and you better enjoy extreme programming because those devices are extremely small have very limited resources they packed with the functionality so you first have to actually find something what device does not need eliminated so that you could put your actual exploitation code so at the end Jason came up with this diagram where he has like put a score on different tasks and he compared the reliability because it's not only the reliability of the attack scenario but it's also reliability of your exploit or implant on the device and so basically it's kind of you have to find this trade of like what is the most reliable attacks implant and attack damage scenario and implant stability so now after this long introduction I am giving back to Joss and he will walk you through the device exploitation and designing like basically implementing so now I have my wish list of tasks and I'm giving it to him to implement alright thank you Marina so before we can do all this so before we can get you know to the cool stage where things are actually exploding we'll have to get to another cool stage which is exploiting the device and basically the process is as follows and I'll walk through these steps and we start by obtaining the necessary materials so we need a couple of things before we can device an exploit and an implant and the first thing we'll need is documentation and a lot of it you know developers guide planning and installation manuals all that kind of stuff and yeah in the case of TriConX it was very useful that these things are safety certified at a certain level and that means that all of the documents were available on the website of the US Nuclear Regulatory Commission so very detailed information was out there in other cases you might have to buy it but this is the first step you go about then the second step is obtaining the engineering software so these devices they're connected to a workstation running I don't know Windows XP with some software that is used to program them and this software usually contains the functionality for talking to this device and the protocols you want to take a look at so you obtain this by just going to the vendor website or asking them nicely which is the easiest route or if you're already compromised asset owner networks you might take it from their own network because you're in there already so why not grab a souvenir or you might go to the various sketchy sources on the internet like eBay or Alibaba or open directories hosting this kind of stuff so you know it's usually relatively easy to come by in the case of TriConX we found a tri-station software for the equivalent of 3 US dollars on some Chinese website so yeah that was relatively easy and then we have to obtain the device and that's a little trickier because you're not going to find that kind of stuff at a yard sale or in a corner shop or whatever it's very expensive and you might have to buy multiple copies because you might have to do a tear down or you might break a device etc etc so ideally you buy it directly from the vendor but if you're a nation state sponsored attacker you don't maybe want to directly do that so you need strawmen or you buy it at a bankruptcy auction or again eBay or Alibaba are your friend so here you can see I'm not sure if it's yeah it's all coming up it's relatively expensive in most cases a couple of thousand bucks for one of these controllers and you need multiple parts to put the whole thing together so it's not very cheap then you need to obtain the device firmware the stuff that runs on the device itself so you have various options here you can sometimes download it from the vendor websites or extract it from some update utility that's the easy approaches or you might have to extract it from the flash chip on the device itself go to hardware or hacking route and this can become very complicated because in a worst case scenario you'll have encrypted firmware you'll have chip readout protection you need to bypass it and do side channel attacks and all that kind of stuff but for triconex that was not necessary because there was no readout protection on the flash you could just desolder it, put it into an adapter and use a universal cellular programmer and you'd be good or you could just get it from the firmware update utility which also holds it all so you know the second step you have to do is usually device tear down and PCB analysis so we need info on what kind of microcontroller is on this device we need the device functional domain so what is happening where on this device we need to know about interesting interfaces like UR, JTAG, what have you and sometimes we're lucky if it's an FCC certified device you have an FCC ID and you get internal photos and documentation on the FCC website sometimes other people have done your job for you and you have public tear downs and that's very nice in the case of triconex the planning and installation guide has very detailed internal block diagrams and that helps a lot because that prevents you from or saves you the effort of opening it up but in some cases we're not so lucky and we will have to do tear downs and now people who don't come from a hardware hacking background they're usually terrified of it but it's not that complicated in most cases you know like the picture shows it's RAM flash or you Google it and there is this persistent narrative that you know especially among some OT people that you know ICS devices they're all different and operational technology it's not like the embedded devices you're used to but for our purposes it's like all the IoT devices out there that are getting hacked by the billions every day for example take these two PLCs by Schneider Electric the Modicon M238 and M340 usually you have a central processor module which does all the heavy lifting and you have a couple of input output modules which you stitch onto it on the side as you can see on the right of the slide you might have an integrated ethernet connector as you can see on the right example maybe you have a dedicated module that does the ethernet stuff and then connects to the main module over serial link and internally they typically look something like this so you have a couple of IO pins you got your serial link or ethernet link or whatever and then you have a microcontroller in the middle and typically this microcontroller will run the main firmware which has the operating system and some of the application stuff like maybe a web server or an FTP server or whatever and then you typically have a logic handling chip which might be an FPGA or another microcontroller executing the actual programs that this thing is configured and programmed in now this might differ between PLCs because this is a generalization but it roughly comes down to something like this and for TriConnex it looks like this so you have three main processors and they communicate over tribus which is an internal bus that does some voting on input values for consistency and it has a triple modular redundancy architecture to ensure that a vote of two out of three overwrites any errors that are introduced there yeah that's basically what it looks like the main processor there you have it, it runs a PowerPC chip and you have a dedicated other chip for input, output and communication stuff and we'll delve into this a little bit later when it becomes relevant so the first thing you want to look at for most ICS stuff is reverse engineering of all the protocols they talk to because in many cases these are legacy and proprietary protocols usually ports of old serial protocols that have been retrofitted onto Ethernet they control very sensitive functionality like starting and stopping the PLC updating the firmware and so on and you might find a way to get into the device itself by remote code execution here and that's what we want so the first thing we need to know when encountering a protocol we don't know is knowing the packet structure and the semantics so we can do this in a couple of ways and you know this is a very quick generalization we'll go about comparing it to functionally similar protocols that have been documented so if it's a port of an old serial protocol maybe take a look at what that looked like and what you can recognize in there test for common encoding structures like TLV, sequential identifiers checksums, any entropic analysis for fields that indicate cryptographic functionality or you might want to do differential analysis of functional batches of packets so you have like one packet that corresponds to starting the PLC in different kind of conditions and then see what kind of bit fields in the packets change and what you can make of that and like Rob Savoy said, believe it or not if you stare at the hack stones long enough you start to see the patterns and that is definitely true for pcap-only analysis as you can see here it's basically like looking at the matrix now ideally you want to assist this traffic-only analysis with binary reverse engineering because you want your reconstruction to be complete and sound, you want to write a reliable exploit not because you don't want to fuck things up but because you don't want to fuck things up for yourself so a pcap-only analysis can be incomplete, inaccurate or opaque, you know you can have undocumented or rare behavior that you don't see in the field but that's in there and you're interested in, you might guess at semantics that might not actually be what you think it is there might be encryption, compression, blah blah blah blah and most importantly pcap-only analysis damages your sanity so you want to do some binary reverse engineering which does that to a lesser extent so in the case of triconex the tristation software has a communications DLL and this has all the juicy stuff we need it's a single DLL in the engineering software and it has debug symbols so that greatly eases our life and as you can see on the slide I hope it's visible basically all these functions they have a good semantic mapping between the functionality it does like starting the PLC, stopping the PLC downloading the control logic and the function code within the protocol itself relatively easily identify all that functionality and you can see that the attacker probably went about it this way now the benefit for attackers is that you don't need to fully reverse engineer the protocol, you only need to understand a few interesting packet types because we don't want to craft a full protocol parser we want to craft an exploit so I don't care about all the rest I care about like that one packet type that does the good stuff now when it comes to vulnerability discovery which is typically the step that comes after this the next step is getting code execution and ideally this is a pre-authentication vulnerability as we'll see pre-auth is a relative concept in the ICS world and in many cases ICS vulnerabilities are often a byproduct of the reverse engineering so you won't need to go about in most cases about fuzzing or static analysis of the firmware in many cases it's insecure by default there's ancient legacy shit everywhere and you shake a stick at it and vulnerabilities fall out so let me briefly drink something so an example of this for example is the Moxa and port serial to ethernet wifi converter you plug a serial cable in converts it to ethernet it has a web interface with broken authentication so it hashes on the client side that's good once you're in you can do command injection in the ping test form so you got your code execution right there another example the opto it's used in fairly sensitive environments it's got fdp it's got a proprietary protocol without authentication now the thing is you can configure fdp with a password you can enable ip filtering all that kind of stuff but then you can use this unauthenticated proprietary protocol to disable ip filtering enable fdp and ask the credentials nicely you know and then you get them and you can use the fdp to upload a new piece of firmware and reflash the device over fdp and there's no firmware signing another example in case you don't believe me yet the modicum quantum plc that's a large plc for process applications it has fdp with hardcoded credentials which allows you to read and write configuration, firmware, what have you it has telnet with a hardcoded backdoor and that's actually a C interpreter which you know is nice it also has an unauthenticated proprietary modbus extension for starting and stopping the plc overriding the programmable logic there's basically a gazillion ways to get code execution on this thing finally to drive the point home another serial to ethernet converter by adventac it has web interface again and the nice thing is that if anyone unlocks the web interface on one pc, let's say a legitimate operator it's disabled for everyone so you know and then you have a command injection in the email alert setup form and again easy code execution right there I mean you get the idea it's like shooting fish in a barrel right and for trident it was basically the same thing it was an execute my packet police vulnerability so the vulnerability was a freebie of the protocol reverse engineering you have this safety program download functionality which is how the engineers put like the actual logic on the thing and it has no authentication and the safety logic that gets downloaded to the device has no secure signing right so you can basically skip all the way from reverse engineering to exploit development and that's neat from an attacker point of view not so neat from a defender point of view and then that brings us to exploit development here we find the suitable vulnerability and we get our code execution we need to craft and exploit to actually not execute just the logic we want but execute the instructions on the microcontroller we want and how would this look like in the case of triton well triton has safety and control applications which are developed in one of these IEC languages many of them look like graphical language like you can see on the bottom of the slide and they typically get compiled and downloaded and executed on the main processor there and that's nice because that gives you another exploit development freebie because you don't need to break out of any sandboxes you don't need to exploit any run times and because in the case of the triton xcontroller the logic was executed on the same chip that the operating system was executed on you don't need to hop across any chip perimeters you're right where you want to be so triton did have to add some additional functionality so it doesn't overwrite the original programming but it appends to it and the reason why it did this is because it allows the safety logic to continue running without interruption because once you're implanting this device you don't want everything to stop and potentially already cause a process shutdown you want to be stealthy at least at that point and our complication is the key switch on the triton xcontrollers so these devices have a physical key switch that allows you to set it to a certain mode and only if it's set into the programming mode then you can actually download new logic to it which will be relevant later when we discuss the payloads another complication of embedded exploitation in general is the heterogeneity so embedded devices are far more heterogeneous than general purpose ones you deal with a billion architectures from ARM to MIPS to PowerPC to whatever a billion kind of operating systems like VxWorks or QNICS or even custom operating systems and in the case of triton x that means that you have different architectures and different versions so version 9 had a national semiconductor chip version 10 and 11 had PowerPC and for the attacker this means that scaling the attack requires writing and modifying payloads and implants for each different version so there's some effort involved there and that brings us to the development of the implant and the OT payload that's the stage that's close to what Marina mentioned of mapping like the TSD to the implants on the devices now we have code execution we can run arbitrary PowerPC shell code so now what? Well, exploitation is just one step among many so for complicated OT payloads we will need to develop this implant and then after that craft the OT payload we have different kind of strategies here so we can directly implant the OT payload straight away after exploiting the device or we can implement a backdoor which would allow us to keep the OT payload a secret until and basically provides you with kill switch capabilities or in the jargon of some people a dormant cyber pathogen that basically would allow you at a later stage to execute any kind of payload you'd want to the second thing we'll have to decide on is whether we'd go for cross-boot persistence which would require modifying the flash and we need enough space there to insert the implant or which is what Triton did we just go fully memory resident that requires executable RAM but it does mean that on reboot the implant is gone now for safety controls this is not that relevant because these have an extremely high uptime so even if it would be gone upon reboot you know it's probably going to be there all the time anyway this has the added benefit for the attacker that it does complicate forensics the other thing you want to think about when designing an implant is the scalability of the implant so you want to target devices that are common throughout ICS so not only for one particular facility you might be interested in but across different kind of facilities different kind of industries and triconex fits this bill so there are 18,000 or more than 80,000 triconex systems in over 80 countries so having these kind of capabilities is very interesting from a strategic point of view you also want to target if you're targeting software instead of controllers you want to target the software that is common throughout ICS so used across multiple vendors so protocol and connectivity stacks are usually used by multiple vendors so having exploits for that kind of stuff really scales well and the same goes for control runtimes or the operating systems in question you basically want to construct an arsenal of exploits and implants against common devices and software stacks because that means that it is a one-time upfront investment and now there is no huge turnover in these devices these have an extremely long field life they get deployed for 10, 20 years they don't update it very often you know if there are even updates out there so if you invest once in an arsenal for let's say all safety controllers that are relevant in the market right now it's gonna be an investment that's gonna be paying off for a long time so try to make more sense as a tool in such an arsenal than a very expensive one off that was engineered for this particular attack so that brings us to the reverse engineering of the ICS firmware the first thing you'll have to do is extracting the firmware so you have to determine the firmware format and then unpack sometimes there will be firmers for multiple chips on the board or data blobs and they're glued together and you'll have to get them out of the firmware and then you'll have to do the compression and the decryption if there is any of that present this might be simple you know keys might be present in a firmware loading utility or you might have to actually do side channel attacks in the case of TriConX this was very easy because the firmware was unencrypted so you know basically the step could be skipped the step that you do after that is accessing the firmware so you need to obtain a memory map you need to know where on this chip does the ROM live, where does the RAM live, where does the external memory live, the purpose registers for interacting with peripherals all that kind of stuff and you typically get this from the datasheet so very important if you do this kind of stuff you need to learn to love reading datasheets so after you know basically the lay of the land here you go about identifying the base address so you need to know where this is loaded into memory now there are many approaches here and this is a gross oversimplification but we simply don't have the time to go into all the details but this can be as simple as you know being loaded at a chip fixed address you can reverse engineer the update utility and see where it places it or you can extract it from something like the interrupt factor table or the boot loader or self relocating coding code jump table string tables all that let's consider for example this RTU firmware piece this is a piece of firmware for a remote terminal unit that's deployed somewhere in the field it's ARM based and we have this piece of firmware and we don't know what the base address is so we load it just at address zero and then we see all these different basically branches does anyone recognize what this is a couple of people well yeah that's the interrupt factor table of an ARM chip so the first entry is a branch to the reset handler and basically if we look at the offset we have 100 as an offset here and here we have 102 if we look at that offset we see a lot of toggling with like these special purpose registers and that's very typical behavior of a reset handler so if we then deduct the thousand basically that becomes the base address we re-base the firmware image and we can see that it cleans up nicely now we know how to load it into memory so after you've done this you know where to load it you know where everything is laid out you have to reconstruct the complete code and data topology of the firmware. Firmware images are not need executable formats like like PE or ELF or MAC O I mean your mileage might vary for what qualifies as a need executable format but this definitely is not it we will have to heuristically identify the functions in there the strings, the jump tables, the structs and all that kind of stuff on the upside this is usually the bulk of the work doing all this reverse engineering the upside is that we don't need to reverse engineer the full firmware only up until readiness for this next step because we want to hunt for interesting functionality we want to reverse engineering a sniper like fashion so we want to know how does the control runtime work that actually interprets and handles this safety logic where are the protocol parsers where are the communications and peripheral IO handlers where is any security or safety related functionality there so what would this look like for the Tricon X 308 which was targeted by the Triton attack well it's firmware that's power pc based which is nice because the Hexra SD compiler is available for that and that saves you a lot of work it's not a substitute for reading this assembly but it eases your navigation across this firmware it uses a custom operating system with 27 system calls and some sparse documentation exists thank you NRC basically this is what the operating system looks like you have a scan task a communications task and a background task and we're really only interested in the scan task which fetches inputs from shared memory which is where all the analog and digital IO gets put then we do a try bus transfer for the voting on consistency and then we run the actual control logic and then we send the outputs again to the to the shared memory and basically this implements all these control loops so the targets here would have been the memory layout and management because we want to achieve this memory residency so we need to know how to do that then we need to look for consistency checks and diagnostics functionality for implant stability we don't want anything messing with our implant while we're running we need to know where the network command dispatcher functionality is because we want to be able to communicate with it over the network and achieve that we want to be able to know if there's any privilege mode management we want to do anything we can on this device so if there is privilege management we want to escalate them so and possibly finally we need to know the scan task and IO transfer stuff so that's basically what the Triton implant looks like and does so it has four stages it has an argument setter an implant installer the backdoor implant and then the missing OT payload so the argument setter it's not that interesting but it basically controls the finnid state machine of this thing which does at first if you can see that properly it does an exploit for privilege escalation and then basically it relocates the implant the third stage so the privilege escalation exploit it looks complicated but it's not it's basically a right for anywhere because of improper handling of memory privileges that allows you to write this address to this value to this address what does this mean basically what happens is that once you invoke a system call on this particular firmware it saves the machine state register and when it returns it restores it and it is stored at this particular address that anyone can write to regardless of privileges so if we overwrite it with this value we set bit 17 to supervisor privileges and escalate it and that allows us to do these next steps to install the implant the first of which is copying the payload into the firmware area of memory which allows us to achieve residents even if they wipe all the safety logic on the controller then we patch an entry in the jump table which allows us to hook a network command and then invoke our implant when this particular network command is called and then finally we patch a certain RAM check which was used for consistency between the firmware on ROM and in RAM which would otherwise mess with our stuff and this is basically the background implant like Marina mentioned it allows for reading, writing and executing arbitrary memory and basically allows you to overwrite this key switch so once you implant it it doesn't matter what position they turn the key switch into it allows you to execute anything you want in memory so you have persistence on this device so the fourth stage which is the most engineering related is the OT payload and this was missing because this carries out the meat of the attack and we can only speculate what this might have looked like and that's something we'll do so because Triton is positioned here in the attack tree the control and safety system this is what the OT payload would have been related to so basically it could have been for example an IO spoofing scenario and that would have been a scenario that Marina would have given to me and said you know we want to do spoofing of input and output values as you can see for example here you have measurement values they come as an input signal to the controller and as an output signal go to the instrumentation and then we want to mess with either the input or the output to cause some unsafe state now internally this looks like this on these devices so you have the physical IO you have the logic and there is an intermediary variable table that manages this on the triconex this is handled by an IO processor that has shared memory connection to the main microcontroller and that's where it places this variable table so we didn't have a triconex controller but we did have a WAGO PLC and I mean it basically comes down to the same thing it's an ARM Cortex PLC running real-time linux it has a vulnerability in the code sys runtime on tcp that you can explode remotely and gain root and basically what we do in our attack is we use the CPU debug registers to catch any access to this memory mapped IO and then we write a custom exception handler that gets invoked when this debug trap is invoked and we change the pin mode from output to input so any writes that would have been going to an actuator like closing a valve are now going to an input pin and they don't actually go out and we have a little demo of this that should be coming up we don't have a lot of time but alright so what you can see here is you have the WAGO PLC on the right and then you have LED which is our stand-in for a valve and it's supposed to be blinking every couple of seconds and now you can see it now you can see basically on the engineering software and the attacker well soon yeah I can't really skip anything here but there should pop up like a shell at some point now you can see the LED blinking there yeah so you know this is a root shell on the PLC we execute our payload which is basically a linux kernel module that does this thing I just mentioned about the CPU debug registers and then we stop it and you can see the LED will stop blinking while it should be consistently blinking because it you know it now thinks it's an input pin and then at some point you know we'll change it back again to an output pin and then you can see it's starting blinking again right if this was a valve it would have been having had an impact now it's a LED and you know it's just a demo so that brings us to the second possible payload which is alarm suppression so let's say we want to mess for example with a chemical process like this and our goal is catalyst deactivation now industrial processes they have alarms so if something is about to go into an unsafe state or something is out of bounds an alarm is raised somewhere in the process and these are propagated throughout the process and eventually lead to a safety shutdown and as an attacker we want to prevent this because we want the attack to continue so how do we go about this we can hide the alarms by compromising like the safety view or the DCS view or some local HMIs and this is the benefit of you know hiding them centrally but still these alarms go out as network traffic right and we don't want any kind of network inspection to see these alarms so we want to suppress them either at the field level the level of the field devices or at the level of the safety controller itself which is what TriconX is which is a PC based HMI solution that allows for management and bypasses of all these alarms and each HMI function is mapped to a TriconX logic function block that looks somewhat, that was a little bit too quick that looks something like this so you have for example a water tank level alarm here you have a water high, a water low signal and an aura of them raises an alarm so as an attacker these safety programs reside in memory as code which is how we got code execution in the first place so the OT payload here could modify the instructions to set the alarm to a fixed vols regardless of the water levels that are coming as inputs to this alarm function the attacker needs to know of course first where the program lives in memory and which instructions of the program to modify now luckily for TriconX these programs are stored as a circular linked list in memory as you can see here this is actually our implant living in memory and then you can walk this linked list and find the target program you want to modify and this is what basically the alarm looks like the core of the logic is a simple OR instruction which the attacker can then hot-patch, you know it's a risk architecture with a fixed instruction size so we just set it to a fixed vols there it's relatively easy and this is the result that comes out of it on the left you have normal functionality water is getting dangerously high the alarm is going off on the right you have the exploited functionality and the alarm is sleeping so some more speculation why did this attack fail I don't think we mentioned that but the attack was discovered because it failed in the actual field it caused a safety shutdown of the process there are a couple of possibilities you could have had a Borg payload the privilege escalation could have failed it could have targeted a different firmware version than they actually had developed the exploit for which could have caused an access violation the backdoor allows for raw reading, writing and executing of memory they could have caused some memory corruption a payload that was written wrongly they could have gotten into a fight with the watchdog so very common on embedded systems you have this watchdog timer and you need to periodically kick it to keep the counter high because if the counter expires it resets the CPU or they could have missed some additional diagnostics there's a ton of diagnostics functionality in here because they patched the RAM and ROM check doesn't mean that they patched everything rightly these are all options another option is that they got into a fight with the triple modular redundancy I mean this is just the voting on the inputs and the output but it's a very complicated patent that covers a lot of ground and it could have been that the OT payload violated something in the triple modular redundancy that in turn led to a safety shutdown that brings us to the conclusions which I'll give to Marina again we probably have just like a couple of minutes for conclusions so just to summarize the purpose was also to go through this process of development implants for embedded system and see whether it is really that complicated so what is the actual real threat is it also only for elite and we should not expect this type of attack or implants on a mass scale or it not and apparently it is not not only that exploitation of all systems relatively easy like it's cheap because there is no any exploit mitigations but also obtaining equipment relatively easy we went through the process obtaining documentation is relatively easy so you kind of have all of the needed components to quickly obtain code execution and be able to backdoor this type of devices so exploit development again easy if I will just go quickly through this implant development relatively easy so for example the case of Triton the attacker already had privileges typically on industrial control system you would not have like separation of privileges so the attacker had privileges to escalate his privileges neat so however for example in case safety system you typically have this triple redundancy sometimes it's quadruple redundancy which makes things a little bit complicated because you have to reverse engineer more so the most difficult part which we were able to establish is actually developing OTP so basically my task is harder and especially whenever we come together like for example Joss and I come together this is my wish lift and he already has an implant this is where reverse engineering started going further and that is becoming more difficult so so basically yes so the open questions which we have and this is I think Joss has covered it very nicely that it's probably what we've seen is just one tool many so basically development costs should be seen as relatively low and I think we should what we definitely should expect relatively soon is copy compared security is a fashionable industry so as soon as you like for example somebody will release a first ransomware malware everybody will be designing ransomware malware so this is what we're going probably to see so many more other like state sponsored threat actors will join this basically game and again it's not necessarily so this is what like people say on one hand yes it's easy to exploit then why don't we see a lot of those attacks because first of all you have to have a motive why would you do that and secondly it's of course like crossing this red line so on one hand it probably will be something like the nuclear weapons you will have the arsenal and maybe you will be showing the capabilities just in a slight manner like it has happened with power grid attacks in Ukraine you just show that you are capable of but you do not cause massive like damage which hurts so you basically will be having it as a like economical and political negotiation so we wanted to thank the couple of people including Felix Lindner who was like a frequent DEF CON speaker who kindly helped us to review the slides we have the slides with the toolkit which you will need and with that we at the end and thank you very much for your attention


The presidential candidate of the Labour Party, Peter Obie, has emerged the Man of the Year in the 2022 Leadership Excellence Awards. In a 14-day Leadership Excellence Awards voting by Nigerians, both home and in the diaspora via online platforms, Obie was voted for ahead of three other nominees. Obie emerged the winner in the voting which took place from August 1st to 14th through online platforms. Hello, hope you enjoyed the news. Please do subscribe to our YouTube channel and don't forget to hit the notification button so you get notified about fresh news updates.


Okay, we are back here, live in Silicon Valley. This is a special presentation with theCUBE, Silicon Angle Media, Wikibon on theCUBE, presenting Big Data SV. The theme is Follow the Money. And our next panel is with the people who write the big fat checks, the venture capitalists, and we stalk them, follow these guys, we'll just follow, we'll find out all the action. So I am super excited to have with me Ping Li, who's a general partner, an Excel partners, friend of theCUBE, has been on theCUBE many times, and he's on the Midas list of Midas Touch investors, also an investor in Cloudera and a variety of big data companies, and we've been following Ping's career and certainly the big data investments with Excel over the years, and been fantastic to watch, and certainly got a great perspective from Ping. Frank Artali is managing director at Ignition Partners, former executive at Microsoft, you're familiar with NT, you all know what that is. Frank was a big leader in that initiative and has a lot of great history, also an investor in Cloudera as well, a variety of other investments. So guys, welcome to the panel, so follow the money, that's the theme. A lot of money's being spent, that came up earlier on Twitter, where's all that money going, and why isn't there more billion dollar companies in big data, where's all the apps, where's all the exits, and certainly Ping, you've had great success with Cloudera as an investment. What's going on with big data? It hits, misses over the past five years, what's your view on it, are you happy with it? Progress, obviously Cloudera's a success since that point, but other investments in general, how would you grade the ecosystem in general? Well, I think that my first observation is, I think it's over here early, despite all the activity and conversations around big data, I think this is a technology cycle trend that's gonna last much more than five years, 10, 15 years, right? If you look at the relational database market, how long that took to kind of ecosystem to get built and get formed, I mean we're talking decades. Now I think technology cycles happen faster now than they did in the past, but I still think we're in the first, second, or third endings of kind of this big data build out. And if I look backwards, I think a lot of the innovation has been around the data management platforms, the Hadoops, the Sparks, and all the underlying technologies that I think will drive a lot of the next generation platforms for applications to get built on. And frankly, I think if you look forward, the application ecosystem on these new platforms is still very early, it's still very nascent, right? Whether it's BI tools, ERP tools, CRM tools, all these different applications that we have used over the last decades to make our lives better as a business user productivity. I think they're all gonna get re-platformed and leverage a lot of these new big data technologies whether it's a duper spark or whatever in order to deliver better experiences for users. And that's just gonna go. So re-platform has been a theme we've heard. So the question for you then is what were the hits and misses? I mean back then, I mean first of all, it was Wild West back then, and Hadoop in 2010, 2011 was like no one even knew what that was. So you made a good bet on your thesis with, say, Cloudera, but what were the hits and what were the misses if you'd look back? And talk a little bit more about that re-platforming and where that is now. Yeah, I don't, I think it's early to predict hits or misses in our business. You never predict these things until they actually happen, then you seem really smart. They're still swinging right now. Yeah, I think it's really early. Yeah, there'll be some long deaths. There's gonna be long deaths. You know, look, I think early on there were 50 different types of NoSQL databases. I think that's definitely a much smaller list right now with Couchbase and a few other ones that have seemed to have getting a lot of traction. So I think that in of itself has consolidated. Early on there were a lot of different ways to do batch processing. And I think Hadoop kind of became the one that embraced a lot of it. There were 15 different file systems. So I think a lot of things have consolidated at that layer. I think the things that at the application layer, a lot of them have gotten off to slower starts just because it took a while for the underlying technologies to take hold. So we're looking forward to seeing how some of those evolve, whether it's some of the BI players. Yeah, I mean, I'll just add to this too is that in terms of what we, many people think about these things as being databases, especially in the situation of the Hadoop, let's just say stack. I mean, what people should really understand it's more like an operating system for data than just simply a store. Yes, it does storage, but it does an awful lot of processing. So it just takes a little bit of time for people to make the transition from their thinking around the relational world to what we're in right now. And this is not like the time back in say the early 1990s where we made the transition and folks like us were around back then, made the transition from mid-range and mainframe systems to distribute it. By and large, back then it was really just getting the same type of tech, if you will, deployed on smaller systems and deployed. What we're dealing with right now is really a completely different enabler. So for someone who is working on IMS on a mainframe system to work on an index file system on Unix, it wasn't as much of a leap as someone going from say something like SQL server today to using something like cloud era. And so the echo pink statements here, it's we're in early days and to say that there's been swings and misses. There's certainly been a lot of swings, but the misses are still, I think those stories are still to be played out. But if history repeats itself, we do have consolidation. We had tens of databases back in the 90s and then you end up with about three, right? And so we'll see what happens, but we're pretty confident that it is early days and this is a, let's just say an area that we look forward to more on the enabling side. We don't want to hear about problems. There aren't problems here. These are about creating businesses which couldn't exist otherwise. So let's talk about the platforms for a second, because I think the replatforming brings up your point about the old days. And the old days, platforms were land grabs. Microsoft, where you worked, had a great platform, had an operating system and application monopoly or a competitive advantage, I should say. So with now with cloud era, for instance, when I talked to Amarawa Dala when you first did the investment guys, he was a scale out guy from Yahoo, basically said scale out's the future on commodity hardware. Yet all the big whales, as Jeff Kelly talks about, are scale up guys. So I want you to talk about this notion of what horizontally scalable cloud and scaling up coming together in that platform. Is it ownable? Is it an opportunity? Is open source change the game? What's your take on that? Yeah, so just like to look at, again, how things are, we never like to say it's different this time. But the enablers here, right, the thing that's something like Hadoop as an example that we're using or things like elastic databases like this elastic processing enable is that you can think about a customer or an organization or a scientist or anyone can think about having an application that can need at a given point in time tens of thousands of nodes of processing.  Even if you say, well, I'm gonna use cloud resources. You know, it's really hard to go kind of string that stuff together. And so when Amar talks about using scale out and elastic and commodity parts, he's saying that this kind of technology enables applications that just couldn't be thought of in a scaled up world because you have to plan for all of that capex upfront. Now with the availability of cheap cores either on something like Amazon AWS or Microsoft Azure, you can just go after those cores at the time you need them. And so it really frees up the people's minds in terms of the kind of applications can be built. So I mean, now there are times where maybe a scale up thing would have certain advantages instead of sort of the old world where you had certain types of query processing that required memory locality but those things were different. They came out of the relational world with certain types of SQL calculus which let's just say don't apply in these different types of applications. I think, don't underestimate Moore's law. I mean, I think what Intel continues to do in the data center has provided over the last decade has created a whole new ecosystem of building software applications and software platforms, right? And I think that's, and we're just harnessing, this software is harnessing all that innovation and translating that to the applications now, right? So I think, if I look at our portfolio and the type of applications we're building, they're assuming really, really powerful, efficient commodity hardware that can scale out and they're looking for software platforms and they're not really databases anymore, the data management platform that can harness the value of this at scale, right? So I think that's the innovation that's already happened, right? So now the question is, how do you package all that hardware and software and deliver that into an application for an end user, a business person to actually get value out of it? So what are you guys investing in that? In terms of when someone walks in the door, we were talking on theCUBE today with the entrepreneur, he said ML is the new SQL, machine learning is the new SQL. Things like that are happening, internet of things happening and you're talking about in memory with sparks, recent success in silicon analytics, insight engines, all those stuff's happening. It is an operating system that's going on on the internet called cloud or whatever you wanna talk about it, but as investors, how do you make sense of it? What are you investing in? What's gonna get your attention and from the money making standpoint, beyond like the small, that could be a double or single, what's the home run pitch that you're looking at? I'll start. This doesn't apply necessarily to big data, I mean just in general, if you can pick a point in time anytime what folks like us do, we try to look at things that are applicable to let's just say all businesses or businesses of all sizes. And so if you look at, so we use history as a guide, right? And it doesn't always necessarily repeat itself but it sometimes does rhyme. And you find all the things that might have happened in the past and you attempt to make, we attempt to look at things we can make a bet on things that sort of mimic things that happened in the past. And so if you looked at things that happened, let's just say around the database, although we're not a database in this context, more like a complete data management or operating system for data, what are all the things that happened in say, when clients server took hold? Well, you have productivity tools that suddenly hooked up to databases, right? And that freed up the data from someone requiring like a software developer to go write a program. So we like things that's kind of like free up that data. Then there are types of productivity applications that are enabled because of the availability of data. And so as people start living in data and we like to look at those things. But in our minds we always ask ourselves, is this a thing, like a class of thing, that any company would say more than 1,000 employees would be likely to purchase one of or need. And that may be something that affects the end user or something that makes the infrastructure protected or secure, something that makes the IT professionals life easier, they're not going away. So that's sort of how we look at things or how I look at things I know. Ping and I talk about these things all the time as well. We're much, again, using history as a guide is I think in the venture world people say pattern matching is something that we do. Your home run pitch, fastball, care ball, what are you looking for for ideas and things? I have to be, I think right now, pretty much every part of the enterprise stack is available for disruption to build a massive company. We are dealing with changes in the last couple of years that I think are incredibly profound, right? Starting from the scale-out stuff we're talking about, where hardware and compute now is kind of pervasive and elastic, right? We're talking about cloud-based software that can take advantage of that. We're talking about mobile platforms that didn't exist before. We're talking about operating systems. We're talking about containers. I mean, there's a complete rejuvenation of the data center. There's a foaming of the mouth ready to write checks. It is a huge opportunity to say. No, I think there's constant talks about the bubble and the tech and in my view is that there's going to be some amazing companies, large companies get created because these are fundamental shifts that are not going back and they're very horizontal in how they impact people. So you're saying there's waves coming? There's a lot of waves coming. This is one wave now, so let's go into that. So I got to ask you, so I was there when you announced the $100 million big data fund and the thesis there was, oh, there's going to be a lot of apps. I think that was like three years ago. I also had a chat with Pat Gelsinger right before Paul Moritz went to Pivotal before they spun that out and they had this big vision, mainframe in the cloud. It just never happened because there was other things that needed to get fixed which kind of stalled the plan. So I don't think the thesis was incorrect but I want to get your perspective on the timing, right? I mean, the apps are coming, you mentioned that. So what's your take now? I mean, obviously network virtualization, VMware's working on that, Pivotal's having their success. Is it the same trajectory? Is apps still going to be the big thing as people realize they have all this data and they're data-full and they have solved their own problems? I mean, is that what you're looking for? What's your take on that trajectory of the apps? I think data-driven applications are just starting to emerge and applications take time to build but one of the companies that we were able to work with last year, RelateIQ, that ultimately called by Salesforce was essentially that, right? They were building off Hadoop, they were leveraging cloud software, running on a scale out hardware and they were processing all the data and redefining CRM, right? They were re-looking at how people interface with their relationships and their contacts and how to build a network and it was all data-driven. It wasn't built on the legacy stack, it was built on the new stack and the way that I've always thought about it is, look at what's happening in the consumer world, from web 1.0 to web 2.0 or whatever version of the web we're on now, the Facebooks, the Twitters, all these guys have built their applications to be data-driven from day one, whether it's the feed, people you may know, that experience is making the user get more value out of these applications. If you look at the enterprise apps, they don't do that. No one likes their enterprise applications, no one gets any value out of their enterprise applications and I think that's starting to change because they're using the same stack, the same technology underpinnings that the consumer guys, frankly, invented because they had no choice, are now being used in the enterprise. So I think a lot of these new, we're going to use just one example, we're seeing a lot of other people that are using these new technologies to build very innovative applications. So why did they sell? I mean, would they sell because they were a feature of someone else's or the global expansion? I mean, this is a question we were talking about on theCUBE today as there is M&A opportunity there's not so bad thing in exits and exit. If you can make it globally, then you can be that billion dollar company. If you can't, then you might be acquired. So that's a reality that people are facing. Is there a reason why they got acquired because it was a better fit for someone to fill their portfolio? Is that a trend that you see and see more of? Look, I think a lot of the incumbent companies are going to have to do M&A in order to kind of stay ahead of this change. And I don't fault them. I mean, if you look at what's happened, some of these trends have happened so quickly. Like five years ago, maybe two companies in our portfolio was built on AWS today, 99% of them are.  Five years. These things have happened so quick that I don't expect the incumbents to be able to react to some of these changes. Well, buyers out there like the IBMs and the VMwares. They're buyers for the startups. You guys there. I mean, so they're looking for innovation and Frank and I were lamenting earlier, we hope not all our companies sell because I think we think some of them have a lot of legs to build really interesting things. Frank, the consolidation is not necessarily a bad sign. There's more waves coming and the market's shifting. Consolidation can be a good thing, right? I mean, what's your take on that? I mean, do you agree that it's not necessarily a bad thing if Jeff Kelly's research is somewhat accurate? Yeah, so just in terms of consolidation, when a class of technology becomes more mature, buyers get comfortable when they can actually buy something from a company of size, right? So typically what we've seen in the last few years or the last 10 years is that large buyers of technology, again, enterprise IT organizations, are willing to try things from smaller organizations, but for them to make them a big bet, they like to go with a leader. And so consolidation will usually produce a leader. And so it is definitely a good thing as long as you're invested in the leader. Well, cloud also requires speed to market. So I've heard some from the big buyers of the startup saying, hey, if they can't go global, they're in a bridge to that. So it can be a win-win in that case. Do you see that happening in most of big companies like the IBMs of the world? So is the question in terms of? On startup, and I'm like, okay, I got some venture funding. I either got to get sales quickly or look for a buyer or this is the classic probably board meeting that you guys go out and walk into. So to be clear, right, like Ping and I don't get paid for getting a company started, putting one round of investment and making a small amount of money for our investors. That's not we do. When we make an investment, every investment we make, we're thinking that it can be a global standalone company. We don't make an investment and say, oh, this would be a great thing to sell to XYZ buyer for 50 million bucks. That's not what we do for a living. There'll be a consolidation of venture firms very quickly if that was the norm. Now that being said, at some point in a company's life, the management team together with the investors do make a decision and say, hey, what is it gonna take for this company to be a durable standalone entity?  What kind of company is it? How much capital is it gonna take? Can it get there in a meaningful amount of time before maybe a larger company wants to come and do the same sort of thing? But by and large, if there's a large enough total available market and you're part of one or two first movers for a high technology where customers are willing to pay and get behind you, then you can usually attract enough capital to keep going and get to be that company, at least to have the perception of being a global company. And it does take a while. I mean, look at companies that are really now just barely global companies, even like Citrix Systems, a place that I worked, even today, like most of its revenue is still, they're probably still overweight, right, in U.S. versus the rest of the world, probably at least I would say two thirds, as opposed to if you translate that to a Microsoft, which is only 28% U.S. Okay, so I got some questions from the Twitter sphere. So a couple of them coming in. How well have you guys done backing startups that build tools, not apps? Or can you comment on that tool thing, and I'll throw in my little two cents on there. Do you have to have a platform, or can you just stand alone with an app or a tool or is platform kind of an inherent, you know, substrate within the app framework? I mean, I have so, yeah, so in terms of just hard examples in the Ignition portfolio, one of our top companies is actually a tools company, it's called Xamarin, it starts with the letter X. Great couple of guys building a mobile application development platform. It'll be actually more than that. It'll do some more things at the runtime and what have you. But it is something which is sold largely to enterprise IT folks that are building enterprise mobile applications. And so that's an example of a tools company that we like. We have not had success with anything that's sold, let's just say software development frameworks or tools to other professional software developers. Because those are the last folks that like to pay. Yeah, and I think the reality is even the tools companies that do become widely adopted end up becoming platform companies over time. So if it just stays as a tool company and it doesn't get broader in terms of the surface area of the problems it solves, then it's probably hard. So you bring up a good point. That's not a bad thing. You can come in and enter the market on a narrow scene with a tool and then sequence to a platform position. No, a lot of times some of our best companies have started office tools and people are like, hey, let's just use this thing. What is this thing? Next thing you know becomes widely adopted. I've always said if you say you're a platform you gotta target on your back from the day one. It's hard to sell a platform. No one knows what to do with a platform. Especially if it's a new category. Awesome, so a fun question came in. When do you see artificial intelligence hitting the scene? Obviously with wearables, internet of things. How far away from full artificial intelligence if you can even define what artificial intelligence is. So I'm sure you get a lot of guys coming in and talking about the big idea. So you seeing AI hit the radar? We do see it. Well, we do see it. We definitely see it. It's unclear. See the ideas or not. Well, it's all depends on what you consider AI. But obviously in the gaming world and Anki, the iPhone car, I mean there's a lot of different applications. What are the horizontal scale ones? It's still a little early. So the next question is what project with the next Spark? No pun intended in the Hadoop area. What's Spark? What's the next kind of big wave or idea you're seeing that's enabling either creativity, innovation, investment outside of, say, Spark? So an open source project or just a way to throw dollars at where wealth can be created? Yeah, I think, well, I mean it's just, that's a really broad question. I think it goes back to some of the things that Ping was saying earlier around applications. And so I'm not saying that the investment area is over for infrastructure. But if I had to look at two equally exciting things and one was infrastructure and one was an application that made all of our lives as professionals easier, I'd tell you which one I'm investing in. The one that's, yeah, the app, yeah. Well, I guess, go, kid, what's the name of the company again? No, I agree, I think it's awesome to see all these innovation at the infrastructure platform layer and I think some of them will pierce through. But when I talk to CIOs, they're just digesting what's currently there right now and getting value out of that is enough of a challenge today and finding the use cases and the workflows to really leverage those platforms. So I think the more applications, as Frank says, it helps the digestion of the existing stuff would be, I think, the first order. And then I think then there's not always new applications break the platform, then there's new innovation there. So this thing will just keep going. And CIO said to me one time, he said, if someone walks in and sells me another platform, I'm gonna shoot myself. So there's a lot of digestive pain from the million dollar, 10 million dollar platform, rolling it out. They want more like Amazon. They wanna stand up something quick. So you've seen that same thing? Yeah, I mean, I think it's getting increased. I mean, the thing that Saul comes back to is agility and velocity of development. If you look at how people are building applications today, it's a very different world than it was 10 years ago. Okay, so I gotta put you guys on the spot for a fun question. Cal or Stanford in terms of innovation in Silicon Valley? Computer science departments. Which ones you've seen the most action come from? Let's see if they answer this one. It's a tough one. Actually, it really depends on what. It's also doing really interesting things. Yeah, I mean, if you look at the AMP Lab at Berkeley, it's been phenomenal in terms of the innovation. The output has been really great at Berkeley, for sure. I still like, I'll take an alternate. I think, I think. Washington, because you're from Seattle? No, I'll say Cambridge University in the UK. If you wanna go have a trip and check out that computer lab, that's a lot of innovation there as well. So that brings up the fun question. Is computer science obviously changing permission? You see how I ducked that? I mean, that's a good question, yeah. You can't win with that when you're at VC. Everything's great. Computer science is great. What disciplines are you seeing in terms of right now? I'll say computer science has been the rage, engineering degrees. Are you seeing cross-disciplinary? We had, I'll say machine learning brings up math, computer science. Is there new faces, new disciplines you're seeing? Walk on the door? Design. I think design is probably one of the most needed talent right now in building killer applications. I think that's, if I look across our portfolio, people need more data guys, they need more design folks. And design hasn't evolved much beyond colors. Two fields and a button. Yeah, this is now how to create and experience the part programmers, the part artists, the part creative. So I think it's a really difficult skill set. Okay, we're gonna get some questions from the crowd so we have the microphone running around. But before we go there, my final question is the big data cloud, you can stand up and start it pretty easily. Open source has been a great thing. We're seeing amazing first class citizens with open source in the enterprise now. It's awesome for entrepreneurs. So what size checks is a series A? What are you guys writing checks?  I mean, entrepreneur comes in and says, hey, I only need a million fives. Is that too small? Call me when you need 20 million. Obviously, you know, Andreessen Horwitz throws a lot of cash around. What is the round size? Is there the pattern out there? Do you need to see something?  As a series A, what's the average sizes that you guys are put into work? Cause your time is limited, right, you guys? So what's going on in the VC world? And then we'll open it up to some questions. Size of checks, your time, feeder networks. What's some of the landscape thing going on VC? I think it's better probably to look at just a broader set of data than what's coming from just from two firms. And I think the median data shows that series A, I know this is kind of a wide range. Four million, more like typical towards eight to 10, if it's towards the higher end of that. And but I think how, at least how we try to look at it, we don't look at it so much as a check size. For us, of course, we're motivated by greed to some degree. We wanna own as much of the company as we can. But also we look at what it's gonna take for the company to get to its next milestone. And that next milestone will logically be a series B. So for every company, it's gonna be a little bit different. If it's maybe something where it's, some folks that have a unique algorithm around doing something, maybe it's only gonna be a handful of people working for 12 months, that's gonna consume a lot less cash than something that's gonna be a big infrastructure play. So the answer is, it really depends on the company. Yeah, I mean, I always say the business strives the funding. The funding doesn't drive the business. So I mean, last year I did $2 million series A and I also did a $15 million series A. So there's really no, two very different businesses though, right? So they have different requirements. But there's a lot of fun data that you can go after that you can go after and you can get the median and average numbers. I'll give you some. Well, you know, I didn't ask any questions on the bubble. I know you gotta run. So I personally think it's very bubble-ish, but the waves are coming. We're seeing it all through the industry. So as long as the waves are coming, it's great. Any questions from the crowd? We got a microphone for the guys writing the checks, following the money, these guys there. I'll speak of a comment on the bubble thing. So I mean, in any given year since we had server computers, have we consumed less cores than the year before? Of course not. And as long as that trend continues, this is the industry to be in. Yeah, I agree. It's a great growing market. Question from the crowd. My question is that, when will enterprise app marketplace take off? As you know, mobile app marketplace has been there for many years. I'm talking about enterprise app on cloud. Enterprise apps on cloud. Mobile's been around for years. When's that gonna take off? Yeah, things like, you know, I'm not sure you guys are familiar with AWS cloud marketplace, AWS marketplace. There are a lot of enterprise apps. Google is working on that. There are others like that. That's for infrastructure and service. And for Salesforce, they have SaaS marketplace. It's called app exchange, right? Things like that. I mean, I think it depends on what categories, but a lot of them are taking off, right? Slack and the messaging space is an example of enterprise app that's exploding. So... Amazon's had great success with their marketplace. People can download apps and they get low cost of sales. I mean, it's fantastic, right? There's been, you know, slow marketing apps on top of force.com that have done very well. So, you know, I think, you know, and I don't think the mobile app and enterprise app ecosystem are different. I think right now every app that we look at on the enterprise side is mobile first out the gate. So I think those two worlds are converging as well. Yeah, and largely for in-house written enterprise applications, there are certain challenges related to access to data securely. And but largely, if you look at some of the way that the networking infrastructure is being enhanced in the public clouds, that a lot of those challenges will, let's just say, become diminished over the course of the next year or two. You have an A that was sufficient to get you to the milestones for a B. Traditional B milestones are repeatable sales model and revenue growth. Any more color on where you have to get to to qualify for a B? So since I answered the question earlier. So for me, when I think about, again, depending on the size of the A, now sometimes if it's a supersize series A, you may even want to get beyond that. But in general, you know, the way I think about a series B is getting to product market fit. And there has to be some sort of metrics around that. So it may or may not be a revenue number. I think we have time for one more question. Because I know Frank's got to catch a plane. Appreciate your time coming on. But one more question? Over there. To the right. All right. Hi, thanks for your time, guys. So just a question about companies which go, IPU and companies which try to remain private.  But if you look like pure storage of some other companies, they have new tonics, they have like a huge market cap and they're still private. So what are the thesis or their thoughts behind it? This is behind going public. Well, as fast as possible. Yeah, some companies try to go as fast as possible. Some are trying to be private. That's a good question. What do you guys think?  Is it a window issue? Is it that they jump? I'm new to the business. I haven't been up and bowed in an IPO yet, but. Well, I mean, the reason they go public is because you need to raise money. I mean, you have no other, that's why companies, big reason why they go public is they need to raise money, right? So, you know. And the PCs can get liquid too, but they don't always sell. Yeah, a lot of times you're locked up, et cetera. So I think that's the primary motivator to go public, right? And then if you can raise capital as a private company, then that's the question of, do you go public or do you go private? And a lot has to do with kind of what the company wants to achieve. Have you raised $300 million on a private round? That's like the old days of going public. Yeah, I mean, the reality is going public has certain constraints to the business, right? Michael Delta is coming private because he didn't want to deal with the other. Right, so I think a lot of companies, I feel like they still have a lot of innovation. There's things they want to do without the constraints of being public, will stay private for a bit longer. The hassle is like in reporting and attention, right? Well, I mean, it's like when you operate, again, when people say going public, the actual, the way to really think about it is operating as a public entity is what it really means. And that means that there's, like you were saying, there's levels of disclosure. There's some benefits of that. Some folks feel that a company that is operating publicly by definition is more transparent and can be more trusted, but that really is meaningless since most of the larger private companies effectively operate as public entities from an internal governance perspective. So I think when people say, well, it's just a hassle to be public. Once you get big enough, you're really operating that way anyway. But I mean, this is just another pool of capital. And it's- You still have the certain hurdles to go public though. You can't just go public for- I think the point Frank made that I want to emphasize, going public is very different than being successful as a public company. And I think time will tell which companies are actually going to be successful as a public company, which means you need to have a real business model. You need to be predictable and needs to be durable and all that stuff. Uber just added another billion to their private round today. So I think a lot of companies stay private because they want to make sure they have all those pieces together. So when they go public, they can be successful for many years to come and continue to grow. So I always say like, don't look at the date the company goes public. Look at them in two or three years from now and see how they did. Guys, thanks for your time Frank. I know you got to catch a plane, but I'd like to give you a final word. What's the outlook? Just share with the folks watching and also here in the room and the big data industry is changing. Obviously you can add in mobile apps and other fabric kind of stuff within the big data world. I said, operating system mindset. What's your final walk away guidance, advice in terms of you're looking at writing checks, you're evaluating the landscape. What's your guidance for this industry? Yeah, and so every day I look at myself at the end of every day and ask myself what I've learned. And I learned something new and that makes it a good day. When I can just tell you it's for the last, I guess I first met the folks at Cloudera, I guess it was maybe in 2008 timeframe. And it really opened my eyes to a set of things that I just thought were not possible from an application perspective. And just my parting thought is that don't accept, you can't accept the status quo. You have to think bigger and it's these big ideas that were just not possible because of things like cost, because of size, because of speed. So we don't have to accept the limits anymore and or at least in the current timeframe and also some of the things that Ping was talking about in terms of what the folks at Intel or other microprocessors folks do around in terms of Moore's law. We're gonna have an opportunity right now to think bigger than we've ever thought before. It's not just about operational savings. That's unfortunately the world that Enterprise IT lived for so long. We're in that replatforming now. So I think it's just the most exciting time to be an Enterprise IT in a long time. Ping, your final thoughts and guidance and learnings and advice? You know, I think just to add to Frank, I think it's an unbelievable time as we said earlier to be an entrepreneur in the Enterprise software IT space. So I think the only thing I would add is it's the team is so important. So if you're starting something or you're joining something, I don't think you can overemphasize that dimension because every good idea will have lots of people doing it. And it's the team that has those instincts and the subtle ability to see around the corners, two steps ahead, et cetera, et cetera, are the ones that are gonna build a great opportunity. So that's one thing that we constantly obsess about is finding those entrepreneurs and that can build those teams and those teams that can build those great companies. Guys, thanks for your time. I know you're really busy. Give everyone a round of applause here for the venture capital, Bing Lee, Frank Artali. Okay, now we're gonna adjourn to the party room next door. Thanks for your time. I know we're a little bit over. I appreciate your time and thanks for everyone for coming for Silicon Angle, theCUBE, Wikibon.