Live from San Jose, California in the heart of Silicon Valley, it's theCUBE. Covering Hadoop Summit 2016, brought to you by Hortonworks. Now, here are your hosts, John Furrier and George Gilbert. Okay, welcome back everyone. We are live in Silicon Valley for Silicon Angles theCUBE. It's our flagship program. We go out to the events and extract the signal and the noise. I'm John Furrier with my co-host, George Gilbert. Our next guest is Jock Eistak, who's the data engineering of America's Pivotal. Welcome to theCUBE. Thank you for having me. So, Pivotal, modern applications. You guys really, I mean, from all indications, there's a line around the corner to work with Pivotal. Because you guys have essentially been doing Agile, really before Agile was Agile. That is correct. But there's some really interesting things happening with DevOps going mainstream. This modernization with Docker containers allows for rapid acceleration of value creation in the app developer space. Correct. Which is one of your core competencies. With cloud, it's the perfect storm. So, can you share for a minute just how that ties together from a Pivotal perspective and what that means from a customer? As they look at the excitement, their eyes are popping out there, but they see the complexity, they go, where do I start? So, back in 2013, when we formed Pivotal, we really put together this trifecta, as you said. So, Agile, DevOps was clearly going to be the mainstay. Everyone was coming through and nobody could do it better than Pivotal Labs. And then in order to really iterate, we have Pivotal Cloud Foundry, which is based off of the open source Cloud Foundry. And that allows us to have a platform as a service to deploy those apps in a Google-ish, Facebook-scale, but for the enterprise. And then what powers all of those apps, putting it all together, is that perfect trifecta of big data or data in general. And so, out of our portfolio, we have three main data products, our Green Plum database, our GemFire data grid, and then our HDB database, which is based off the Apache Hock version. And that allows us actually to have really the every piece that enterprises are looking for today, in order to modernize where they've been for the last 20 years, moving to the future. What are some of the table stakes for an enterprise that wants to accelerate that front end and kind of drive some business value top line, not just cost-cursing, really the innovation strategy? What are some of the table stakes that they need to have in place today to go down that road? So, I would say the first thing you really need is a change agent. So, time and time again, without somebody who's really willing to put their neck on the line and actually change your organization, it becomes very difficult to do. We tend to take a very iterative approach, so we try to introduce ourselves in either a pocket, a couple applications that need to be refactored, a couple new applications, a certain data platform that needs to be sped up or perhaps modernized or perhaps extended upon, but in an open way. I think you'll agree with me that the world has moved on beyond just having a proprietary system that they buy into. They really want to have an open platform that they can rely on that's going to be there for the long haul and longevity. Once you have that, a lot of it has a do with process. So, I think a lot of organizations have put together a process that is mirrored in legacy and that legacy needs to change as well. You need waterfall. Waterfall, but waterfall's one, clearly, and I want to say process, even outside of your classic application development, but process in general. So, there are folks, when I go to an organization that when I leave, I think their whole job is really to say, no, we can't do that. So, we need that process to change so that they can actually compete in today's market. And you see time and time again where organizations are looking around and seeing their markets disrupt in the case like Uber. What's the change there? It's shadow IT is the fact that the person who said no no longer has the power to stop a good idea. Is it because they can stand up an app and say, here's a prototype or is it just more business value? So, I was just going to say, the word that comes to mind for me is success. So, when the business sees success, success begets success. And so, I think you see it time and time again. So, start small, think a small project, get going that way. For sure, for sure. And then it just blossoms. And I'll say the other thing that, to your original question, I think what happens is in an organization there's a lot of caution. And if you look at a developer, you look at a DBA or a system administrator, what they want to do is, they want to see the fruits of their labor actually be used and drive value. That's what they actually want. And so, when they see somebody having success in a small spot, they want to be there too. And so, I continue, it's just snowballs. What's the data angle for you guys? Because one of the things I want to get drill into is, how do you talk about the valuation of data? Because now you've got IoT coming down the pike, you've got all those new processes coming in, unknowns coming in. You've got to be kind of set up, not to foreclose the future. You want to take advantage. So, is that an architectural challenge? Is it just in general concepts? How do you guys talk about that development phenomenon? So, I think we start with, you've got to have the right architecture before you can do anything else. Otherwise, all the data that you have, you won't be able to actually leverage. I think what organizations have been doing for the last five or six years, as they've explored the Hadoop ecosystem, is they've really seen that, wow, we've got a lot of data here. Let me see if I can make use of it. I think the progressions in the technology. So, Hadoop, I think you'll agree has matured almost more rapidly than any other technology that I've seen in the last 15 years. And so, that ability to become enterprise ready has really enabled more folks to actually leverage it. But you extend that just a little bit farther. I think there is that caution, the general caution in the marketplace. And what we've found, and the way Pivotal has changed and migrated over the last couple of years, is where we bring the most value is enabling our customers to actually take advantage of that scale and advantage of that technology. So we created HDB, which is based off of Apache Hawk, which is an open source database platform, fully ANSI SQL compliant, that runs on top of an ODBI distribution like HDB from Horton. So, now that you have this platform in place, the methodology, the platform as a service, the core data management services, what are some of the most, what are the real showcase applications, both in terms of application sophistication and scale that you point others to? So I think, I mean, it's funny, there are two places to look there. One would be, once you have all of that, and once you have it in place on a scalable and easy to manage platform, you have the ability to iterate more rapidly. And not only more rapidly, but also, again, kind of in small notions. So that circular event of having an application, watching people use it, interacting with other users, your backend systems, other data, leveraging analytics and data science to really predict instead of react, and then change the application to take advantage of those. So one such idea would be the idea of not taking your first offer, but taking your next best offer. And so we work with a couple of our customers where we try to upsell folks. The first time that you present an offer may not be the one that you actually choose, but being able to individually target folks based on their patterns and their techniques, which again, we can only really do now with the plethora of CPU and storage and technologies that you see out here. Once you're able to do that, it becomes a very personal experience for our customers and their customers. What do you see as the role of data vis-a-vis data warehouses in real time? Because real time is one of those things everyone wants now, and it has to be embedded in the fabric of the application. Yet the application developers, like they used to deal with infrastructure, move the DevOps. Is there a data ops kind of role going on where I just want to have access to all this free data and not have to worry about schemas and structures? And is there a DevOps phenomenon out there coming that developers would just be dealing with data in an easy way? So you hit nail on the head, and we're seeing this with a lot of our customers now that are actually waking up and saying, I live in the database team in the data warehousing environment, and what I've realized is that as data comes through my purview, it takes so long for me to model it and integrate it in with everything that by the time I can provide business value, the business value is a year, 18 months down the line. So the ability to actually land the data and let business users take advantage of it right away, while still maintaining. So I will argue that the EDW is not dead. The EDW will live on. But what Hadoop and what technology? Then I'm usually exclusive. You can have the data lake and have stuff stored. Exactly. That's resting data. That is correct. And then you have very purpose built, and this actually fits in very nicely with our data microservice architecture and kind of thought process, which is let's make purpose built, small, I'll say singleton use applications. Data containers. Data containers. I like that. I may take that. I'll be right on the cube. Open source. Now feel free. No, but taking those and actually being able to leverage that, royalty's coming. Royalty's coming. This is the concept. This is the idea of making things frictionless in this mind of a developer. Right, exactly.  You know, I think over the last 20 to 30 years, we have siloed not only IT from the business, but also within IT, we've siloed individual groups by themselves. And all that's done is caused roadblocks to get things done. And I think, you know, one of the nice things about a lot of these other infrastructures as a service players like Amazon, like Microsoft who are coming in, what they're doing is actually showing the rest of the business that it doesn't have to be hard. It actually can be relatively simple. And so what I'm seeing is customers are looking for a path to make that Amazonian experience enabled in not only their data center, but also within Amazon. And I think our products actually allow you to cross both assets. Do you need to change or are you getting pressure from customers to add more and more services to, you know, Pivotal Cloud Foundry so that you can take the richness of choice, the richness of choice from Amazon, but the local control that you might have with, you know, local Cloud Foundry, on-prem Cloud Foundry. So I would say in a way, we are seeing our customers push for and really need more data services within Cloud Foundry. And we're very focused on the Pivotal data side in order to enable that and partner with many of the folks you see back here. I think that what they're really saying, though, is help me get out of the business of managing things that I don't need to manage and help me make use of all the data by operational excellence, by being able to scale, you know, one of the things that we've taken in an iterative approach is Pivotal started on the data side a managed service so that customers can stop leveraging and having to scale up their own internal folks to be able to manage these data products, but instead can actually make use of those products. So I think what you'll see is more and more of a convergence of ease of use, ease of operations, again, so that customers and businesses can just make use of that underlying technology. Have you ever tried benchmarking the cost of operation of services hosted on Pivotal Cloud Foundry relative to Amazon or Azure? I know you run on one or both of those, but, you know, like a private cloud cost of ownership. Sure. So I think, so yes, so we have benchmarked, but we have done ROI calculations around that. I think, you know, what I would say here is there is a barrier of entry to Amazon. There's a barrier of entry to your own cloud infrastructure, but what is probably more indicative of our value problem is the barrier of exit. So by leveraging something like Cloud Foundry, there is no barrier of exit. It's much more tell, easy to get in, hard to get out. Yeah, exactly. So I got to ask for the cloud, since you brought that up, I mean, this is the dynamic architecturally and or business logic policy-wise is the relationship between data and the cloud. Your thoughts on this and the conversations you're involved with engineering teams and customers. For sure, so I think, again, the cloud is really changing the dynamic for a lot of organizations as I'm sure you guys seen. Storage, which was one of our parents' mainstays, has become quite commoditized over time. And I think there exists a spot now, more so than ever, where customers and organizations are amenable to leveraging the cloud for data storage. So as of today, what I would tell you is all data that originates in the cloud, most customers are fine and want to keep it in the cloud.  All data that's originated internally will more or less stay internally unless they can start to leverage pieces of the cloud in order to do scale and elasticity kind of applications and analysis. So, but that gravitas towards, again, I think businesses are not in the business of making IT organizations, but really in the business of their business. And so if they can leverage somebody else's infrastructure, they are going to do that. We are now in a spot where most of these companies have made big investments over the course of the last 20 years that will continue probably for the course of the next 20 years. Data centers they've acquired or servers that they've purchased, that those need to be depreciated over time. But I think what you'll see is a bigger and bigger push towards a lot of these other players, again, so that they can really focus on their business. So, Jacques, thanks for sharing the data on theCUBE. Really appreciate it. Final question I want to ask you to end the segment is, the career path of data engineering is not well-plowed. You're pioneering a lot of this and it really truly is cutting edge, science now, and role. You know, so what is just, what is data engineering and what does that mean for prospective students, hires, people looking to get a career in data engineering? So, to me, data engineering is near and dear to my heart. It has meant many things over the last 20 years. Today, as a data engineer slash data wrangler, I think what we're teaching people is the ability to, there is no right tool for the job, or sorry, so there's no one tool for the job. There is a right tool for the job. And things like Hadoop. But don't be a one tool for the job, the hammer, everything's a nail. That is correct. Literally metaphorically, don't be the one tool for the job. But as it evolves, so you can leverage all these tools and what they do is negate your ability or your necessity to have to be the wrench turner who's just moving data from A to B. But instead, you know, gravitate towards more of that analyst, statistician who's actually, you know, going through and finding nuggets of information within the data. So, a data engineer to me is everything. It is, you know, somebody who's looking at the data who is actually, you know, making sense of the data, but also overall architecture, because even today, even with all of these wonderful software technologies, without a proper data architecture to support the application. It's an engineering position. That is correct. Yes. Like they say in baseball, multi-tool player. We have multiple tools for the job. Jacques, thanks so much. Sharing the data and insights, engineering it all here at Hadoop Summit 2016. I'm John Furrier, George Gilbert. You're watching theCUBE, we'll be right back.


The St. Lucia Meteorological Services indicated that moisture and instability associated with an approaching tropical wave, together with a low-pressure center, will cause cloudiness, showers and isolated thunderstorms over the Leicester Antilles from tonight into Wednesday. The system is about 750 miles east of the Leicester Antilles and is moving westward near 23 miles per hour. One slow development of this disturbance is possible later this week and this weekend. A second tropical wave located over the eastern tropical Atlantic is moving westward near 21 miles per hour. Additional slow development of this system is possible during the next several days. The public is reminded that St. Lucia is currently in the hurricane season and that the necessary steps must be taken to ensure safety and preparedness in the event of a hurricane. The National Emergency Management Organization, NEMO, advises the following. Before the hurricane ensure you know your emergency shelters, trim overhanging branches from trees around your home or property, develop an emergency communication plan, secure buildings by boarding up windows and doors, store valuable documents in a waterproof container or ziplock bag, develop a family disaster plan, have a disaster kit or disaster supply on hand, place indoors any loose outdoor objects and secure those which cannot be brought inside. One must ensure that a disaster kit includes water, non-perishable food items, flashlight, batteries, portable radio, first aid kit, essential medicines, change of clothes, mask and sanitizer. In the hurricane, individuals are advised to listen to the radio, television or hurricane updates, stay inside away from windows or glass doors, avoid open flames such as candles and kerosene lamps, if power is lost turn off major appliances to reduce power surge. After a hurricane, individuals are advised not to go outside until the oil clay is given, only NEMO can issue an oil clay order. Be aware of outdoor hazards such as downfallen trees and broken power lines and report them to the relevant authority immediately. Avoid flood water because it may have harmful contaminants and dangerous debris.