I'm gonna be talking about a bit of a journey of the Shopify infrastructure This is the kind of talk that I would love to have seen from companies like Twitter Facebook or some of the other companies about how their infrastructure got to where they are today and Google has some of these talks where they talk about what Google looked like in 2001 and what it looks like well today It's so complicated. I think they barely understand how it all fits together But it was really interesting to see how their infrastructure evolved So what I want to talk about is as ratio said the past sort of ten years of a very specific history of one platform And how we got to the scale that we're at today so I work at a company called Shopify and What we do is we help people sell things whether you're selling them online or you're selling them on your phone or wherever and in the past ten years we've grown a fair amount and I Want you to keep these numbers in your in the back of your head while I give this talk because it's quite applicable for everything that happens We do tens of thousands of requests per second. We can do a massive amount of checkouts. We serve a We've served a lot of of transactions a lot of money downtime is very costly for us and our merchants and In the past couple of years. We've also grown tremendously in terms of our the number of shops that we serve and We have one particular problem that sort of dictates everything that we do on the infrastructure team at Shopify Which is what we call sort of the flash sale problem We have these merchants who can drive a completely disproportional amount of traffic to us and this creates Massive problems for us. It's things like from one second to the other There might be a 2x 3x difference in the amount of traffic that we have because there's some launch of this guy's shoes or We have Super Bowl ads that drive a massive amount of traffic And if you've ever tried to visit one of the websites doing the Super Bowl and for all the Europeans It's a big North American event. That doesn't matter that much to us If you try to go to some of these websites doings is Super Bowl. It is they're always They often have problems and we have tons of merchants that go into Super Bowl every year at this point not tons But one two three I think three last time and then like every good story And this is the story of our infrastructure. There's a villain and currently our villain is Kylie Jenner She's an American celebrity sprung out of the Kardashian family family, which is famous for being famous and she sells lipstick and when people can't access this lipstick they get fairly mad and This is a big problem for us It hurts our merchants trust because with Shopify it's sort of a white label experience Their customers has no idea that it's Shopify's fault that they're down due to their scale And not Kylie's fault. So she gets all of that and then she gets mad at us and we try to fix it I think there's even episodes of the Kardashian reality show where Shopify is down and they're filming. It's it's a mess You can you can find it later. I'm sure So really we're dealing with these problems of scale We've grown a lot in the past 10 years and I want to tell sort of this story of the evolution of the Shopify infrastructure and what we tackle and how we're getting to the point that in the next two months We're gonna be running out of multiple data centers, but 10 years ago. We weren't even thinking about that stuff so The way I think about Shopify is sort of on the spectrum from a single tenant platform to a multi-tenant platform it's cut off a little bit here and When Shopify started or on both sides of the spectrum there are sort of Benefits if you're a single tenant platform, it means that you're running just one shop There's one database per shop on a multi-tenant platform into most naive implementation You have one database for all of your shops and both of these approaches have sort of their own benefits, right? Okay, so I have to be the guiding hand here a little bit on the sides here So we have single tenant on one side multi-tenant on one one side and these this is sort of a spectrum if you have a Completely single tenant solution with one database Then you have a you have some advantages and if you have a multi-tenant solution where everyone runs on one database You have some other advantages So there's really a bunch of spectrums hidden in this if you have a multi-tenant platform There we go if you have a multi-tenant platform you have a large amount of capacity you have a massive database So if one shop Has a big sale they can sort of steal a bit of capacity from the other ones And then later when another shop has a big sale they can take some from some of the other shops But if you have a complete single tenant platform that becomes quite expensive because you need to provision a database for each one This brings us has sort of the secondary benefit of having good utilization If you have a big multi-tenant solution with many day many stores running on one database Then you can utilize that database and the hardware that you have better by sharing resources between the shops Whereas for in the other solution of having just one store per sort of set of hardware you don't have that and this sort of gives us a platform that is good for flat sales and multi-tenant and The single tenant is not quite as good for flat sales because you have this problem of a single store taking a massive amount of traffic Will be very costly per shop And there's some companies out there that are really good for doing stuff like that the single tenant solution Then there's things like the multi-tenant solution is just kind of cheaper You sort of leverage the network effect of having all these shops to sort of take costs from each other And on the single tenant side you it's expensive you have to care about hardware for every single shop We have 300,000 shops 300,000 databases. That's way too many to manage Then there are some benefits on single tenant like you have complete isolation if Kanye shop Completely breaks it doesn't break anyone else's shop. Whereas in this in the multi-tenant this this sort of naive multi-tenant solution You have that problem Scalability is also great for single tenant because you can just make that one database bigger if that shop grows big enough and so There's some of these don't even have Sort of a green end There is these are the extremes of a spectrum and in the middle of this spectrum We have some kind of middle ground that tries to harvest event the best of both worlds And this is sort of what I've been exploring for the past three years of how do we guess get the best of a single Tenant platform and the best of a multi-tenant platform in one platform I refuse to accept that we just have to be multi-tenant and just take all of these things and then give up all the things on the left I want to get the best. I want the entire D2B green so over the net over this talk We'll talk about how to achieve that So if we go back to the spectrum in 2004 our founders created this snowboard store And that was the first Shopify store they created a platform to sell snowboards and it looks something like this classic like mid-2000 web design and So we launched that but they realized that their business wasn't really in selling snowboards in 2004 e-commerce was kind of finally reviving after the dot-com bubble and There was a severe lack for good customizable e-commerce software out there So they sort of pivoted and two years later They created a multi-tenant platform to support many snowboard stores many stores in general that could That be and then Shopify became a multi-tenant platform instead of this Ruby and Rails app hosting just the one store It could now hold host well as many as it could until the database broke So Where is that go to middle where where do we get the best of the single tenant world of just running that one snowboard store? That can never impact any other store and then having that big platform that supports all of them So from 2006 to 2012 we moved a little bit in on that spectrum We started doing a lot of work Here and there to optimize the application to get a little bit more of the benefits of the single single tenant side and a little bit Fewer of the drawbacks of the multi-tenant side So for the first six years of Shopify's history we did sort of what what most companies did We focused on our product and every couple of months someone would go and optimize the performance There's a lot of low-hanging fruit if you did if you did a profile on a application You might be able to get five ten percent of your CPU time back for some dumb n plus one somewhere or Spending a lot of time just doing operations that you don't need to do you can do a lot of vertical scaling meaning you can just increase the size of Your database cluster increase decide the number of workers that you have and all these kinds of simple things You can do a lot a lot of caching our application is very cash heavy Because we have a lot of sort of just traffic onto stores that don't do right operations So we did a lot of this sort of stuff for the first six years But then at some point you reach a limit and we reached that ceiling in 2012 And there's a legendary picture of our CTO past past out place planted on the floor after having worked for months to try and support one of the biggest stores at the time and We reached a point where We couldn't vertically scale anymore because at the end of the day you can't cash a right You have to scale your rights at some point horizontally So in 2013 we started working on database sharding and this is when a full-time Infrastructure team was was created at Shopify because we had to start working on database sharding and scaling horizontally And only the mysql side was was sharded so before our infrastructure looks something like this We had some load balancers some workers that did some work at database some retuses some memcash and this all sort of worked together And then in 2013 we split the database into multiple shards so that the right traffic could go out And the reason why we have this right traffic is that if you're trying to sell something and you're trying to sell a Lot of it we need to create checkouts. We need to create orders. We need to take payments There's a lot of rights that have to go on and it have to be quite consistent and that's one of our big challenges So we did this in 2013 and really all of the deadlines of our team is around Is around Black Friday and Cyber Monday, which is a big American American holiday where Americans shop more than usual So we did that and it worked really well for for a long time and until then Sort of the right capacity or the scale capacity of Shopify had been the biggest threat to our existence We were terrified of these customers taking us down and on the other hand We didn't want to say no to customers who were driving a lot of traffic We could have just said we're not going to be a platform where you can do this kind of thing But instead we acted in a way where we decided we want to be the only platform in the world that can support these kinds of sales So in 2014 the biggest threat to our existence was that we had this like pretty big Abba this this point we were in the hundreds of servers in our data centers We had tens of data stores and all this kind of stuff and if and many of these sort of became single points of failure And it may sound somewhat embarrassing that just two years ago We had single points of failure in our infrastructure But it just hadn't been a priority to us because the threat the the threat of all these celebrities selling was much much higher Than that of a single database blowing up so in 2014 we started working on resiliency and the way I like to think about resiliency is kind of like in Chemistry if you have a big surface area You are gonna have more reactions and it's sort of the same with errors if you have a lot of servers Then there's just inevitably going to be more errors happening If you have one server the probability of you running into a kernel bug in a year is pretty low If you have a thousand servers the probability of you running into a kernel bug in a year is pretty high If you're if you're Facebook if you're Google and you have servers in the millions You're running into kernel bugs all the time and they have kernel teams to solve this So as you get larger more errors will happen and you have to deal with it This is really when you have to transition from this sort of Pet mentality of your servers in your infrastructure to the cattle mentality So errors are proportional to the surface area of your platform and we started reaching that critical mass in 2013-2014 where outages of these small components would sort of Disproportionate propagate into the rest of the platform and cost each these cascading failures so we started Operating with this mental model of a resiliency pyramid where we had to think about Certain areas that were happening in our platform and start getting them to not propagate downwards So we were really at the very bottom here We didn't have any idea what happened if one of our database nodes really blew up It hadn't been a threat to us at that point So we started working on things like mapping out all the single points of failure Making sure that if the caches were down and we hit a store that it would be fine that if we had warm caches And we hit a store and the database was down that it would be fine wrote test cases for all of this stuff and then started fixing them and we wrote tools to do these things we wrote a Little little proxy that sits between you and all your data source and development that allows you to easily emulate error conditions in development So you can do like hey for this test take down this database and see that everything happens And so we sort of mapped out the entire application all the single points of failure and did this in 2014 so 13 we optimized the database in fit in 14 We did a lot of resiliency efforts and in 15 we started working on multiple data centers So at that point we'd only really run out of one data center The risk of that one data center going completely out and being able to fill which were another data center Was just not as high than all of these other things worst case We would have to just provision a new data center We would have to be down for a couple of days, but it wasn't the biggest risk at the time but in 2015 that became the highest priority and we started setting up a second data center and What this what this boils down to is that you have one data center that runs all of the different charts and so on all of Shopify and then you have another data center and you set up a replication from the one data center to the other data center And then you somehow sort of move all of the traffic So what we did was we did a lot of work to go from the one data center to two to two data centers And then we built this script that can run the entire failover process in just about a minute of downtime on Shopify So when sometimes if you see a maintenance window on Shopify for going down for a database of failover It's me running this script to fail over the entire data center in about a minute And so it has a couple of steps here It updates sort of a service discovery layer and says okay The data center is now going to move and then it takes this the checkout down So if you're going through a checkout while we're moving data centers, it will show you an error Hey, please come back later if you go to a store during the time because the store is read only that's completely fine So most stores don't even appear to be down unless you try to perform a right And then we sort of stop everything the one data center make sure that the replication has caught up in the other data center Then start everything back up We tune our load balancers So all the traffic that went into the old data center now is proxied over to the new data center While we sort of update our internet routes to announce the IP's out of the new data center And then everything is moved and this takes very very little downtime And then you end up in this situation where data center 2 is now the primary and is replicating in the other direction And then we could always fail back and we've done this exercise probably around 10 times or so the very first time We did it. We were all sitting in a room There's about 20 different people looking at 20 different dashboards and it was kind of like like what I imagine like well if Like after a space mission they all like hug each other afterwards And that's what we're doing and people are on hangouts and it was a big thing And now it's like two or three two or three people running this script And it's still a little bit of a bigger deal than I want it to be but it's it's pretty it's pretty small thing to fail over a company hour size At this point so that's really great. So we got really good at this multi DC thing and then in 2016 we've been working on this concept called pods and The idea of pods is really let's take all this stuff that we learned from 2006 to 2012 about running a really really good Application and optimizing it and performance increases. Let's take that Let's take all of this stuff that we learned about database sharding in 2013 and all the stuff that we've learned since that Let's take that as well. All this stuff we learned about resiliency in 2004 and all this stuff We learned about multi DC in 2015 and sort of see if we can marry all these concepts in a way that makes sense and Do something with that so what we came up with was this idea of a pod And the idea of a pod is to take this sharding idea further So I shard by itself in sort of Shopify terminology is just starting the database We didn't shard everything else only the database because that was the most critical thing at the time So what a pod is is taking that concept to the extreme we take the workers We isolate them for for all that for a group of shops. We take Redis and we isolated for a group of shops So basically it becomes like running multiple deployments of Shopify at a smaller size So the goal is that if we have these completely isolated units of Shopify Then we can run those in multiple data centers one can run in Asia one can run in Europe one can run in India One can run in North America. It doesn't matter. They're all one shop does not need to talk to another shop There are some exceptions to that things like if you're talking to the API You don't want to have to care about shards or any of that or if you're a partner and you have Shops in Asia and you have shops in in North America. You don't want to care about that So there's all kinds of sort of global challenges to this So we had this 2013 had led us led us with this Architecture here when we did the database sharding and what we wanted to move towards was something more like this But we have these self-contained pods self-contained deployments of Shopify and instead of managing one big Shopify We manage a lot of small Shopify's And then if we have that then it doesn't matter where these things run some of them can run in data center one In North America and some of them can run in data center to in Europe And then all the European customers could be on data center to in Europe and all the North American ones could be and the ones in North America and we have multiple data centers there replicate them fail over the pods independently And we just have this really nice infrastructure And then if we return back to that spectrum chart from before we've arrived at something really nice We haven't quite reached the middle yet, but we got a lot better so in the Capacity on the capacity spectrum we have much better capacity than before and they even multi-tenant solution doesn't lend itself very well It's running in multiple data centers, but now we can do that if we wanted to spit up Google cloud pods of Shopify We can do that if you wanted to go into Amazon if you want to start experimenting with the cloud We can do all of that stuff its utilization is a lot better flash sales is also better because of the more capacity It's even cheaper because now we can start sort of having vendors and data centers bet against each other Isolation is better because now you have groups of shops so if one shop blows up the blast radius is sort of limited to that pod and not the entirety of Shopify and The scalability is also better because we can now run out of multiple data centers and scale sort of independently and So we're not quite there yet. Everything is not green, but it looks a lot better and Flash sales was one of these prerequisites prerequisites that I set up before it sort of is what everything comes back to at the infrastructure team at Shopify and what we do with flash sales is that We're limited by the CPU under workers the databases and are all over provisioned and not the bottleneck for us instead It's the workers. It's the Ruby and Rails workers that serve the application. And so what we have is we've written on top of some of our Resiliency primitives we've written it so that workers can sort of be shared between pods So if one pod is experiencing a massive sale It can take some workers from some of the less busy ones and then serve the traffic and then give them back to The other pods and this works really really well and we sort of get the best of both worlds So let's let's see what that looks like if we get a request in this setup you are going to do a request here where you get to these beautiful shoes on my shop calm and My shop calm resolves to one of our IPs our IPs are announced out of multiple data centers and you go to the closest data center and enter our networks and When you enter our network you reach this this software very dearly called sorting hat And what sorting hat does is it looks at the request and based on the request it thinks a little bit and Decides which pod this request is destined for and then routes you to the appropriate data center So your request goes to one data center enter our network the load balancer sorting hat module looks at the request decides is this This request is for pod to okay is pod to Local to this data center and active in this data center if yes, then you just send it to this data center If not you proxy it somewhere else So if you're going to Amsterdam you might be proxied to say Virginia to be serve your request And then you enter another sorting hat in that data center It knows that it's local and it's all it's all fine So this is sort of the multi DC strategy in a nutshell and I just want to give a shout out here to Open resty so this sorting hat module is written as an engine X module and we use Lua for this and This is some of the best software Like some of the best software that we run in my opinion engine X is extremely stable we found very few bucks in it despite our surface area on this and With open resty you can extend it with Lua you can do crazy things like customize the load balancer So we implemented an exponential Weighted moving average for our load balancing you can do things like serve as a cell search that are different in our case All of our customers point to the Shopify IP but based on the host name We can serve them the appropriate cert dynamically out of a database with this You can do anything that you imagine at a database layer if you're running Docker containers You can get all the containers from a service discovery layer What this stuff can do is incredible and what we use it for with sorting hat is that we go into the The request and looks at look at it and we route it appropriately Based on what the request looks like that's the sorting hat module So if we have this architecture We need to set up some rules of what Shopify can do to run in this setup And the two rules are rule number one is that any request that comes into sorting hat needs to be annotated with wherever It's going so sorting had needs to make a decision of where this request belongs to what shop Does it belong to what pot does it belong to if you're going to a data center in Chicago and the record and the sorting hat in Chicago can't figure out where This what what shop this request belongs to it just has to for for you It doesn't know whether it needs to send that request to Amsterdam or to Asia. It has no idea So it's just gonna error so every single request needs to honor that rule Rule number two is that any request can only touch one pod if it touches multiple pods multiple Shopify's it means that You might do a request to Amsterdam that also needs to reach to Asia and North America and everything in the same request And that's a mess that has really bad consequences in terms of resiliency that these requests now rely on multiple deployments of Shopify So these requests could look something like this you get a webhook from say stripe from a payment And then instead of having the shop domain in this case you would figure out which shop this request is going to by looking at the URI so you have to teach the low balancer about these different things The other problematic request might look like look something like this where in the past We might have gone over every single shop and tried to count all of them Or we might have had some operation to try and upgrade all the themes around all the Shopify stores on all the different Deployments of Shopify all the pods and these we can't have any longer So in the first case we need to teach the low balancer how to route that URL and in this case We need to teach Shopify to not reach to multiple shards in the same request and So we came up with this thing called shitless driven development because to honor these two rules we had 500 if not a thousand different code paths that violated these rules our Infrastructure for 12 years had relied on the very underlying assumption that you could do these two things You didn't have to care Shopify could just figure it out at runtime where the request came in so with this overwhelming list we came up with primitives to sort of create a deprecation API and Usually with deprecation what happens is that if you use a new API something would will yell at you But you can still use it if you want to what we did with shitless driven development Is that instead of just deprecating and spitting out an error message?  It's European and fresh and whatever and then all the pods respond back like I'll take it I'll take it in low bouncer picks a random one things like changing your password or Recovering your password you actually have to ask every single pod because we don't want to leap that abstraction to the developer of or to Merchant about what pod they belong to so all of these are sort of shit listed because we don't know how to route these So we added this syntax on top of the rails default Route syntax to have a routing method So in the first case we know to extract the shop from the program and in the second case We know to use this special sign-up method and in the third case We need to try all the different pods and aggregate the results somehow and in this last case If you just visit a store you can extract the shop and therefore know the data center from the host From from the from the host parameter in the in the HTTP config And so somehow we needed to teach the low bouncer about all these rails routes So what we did what was we parsed this entire? AST that the rails routes generate and made it into JSON that our sorting had Lua module could read so it looks Something like this the rails routes are serialized into JSON and the JSON is then read and interpreted by sorting hat Sorting hat being this module that sees request and routes it to the appropriate data center So this Jason might look something like this It's basically just has like five thousand or like five hundred or a thousand different routes that just match the regexes It knows the routing methods so it knows when it sees the request that matches this Regex for the uri then it knows to then it figures out how what the shop Or pod that that request is going to from the method designed here And the mass vast majority of them are shop from hosts, but there's a ton of others that don't go through that So if we then reiterate this this diagram What happens is that you get that request is sorting hat sorting hat looks at this giant JSON file that is generated from the routes from the web app And then it knows based on that request which pod in the world that it belongs to and Then if you on top of that honor these two rules of every request being of sorting had being able to know where every request goes and every request only touches one pod you have the isolation to be able to run in multiple data centers and I think it's really neat how all of the different efforts from over the years sort of play together the Resiliency that we get through isolation is quite well, but we also get isolation through resiliency We get scalability through all of these different things and the multi DC that we achieve really comes through the isolating of Shopify into many many small Shopify's that can run independently And so just a couple of thoughts of the on the future because we haven't quite reached that middle yet from the spectrum before not Everything was green in the middle. So we're pushing the boundary further and further towards the center But we're not quite there yet Maybe 2017 is when we'll reach it and two of the things I would like to work on in 2017 are things like doing zero downtime pod failovers when I failover a pod from one data center to the other I don't want any customer impact whatsoever So what we're thinking about doing is that when we fail over the data center from or a pod from one data center to the other We tell the load balancers in the origin data center to pause all of the requests Then we move everything and then we resume and let the request flow through this means that there is not really any disruption for customers If you're doing a checkup while we do a pod failover it will just be slow But it will still succeed and nothing will be dropped Something else I would like to do is isolating the shops further We still have the blast radius of one store blowing up will sort of propagate into a pod I would like to put more constraints on single shop So that doesn't happen as much by looking at past disruptions But that's really what we're what we've been looking for for 2017 in just about one or two months My team will hopefully launch Shopify and multiple data centers with this pods approach in time for Black Friday Cyber Monday if we don't Well fuck Thank you so much So the first question is how do you handle transactions in the sharded SQL? Okay, so I think what's implied by that question is How do we handle transaction between the multiple shards in this setup? That's how I'm going to interpret the question We have this nice property in Shopify of one shop not caring about another shop So the transactions are really isolated if you're doing a transaction on your shop in updating an inventory count or Products or things like that you're not going to reach across the other the other shards because you can't do transactions that span multiple shards It just doesn't work. There are multiple databases. They have multiple sessions There is one database that I didn't talk about because it's sort of a very lengthy talk on its own And what I've spent the past four months on is that we have another database that is sort of the master database It hosts things that are not that don't belong directly to a shop things like billing things like partners things like API all these things that don't inherently belong to one shop and You can't do transactions between a shard and that either So we just kind of like do best effort and put all of the transactional things back in the shop and then backfill them into the master So basically the answer is try to avoid doing cross-shard transactions every theory everywhere you can If you can't you need you need to try to refactor your code or just live with the drawbacks of not having transactions cross data store Yeah transactions hard Okay, the next one Did you extract microservices for your app and did that help with the scaling? Yes, so this is something we debate a lot internally as well So without going on a too lengthy rant here. I think microservices in many companies are sort of overcompensation I think good object-oriented programming can get you out of most of the most of these things and I don't think you need to enforce this TCP IP boundary to have a good software That said there are some boundaries when your application sort of obviously just belongs into different apps The example of Shopify is this we have these sharded tables where a shop has all of its data But we also have like the API and partners and billing and stuff like that that belong to sort of a global database So we want to actually extract these things into services because they don't belong to a single shop and Shopify should only care about shops So billing should live in its own thing partners should live its own thing and API should live in its own thing because these three things don't belong to multiple shops and with API I don't mean the actual API of retrieving your products or performing a checkout because that still belongs to a store I mean things like authentication or creating apps and things like that And More how do you handle the pod deployment? So the way we handle deployments right now Is that we just deploy all the parts at once in the future? I would like to do canary deploys or ring deploys where we deploy to one part at a time Right now we just deploy to all of them at once and that works fine But something I would like to do over the next year or two is to try and deploy to one part at a time Because it just minimizes the risk when we have all of our developers able to deploy all day All right, and the last question is are you using Kubernetes? I don't think that's public yet Okay, all right. Well, I'll just one more round of applause for Simon. Thank you so much


ौando ॉ chewing as a ॉ ॉ ॉ ॉ  BJP ॉ ॉ ॉ , where are ni in Ling Med Whatsapp , everywhere those small kids all kids will be there who are starting competitive extikkions some teachers, special teachers carry on some doctors and some nuns othersieseero so all small kids are just driving on films study room sergi Thunder is riding his foot in Antalya after the rain. In Ayurhipapura, no matter what this is, Immerothir will stay there considering the fact that no government has item in next Inf carriage. Today in Ayurhipapura, not people wish to take組 doubles in father country. I have purchased 2 Самas in IPPS and neither wrong country has anything from впNJD, nor are there any civilised returned et un dh backgrounds in PNJ. This is not done in country USA with little impact. This is flashing light that , crushed typk Blessing ute ute bh ko pip ka angledin panyaab shiv Ji Main র एमदश ॔ॉक्मूक्मं छॉक्मातॏ ॉ ै़ृ कि ना कयांघ। ॢिखुastywnakya ौ nineteyo ृ the ृ kusuh ఇింరార్వినీవార్లిందాస్ర్నలిలూరార్యంని ఇప్సార్రనాలెమికర్నూలనాస్లి నింరారంలరాపచనెరిండీంపచాస్సి కారేనిణ్లి సాకిస్యకా తిర్ఛతఢి ఫంటర్జ్athan న nouns�రోత్ని� uh గంపంటోవరు ఈచఽటో 늨౦త� Happiness In Tamil, Khmishal Chetty end, X harmonisyt an output pass, O greetings from the government, During all these challenges and the aggression fence taking place for us. To make sure that our kids again or that we act as the sports team that we play against the others. To have a good внимarency towards our library, So that a Punjabi媽 OK student can serve their kids in the Ass ya bhajab. You can make good names for yourself, or if you like with it city or Skill ad ka ن했습니다. So you can make help to those who aren't involved in any sort ofzę jail in their home, you can make it at home and a student can help to those vulnerable. ఠాశింటిక్తి, పఎసింటకితోనిఆనికరంవ్తోతోనిం నైంటానాసిచాప్రాల్ అనీటిన నిఱ్.


Good afternoon. If there's one brand as a marketer, which you all would have heard of and I'm sure studied as case studies It is the Mastercard brand The brand reinvented itself with its priceless campaign and we have Mr. Raja Raja Raja Manar with us to speak about more about the brand Sir you had a session at the consuline festival and You know the two words in your session with happy intention. These are two words that usually do not go together But your session was called happy tension the power of CMO and CFO partnership So how do you see the relationship between the CMO and CFO evolved in the last five to six years? So firstly you're absolutely right the two words happy and tension you don't put them in the same sentence, right? And that was intentional how we put it together Historically, there was always and there is always a tension between the CFOs and the CMOs of organizations While the larger objective of both is to drive the results of the company for the better The immediate focus of both is very different. The CMO is focused on three pillars is my brand moving in the right direction Is it really becoming stronger by the day? Is it being more preferred by the consumers and so on but the second pillar is is marketing driving the business results Whether it is top of the line funnel or it's a bottom of the funnel. It's about the margin improvement It's retention loyalty all these are concerns that marketers have on the business side Then you got a third one where marketers you want their focus on is on building Platforms that give the company an advantage the competitive advantage on an ongoing basis So these are the three pillars of marketers are focused on now something like a brand may not be Immediately producing results for you within the same quarter So when you're having a huge brand spend, it's very natural that people outside of marketing particularly the CFO can say Why are we spending this and why can't we actually take it back?  One is you have got your people second You got technology third is you got marketing People you don't want to play with people quarter to quarter Technology has long lead times today that the startup or the close So people particularly CFOs find it easier to gravitate to marketing to take in the funds out of In a short term or to give funds more funds either way, they see a lot of fungibility in how marketing pools operate Now that is where the problem lies in because in during the economic crisis If a brand pulls back on its investment, it will lose ground Particularly if the competitor is not pulling back and the amount of money you have to spend to regain your ground is going to be huge Well business results seem to be something which are more appreciated immediately Brands impact on business results is less understood and less appreciated But the fault of that is also of CMOs where you have to make your colleagues understand You have to evangelize and you have to not evangelize based on emotion But based on facts based on data and you need to understand the language of the CFO Now this is where the pension have actually happens And so this time actually what it decided is to share with the group from the main stage of can lands on day one With my colleague and the chief financial officer of Mastercard So both of us with the stage together and we started telling what is life from each one of our respective lenses And how do we crack the code at Mastercard in an effective way? So now I consider my CFO as a best buddy a great partner and it's been working for years And so I was actually diving into those areas and explaining to people the tension will always remain There's no question about it But you can make it a happy tension by understanding each of the well by talking in each of this language and Coming to certain working arrangement where You make them as an finance folks understand what marketing surreal impact is and for us as marketers to understand What is a predicament of the CF house particularly when there is an economic down? Rightly sir because even whenever you see when the economic downturn is turn is here Like you mentioned the marketing budgets are first cut and you also use the word evangelize So if you could just tell you know of a conversation that you had with your CFO and how did you evangelize him? So I'll tell you in fact my conversation with one of the previous CFOs It was very interesting and there were a lot of questions about does really brand matter Because if you go and you give a fantastic price either to your client, which is a bank or a Merchant things will happen. Why do you really have to build a brand for the consumers, etc? Not to say that the brand is unimportant, but the amount of money that you spend on brand is it really worth it? I just asked the person Which watch are you wearing? Of course, it was a luxury brand So I said, why do you wear this because at the end of the day this particular brand that you're wearing Only shows time and nothing else. I can give you a Antacrona cost them in the thousands of dollars I can give you a small cashier watch Which will cost you less than five dollars It'll show time. It has calculator. It has contact. It has so many functionalities You being a CFO, how can you be so irrational? And by that luxury item As opposed to this I said that is the power of brand I think that was an aha moment of that individual So we go in this kind of a fashion you have to take time to educate them And you have to understand why they feel in a particular way when you understand the root causes You address those root causes in their language Like for example, you know, I spent half of my career managing PNL's businesses, right? And other half managing marketing When I was managing PNL, I used to ask this tough questions to my marketing folks What exactly am I getting for this investment? I'm giving you And many times I get the looks of a deer caught in headlights In that kind of a situation, I won't get confidence Or worse if they start telling me oh my brand awareness has gone up and my predisposition has gone up As a marketer, I understand but as a business person, I say that looks like a fluff It's it's it's a lot of fluff out there and you're waffling. I know I want you to tell me I'm asking a question in dollars and cents. You answer me in dollars and cents I need to understand if I'm asking you time, tell me what time it is Don't tell me time is philosophical and then it can move Slowly if you're happy or fast, if you're not happy, what a real city I'm not interested in the philosophy. I want facts. That's what the CFOs and the CEOs are looking for And so now when I come over on this side and as the chief marketing and communications officer for Mastercard I'm acutely aware of this particular dynamic and therefore I try to speak always in the language of the CFO So that that person can understand exactly what I'm saying and be able to convince them The CFO, the CFO, I'd like to introduce the third angle, the CTO With technology being such a big part now How is your relationship because it must have been a drastic change in the last 10 years and probably interactions with CTO has probably got up multiple. So how is that changed? So this is another excellent point that you make which is CMOs don't operate in a vacuum Correct We need to collaborate with the CFO on the aspects that we just discussed Marketing is increasingly and a lot more technology driven than ever before And the future is going to be even more tech and data intensive And you don't want to go off and build your own tech stacks or contract with third party vendors Because as a marketer, you may not be knowledgeable enough to make the right call Not just for today, but do you have a map? Do you understand how to get from here to there? Your CTO can be a brilliant partner One of the things at MasterCard what I have done is within my team I have got a head of technology Who reports to me and also to the CTO? I have got a CFO who reports to me and also reports to the company CFO And I have got a human resources person because that's another important point of intersection Who reports into me and into the human resources head? So what we are trying to do is now marketing therefore is not just a standalone entity, which it is in one sense But it is extremely well integrated with other parts of the company on whom we are dependent quite a lot To make the company overall successful through the success of marketing You know, everybody now is talking about experiences But MasterCard was one of the first brands to actually deep dive into this And it's been a decade since you've pivoted from advertising led marketing to an experience led marketing strategy So how is this strategy of and the promise of creating priceless experiences helped MasterCard build brand recall as well as loyalty So in fact, this is something which I feel particularly proud of Right in 2013 when I joined MasterCard, one of the first decisions I took was to pivot priceless from being an advertising platform To having your entire marketing strategy hinged on experiences. So we became an experiential marketing company in a big way And the result of that if you just fast forward today From number 87 in the top 100 brands today. We are at number nine So we advanced 78 slots Which is not bad, right? And we are now at our top 10 brand of the world it is and they That's number one we built What do you call audio brand for the last four years? We have been rated as the world's number one audio brand and we have If you look at our brand likability scores, if you look at the brand momentum If you look at our brand affection scores, they are fortunately very very good So I think this strategy has been paying off extremely well for us and I'm so glad that I have Embarked upon this 10 years back and it continues to work and this is the direction that we continue to keep moving forward on Now we spoke about technology in one sector, which has been significantly disrupted with technology has been the payment segment Now digitization i'm talking particularly from the india standpoint of view Digitization has seen the you know with upi. We've seen the rise of ru pay g pay. What's up payments So how is mark a master card evolving its marketing strategy? in this scenario So firstly, we should look at payments as an enabling activity Okay, so for example, I'm not waking up nobody wakes up in the morning saying that oh I need to today. I want to make payment on On a soap or a shampoo or a movie ticket They want the movie ticket. They want the soap and on the shampoo That's what we think about and payment is a enabler, right? When more companies actually come in into the digital space Because today digital payments are still a tiny fraction of the entire payments space So there is a lot of opportunity to convert from cash and checks into digital payments So when many companies come in consumer education goes up consumers awareness goes up and all the companies rise It's all the boats rise when the tide goes up. That's number one number two What we try to do is to always look at What is it that the consumer truly wants and can we be there? providing Experiences that are truly priceless, which means we connect you to use something which you are passionate about In a way that only master card can do it Part of it. We do it through our priceless platforms Part of it. We would do it through our product construct Part of it. We do ironic certain kinds of loyalty programs, etc And like I'll just give an example now in India. I always keep joking that The three religions in India are cricket, bollywood and music Right. So we got into cricket last year with her partnership at PCCI It has worked like a charm What we did there the way we activated was not very traditional like what typically a brand has we went very very 360 and in very unusual and unexpected fashion That resulted in a wonderful lift in some of our brand scores in some of our brand preferences and so on It really works very well. And I think you know, so the key thing is we're not scared of competition At the same time, we are hopefully smart enough to see the opportunities to connect with the consumers And influence their preference in favor of our brand as opposed to somebody else's brand Can you elaborate on the kind of uptick that you see in post the PCCI association? What I can say is it is significant Unfortunately, we do not reveal in the media what specifics are But I can clearly say and you can quote me on that master card I have seen significant uptake in terms of how the brand is being perceived Post this PCCI Now the chatter at Khan this year has been all about AI And chat GPT You also, you know, priceless expressions is what mastercard goes by So, how do you see this integration for do you see AI being integrated into the mastercard marketing strategy to deliver priceless experiences So, you know, this is one of the common misperceptions That AI is a new phenomena We have been Infusing AI into marketing and to every aspect of marketing at mastercard for the last seven years So for example, about four years back, we launched something called the mastercard digital marketing engine Which we developed and launched out of Singapore That would actually deploy there be deployed artificial intelligence to be able to predict What the next micro trend will be And intelligently look at all the offers and messages that we have in our repository Pick out the most optimal either offer or a promotion or communication message And then leverage that in the context where that message is given or the offer is made and it buys media It creates an ad it does a B testing it optimizes the ad measures the ROI All this was automated and AI was at the center of it. It was a few years back for years back plus So AI is not new Generative AI is what is new So it has really literally broke open into the world so to speak And there is so much amount of excitement around chat GPT and you know, Dal E and mid journey And all this wonderful AI generative AI tools whether it is for creating images or creating music or creating what do you call text etc And We are actually very carefully looking at everything We don't want to rush in and do something because Generative AI if it is trained on public data or data that belongs to somebody else that are IP related issues We have to be careful about There are things which can creep in which have got intrinsic bias. We have to be careful about that So what we are now doing is to train AI Generative AI engines on internal mastercard data or internal mastercard communications, etc And then come up with Things that we can say this is based on our own proprietary information And we are not really taking anything from the public. So this is something which we are on But i'm extremely excited about the possibilities Finally, you mentioned about cricket, but mastercard has also associated with women's cricket badminton golf So what next can we expect from mastercard in the indian context any big sponsorship up on the annual? Uh, we're constantly looking for opportunities right and when some good opportunity comes We do a cost benefit analysis And if it proves to be attractive We absolutely go in and then do it if it fits also with our strategy obviously So it has to be a strategic fit. It has to be economically attractive and then we just go and do it And you're right. We went to the badminton. We went to cricket and you know, we will constantly keep looking for opportunities Like we went to the golf for example in india and uh, it's it's a journey and uh, you will hopefully Uh, sort of you know, see more action coming from mastercard and I cannot say exactly what is going to happen Thank you so much for your time, sir. Thank you. Thank you so much for having this


As we know and realize that a cyber space or a cyber society can be affected or is impacted by cybercrime, in order to effectively deal with that cybercrime, we need cyber laws. And therefore, in this lecture, let's have an introduction to cyber law. Let's first define what a cyber law is. Cyber law refers to all the legal and regulatory aspects of the internet as well as its users. Now as I said earlier that when you know the cyber space or the cyber society which you know came into which took birth with the advent of internet is affected and impacted by cybercrime naturally in order to best combat you know the downsides of that cybercrime or in order to serve as a deterrent in several cases, we need cyber law. Let's still look at some specific examples which will make the case for having effective cyber laws even more clear. For instance as an example, let's let there could be you know a situation of a cyber murder. Now what can be a cyber murder? A hacker for instance changes the value of insulin in a patient's online prescription who was admitted in the hospital and the nurse injects that quantity and the patient expires. Now this is just one extreme example of a cybercrime where it can lead to a murder. Looking at other you know issues where effective cyber laws are needed. One of the backbones of our economy is business and commerce and with the advent of internet we have all increasingly moved or let's say a major part of economy now is e-commerce and e-business. So in order to you know to ensure safety and security for the e-commerce we require cyber law. Then there are several other regulatory issues, there are unprotected emails, there is social media where from bullying to stalking to identity theft various things can take place. Then we need laws to you know let's say to prevent hate speech, to prevent cyber terrorism, to prevent software piracy, to prevent unauthorized video streaming. Moving on we need cyber laws to prevent email spoofing, email spamming. We need cyber laws in order to regulate various kinds of financial crimes. In order to regulate online gambling we need cyber laws in order to you know monitor and prevent the sale of illegal items and articles. We need effective laws in order to safeguard people from forgery, from cyber deformation, from cyber stalking. Moving on you know we need effective cyber laws to prevent denial of service attacks which tend to affect individuals as well as businesses. We need cyber laws to prevent individuals, businesses you know those with which are using home computers from Trojan attacks, from viruses, from worms. We need cyber laws in order to you know combat data diddling you know which is which is you know a change in a certain data entry into a certain system by let's say a malicious virus. We need effective cyber laws in order to prevent intellectual property crimes. In order to you know in order to handle cyber disputes in order to prevent unauthorized accesses to computer systems and in order to prevent salami attacks just as an example of some of the the types and kinds of cyber crimes which may exist or which are being which are happening around us. As I also mentioned earlier that a news report suggests that in Pakistan alone the last three years have witnessed an increase of cyber crime by 83 percent and it is estimated that worldwide the amount of losses financial as well as you know non-monetary as a result of cyber crimes is going to increase increase many fold in the coming years. Let's now look at how the cyber regulation or cyber laws have evolved over time. United Nations Commission of International Trade Laws UN Citral came into being in 1966 to start off with then it was based on those that model laws on e-commerce came into being in 1996 and model laws on e-signature came into being in 1996 as well further word intellectual property organizations copyright rules came into being in 1996 similarly word intellectual property organizations performance and phonogram street rules also came into being in 1996 both these copyright rules as well as performance and phonogram treat rules regulate copyright infringements and trademark issues. Moving from these from these effort by the word intellectual property organization as well as the United Nations which gave these model laws in 1998 internet cooperation for signed names and numbers I can give uniform domain name dispute resolution policy in the US carrying on from here while this uniform domain name disputes and resolution policy is you know adopting or is helping implement the VIPO word intellectual property organizations copyright rules and treaty it is in it is helping in implementing that in US carrying on from this DMCA which is digital Millennium Copyright Act is is you know is again you know extending on these these this copyright rules and treaty by VIPO and that was been enacted in the US in 1998 in a similar fashion European copyright directive EU CD was implemented in 2001 which again is helping implement VIPO's copyright rules and treaty then information technology Act in India are the major you know a major law or rules which are regulating you know the cyber crime or their major cyber laws prevalent in India from year 2000 Pakistan too has over time realizing the importance of having effective cyber laws you know presented and and past its own ordinances and laws the first one was ETO electronic transaction ordinance which was passed in 2002 this was the first of any kind of IT related regulation and laws which were passed in Pakistan and while they were largely based on the cyber laws of foreign countries they were nevertheless dealing with 43 different kinds of cyber crime there was an improved version of it in the form of prevention of electronic crime ordinance which came in 2008 and then finally in 2016 there was prevention of electronic crime ordinance which provides a very comprehensive framework to deal with cyber crime these cyber regulations in Pakistan deal with various kinds of cyber crime including you know cyber terrorism cyber stalking identity have denial of service attacks cyber stalking cyber bullying and several to name as per prevention of electronic crime ordinance 2016 for various offenses there is jail time there is a large amounts of fine and and you know the good part of the upside is that these side that this an effective implementation of these cyber laws are preventing a lot of let's say damage which could have occurred in their absence.


Hi everybody, we're back, this is Dave Vellante of Wikibon. I'm here with At Stu, Stu Miniman of Wikibon as well. This is theCUBE, SiliconANGLE's continuous production, live from IBM Edge, we're here at the Mandalay Bay Hotel, in wonderful Las Vegas. Stu, we spend more time in Las Vegas in May and June. Yeah, Dave, everybody's been asking if we have, you know, the theCUBE, like, you know, Bungalow, where we can all just stay here. Our equipment lives here, but luckily we don't. Even my kids now, when I talk to them at night, they say, hey, daddy, how's the loss of wages? You still have your shirt. Anyway, Ron Reif is here. Ron is the storage software business strategy head for the group formerly known as Tivoli, now the cloud and smarter infrastructure group. Ron, welcome to theCUBE. Oh, it's good to be here. Thanks. Good to see you again. So, let's see, I'm not going to ask you about the group name. You told me off camera that that's, that's something that others make the decision on. So, but congratulations on the transformation. Thank you. Let's just leave it at that. So, big talk about SDS here. I want to get into that, but before we do, let's talk about your role. So you, in a strategy role, and I mean, a lot of companies kind of don't want to be in a strategy role. IBM is a great place to be in a strategy role because you actually have impacts on decisions, M&A. You've got ways to leverage the portfolio, internal, external collaboration. Very exciting role. So what specifically are you doing, within the division, within the group, in a strategy role? Well, my job in business strategy is to go spend time with clients, understand their business. We've been talking a lot this week about economics. My job is to understand the economics of their business problems. What is it that is causing them grief and IT, specifically in the area of storage? Think about what the solutions that IBM has from a technological standpoint research that we could bring to bear to solve those problems, and then work out the economics of, can we invest enough money into it, create the solution, charge clients in such a way that they get value, get their problems solved, and we end up making money. So I'm a sort of a business strategy, investment strategy just for storage software. And you have to figure out how to do that and get a return, as you said. Get a return. And so there's external factors involved in that.  Now is it the Ron Reif methodology? Does IBM have some kind of methodology? Is it a little bit of both? We have processes, procedures that we follow, but there is a lot of an art form to it, trying to understand what the bad guys are up to, listening very closely to what the analysts are telling us, understanding. Bad guys being hackers or competitors? No, no, competitors, no, no hacking. We talked a lot about information security this week, we're not involved in any of that. But just keeping our ear to the ground on what's happening in the marketplace and what technologies are coming out of research. So I spend a lot of time with clients, a lot of time with analysts, a lot of time with our researchers, our developers, just trying to put together the entire puzzle and help guide decisions on where do we put our investment dollars? Can we get the technology out the door quick enough to be competitive in the marketplace and make a good return? Okay, so what's the market telling you? You spend a lot of time with customers, reading the analysts, reports talking to analysts. What are the big trends that you're seeing that are driving some of your advice to your colleagues and some of the decisions that you guys are making within the division? There's a macro trend. It's really sort of simple, but it's one that you have to sort of back off and look at for a moment. For decades and decades, we've been dealing with data is growing. My whole career at IBM and even before then, we've been worried about data is it gonna get too big? Can we afford to store it? For decades, data was growing, but the hardware vendors that were building storage systems were able to improve aerial density a little bit more quickly than the data was growing. So from a client perspective, an IT manager, the solution to the problem of data growth was they bought a current generation storage device that was denser at a better price per gigabyte and they were able to keep up, keep that budget moving relatively flat while data was growing. In the last several years, it's been fairly well documented now that data is growing faster than hardware vendor's ability to keep up with aerial density improvements. And so what's happening is if the old way of doing storage administration doesn't change, CIO's IT managers are gonna end up spending more and more and more of their IT budget on disk hardware because they have to have more of it. And so it's driving shifts. One of those shifts is this move toward software defined storage. So, part of me says, why don't you guys all just sit back and let it happen and you make a ton of money? But we all know that would kill the market. Kill innovation. Kill our clients, right? They wouldn't be able to afford to. Something has to change. So venture capital would come in and solve the problem. So when you see Hadoop in many respects was designed to solve that problem and others. So okay, so that's a major shift that you see. So let's talk about software defined. Let's start with, what is it? What is it to IBM? Yeah, so software, one of the big difficulties in the industry has been settling on a definition. We'll come up with an idea and we'll communicate it one way and vendor X, competitor Y will come up with a different way and everybody's trying to co-op the term. Our view is very similar to what IDC recently sort of communicated as a full taxonomy of what is software, they call it software-based storage is that it's a software layer that gives the full set of storage services but does it on commodity hardware and it's not in any way tied to back-end disk capacity. It's just a means of managing storage with commodity hardware. In a lot of ways it's very similar to what has happened in the server marketplace with server hypervisors. Today when you talk about a server, you're not talking about a physical machine, it's a virtual machine that can live on any kind of commodity-based hardware. What's happened is that all the function has moved into that software layer, relatively inexpensive compared to the rest of the IT budget and it's allowed the hardware to commoditize. And given CIOs, IT managers, a lot more flexibility and the ability to drive down the overall cost of servers in their budget, same thing's happening in storage. So Ron, I'm wondering if I could poke on that because I really, I don't know that I agree that kind of software-defined and commoditization, I think there's a connection but I don't think they necessarily go hand in hand when I think about what server virtualization did, it did not necessarily commoditize. Everybody's not running white boxes. Server virtualization changed the architecture of hardware and sure, we collapsed down and consolidated, got greater efficiencies but IBM's still selling lots of servers and they're actually selling bigger servers for the most part, beefier cores, everything like that.  As you were saying before, storage growth we can't keep up with, so software-defined is not going to reduce how much capacity I need, it's going to keep increasing. So, I guess- Well, kind of depends on what you mean by commodity, right? So, but let's give Ron a little latitude here. Let me try to weigh in on this. You basically said like servers, let's say commoditized, and what allowed that was a virtualization layer and abstraction layer, so basically you're talking about abstracting the underlying hardware. That's right. Okay, and that leads to, you can essentially then run that on what we're calling commodity storage. Right, so I think it's a fair question. In the server world, you're right, not everybody's running on white boxes but you have the flexibility to run on a white box if you want to, or an IBM System X system or- And what I'd say is, you know, of course we got any workload really can go on x86 this day, so it's standardization kind of across there. And the value has moved up into the software layer and so there's not a lot of, I mean the differentiation in the servers, they provide the MIPS, the megahertz that are needed for compute, but that's about all they provide. They provide that and maybe reliability, that's the differentiation that you can come in. Same thing's gonna happen in storage. You're right that you're not gonna stop the need for gigabytes, but what you're gonna get out of the storage device other than gigabytes and maybe availability and IOs per second, not much because all the rest of the services come out of the software layer. Yeah, and I'd agree we're talking about it to the services and the value that you add, not, you know, the sheet metal in the hardware. The other thing that we're finding with that software base approach is that in order to get tier one kind of services, traditionally IT managers have had an overabundance of what you would think of as a tier one disk array. 70% of your shop might be tier one, so you can give those services to the applications and the rest of it might be tier two or tier three. With a software defined layer, we're finding that clients are shifting the mix. 30% tier one, 5% flash, and the rest of it, tier two and tier three, the overall cost of the dollars per gigabyte in the infrastructure ends up dropping like a rock. And that's part of what we think of as the commoditization of the disk arrays. All right, so let's keep picking at this definition here and we'll get it and we'll sort of talk about what it means to IBM. So we got the commodity piece, but that seems to be almost an attribute. I think that the ability to pull those hardware resources is key and that leads to commoditization of the hardware. But then you've got this discussion about people say separate the control plane from the data plane. So you've got essentially data services that could be somewhere, for instance, inside of an SVC or V7000 that might be in a disk array itself, but you've got some kind of other so-called control plane, which is what, an orchestration layer? Is that right? Is that potentially open stack or maybe it's VMware or IBM hypervisor, is that? Yeah, so we think of that software defined layer as the control plane. It's the layer that provides all of the storage services that you need from snapshotting to data placement to replication, the thin provisioning, compression. Everything that you would need to provide for the data is in that control plane. It also includes a set of APIs out the top end that integrate with orchestrators, like open stack or smart cloud orchestration from our standpoint or VMware, if they're provisioning storage along with a virtual machine. So everything that you need is in that software layer. Underneath that is the physical capacity. Kind of like VMware has pretty much everything you need for a virtual machine. And underneath that is some Intel cores that provide the compute capacity. All right, so let's double click on that. So the services are part of that software layer. Is that right under that definition? The way we think about it, yeah. Okay. And so those services could reside, I presume I'm correct, within an SBC or maybe you're saying no, they ultimately need to be extracted. Well, so we've talked about our, yeah, it's a good question of where exactly does it run. First, it needs to be software. Second, we think it needs to be able to run on really any commodity, in our case, Intel-based engine that you wanna choose. Now, we've talked this week about the store-wise software platform, which is our software defined storage layer. That software code base, we package it with a variety of different Intel-based commodity engines. One of them is the same volume controller engine, it's a gateway that doesn't have any of its own capacity, it federates capacity from, you know, we would say commoditized storage, any storage that you might want to choose, white box on up to high end. It's magic. It's magic.  We package it with engines that have their own spinning disk as well to augment the capacity of whatever you federate together. We have entry, mid-range, enterprise configurations. It's just a lot of different engines, but one software layer that you can use and choose whatever storage hardware you like. Okay, and then like I said, you got the northbound APIs into... I think those are key.  So let's talk about that a little bit. So, and I want to tie this into where you guys are at today. So let's talk provisioning. Okay. Okay, presumably I'm able to provision not just capacity, but also performance. Right, think about the, in the old days when you were actually dealing with a physical device, the act of provisioning was, hey, I want this many gigabytes on that device in this RAID configuration. And it's got these attributes and I need that much horsepower from my app. It took a storage scientist to figure out where to put stuff. What scientist? Why didn't we come up with that name? Could have been heroes. But, you know, in the world of software define, you're dealing much more in SLAs. So the kinds of questions you ask when you're provisioning are, what service level do you want? And that service level can include performance, can improve resiliency, disaster recovery, the needs of the workload in a service level. We put it in the form of a service catalog entry inside our Smart Cloud Virtual Storage Center, the software layer. You ask what service level you need, you ask how much capacity in that service level you want and you ask who needs access to it. Those are really the only three questions that you need to ask when you're provisioning at a software layer. Software layer then takes care of figuring out what commodity hardware that you have in the environment that will meet those service levels and it builds the virtual volume. But specifically I could then, as an example, make a provision, 50 gigabytes capacity in a thousand IOPS, make that my policy, maybe give it a retention policy. And a replication policy and so forth. And I can do that through an API call. That's right. So the way we make it available is two ways. The human being can sit down and say, you want 50 gigabytes of database capacity and make it accessible to that guy. Or the API calls coming from things like OpenStack or VMware or orchestration software that are provisioning a virtual machine and a virtual network and some virtual storage for a workload all at once. Okay, and I can change that policy essentially on the fly. That's right. Through an API call. That's right. You just, the policy is bound to a virtual volume that has your data in it. You can change the policy and the system will relocate the virtual volume based on the policy changes. And I can do that today? Sure. Okay, I would say that's software defined. It's smart cloud virtual storage center. So let me give you a use case, an interesting use case.  You've got some physical desk. You've picked from high end vendor X and maybe mid-range vendor Y and low end vendor Z. You've got three different vendors, multiple tiers. And you've got some workloads spread across those physical desks. With the smart cloud virtual storage center, a software defined layer, it will analyze both the physical capability of the hardware that you've handed in, as well as the IO patterns of the workloads that, what they need. And it'll make a connection between the workload and the physical capability of the storage. It'll spread out workload across the physical infrastructure to optimize the capabilities of everything you've given. So nobody gets too hot, nobody gets too cold. And then once the data is placed in the right location, it'll watch for blocks of that data that are getting hot. And it'll dynamically, transparently, move those blocks over to flash storage, for example, to keep everything running optimally.  We're seeing that CIOs then can choose cheaper storage, more commoditized storage, and everything operates well because of the software defined layer. So, Sir Ron, my last question I have for you is I'm actually looking at your blog, and you talked a little bit about across sites. So how do cloud and kind of object storage fit into the software defined storage message? So, you know, cloud, we ran an interesting study recently with Enterprise Management Associates. We were trying to identify cloud, talking to customers who had made steps into cloud, and trying to figure out what are the interesting tidbits that we can learn to teach other people who are moving into cloud. One of the things that they discovered was that a big hindrance in doing cloud at all is physical storage. They needed to move to virtual storage. So, this idea of a software defined storage layer has become critical for cloud deployments because of the flexibility. Excellent. All right, Ron Reif, we're getting the hook. We've got to go. So, I really appreciate your time. Thanks for coming on theCUBE. It's good to talk to you. Always good to be here. All right, everybody, keep it right there. We're right back with our next guest. We're live from IBM Edge in Las Vegas. This is theCUBE. We'll be right back.


Hi everyone, Joe for jaskiescasebrakes.com with the one box break of 2020 top's inception baseball with a shot to win to transcendent spots right here Now there's also no cardinals in this so top two spots win a spot each So big thanks all of these folks for getting into it right here No cardinals like we said, so let's do this quick one box break four and a one five times for each list and Fifth and final time after five. We've got B. J. Jordan at down to Aaron a pettit four and a one teams fifth and final time Four and a one mariners down to Indians. All right, so there's first half of the list right there Second half right there. Let's order these by team No one's trading right? All right. Looks like no. All right. Good luck boys and girls Here we go What do we got in here? I know the whole point of this break is to see who wins those spots But there could be some a nice hit in here, too So first of all, there's that final printout hot off the presses. Oh There's a bonus in there. See that bonus autograph. All right. Okay. Hey, so Cezardo to 150 and behind Jose Uriquiti Is Tyler Alexander? Rookie Otto for the Tigers That's 21 out of 25 for Quentin Carter in Detroit. Nice. Oh, sorry bonus is Whoo Boba Shet 005 out of 155 Toronto Blue Jays Oliver With the bluebirds This toddler not worthy of Boba Shet. This one is a little bit cleaner a lot of top loader dust and Plastic bits. There you go. Nice. It's pretty strong All right, there you have it Nice quick little break turn it to be a nice one for Oliver for a little one boxer Now let's go back to this list. Let's get everyone from Leo down to BJ top to bottom and Let's roll it. Let's randomize it Three in a two five times so top two spots get spots in The transcendent Hall of Fame Bray Hall of Fame edition brand new edition after five top two after five one two three four and five after five times top two spots are Leo and Zach Congrats to the both of you. Thanks for getting in You'll put a little rooftop next to your name and we'll put in set six next to that as well So, you know, you got it from this specific break. Congrats again. Hey, thanks everyone for giving it a shot There'll be some more opportunities to win spots in the store on jazby's case breaks calm check it out And I'll see you next time. Bye


I've had the most amazing week of my, maybe my life, I may be the birth of my kids who beat you guys out, but just bear with me, just bear with me.  I've been saying it all day so I'll just keep this short. There is no nerve tag without the people who come and they keep telling me, well there's not one without me, but it really is kind of the other way, I can't really do this without you guys. I can't do it without the friends who came to help me, people on stage, people who did panels, people who helped us move stuff around, my wife, my kids, my relatives, my friends, my neighbors, just not possible. So this worked and didn't have a single complaint from anybody despite all the landslides because you guys are awesome and I keep telling me how friendly everyone is. People who aren't in this group who just were here to vacation are freaking out about it. They think, who are these nice people that want to sit out and explain magic to gatherings. So thank you all so much. Some of you I know aren't here, some went back early because they're trying to sort of beat the chances of raining and stuff tonight. I know some are staying over. Again, we are over here overnight. I'm going to be real sad when this is all over, but thank you so much. From the bottom of my heart for being a part of this this year and please come back and let's do it again. Okay? Yeah!


Live from the Sands Convention Center in Las Vegas, Nevada, it's theCUBE at AWS ReInvent 2014. Brought to you by headline sponsors Amazon and Trend Micro. Welcome back, we're here live at ReInvent in Las Vegas. I'm Jeff Kelly with Wikibon, you're watching theCUBE. Joining me now is Will Schoberry, he's the SVP of technology, a company called MediaMath, a marketing optimization company. Well, thanks for joining us. Yeah, of course, my pleasure. So for those out there that don't know, tell us a little bit about MediaMath and kind of what you guys are doing for your clients. Yeah, so MediaMath is a technology platform for marketing automation. We help, I guess Fortune 500 companies like Target and LLB and eBay and others that are household names drive their digital marketing spend. And we kind of focus on outcomes, right? So instead of blasting money towards a particular online advertising set, they are trying to move a particular product or drive some particular outcome and we optimize against that. So an LLB, for example, might have a new product and they want to reach a certain type of consumer and get an ad in front of them, whether it's mobile on the web. Exactly, yeah, and so we help them across digital media kind of portion their spend and manage that and optimize it. Yes, that whole ad tech business is, I mean we've covered this a little bit here on theCUBE. We've talked to companies in this business and we've talked to a company called Aerospike, who's one of the technologies that some of the ad tech runs on and it's fascinating and the performance requirements are just mind blowing when you think about it. Walk us through a little bit that kind of process when from a time, somebody logs on to a website, then they see an ad and it happens like that but there's a lot going on in the background. Yeah, it's kind of insane when you think about it. When you pop onto a webpage, ESPN.com, you're looking at whatever baseball stats that you want. That ad slot at the top of the page actually is a full auction, right? So it goes off to an exchange and we have about 40 to 50 milliseconds to reply back. Now the crazy thing is we do this at the scale of the internet, right? So this happens about two million times a second, right? So we have to kind of coordinate this across all of our exchanges and all of our data centers across the world. So it's a pretty interesting data problem and computation problem that has to happen at real time. Yeah, absolutely. So tell us a little bit about why you're here at the show. Presumably you're using AWS in some capacity to help you deliver on your mission. Talk a little bit about what AWS is doing for you and how you're using AWS to do that mind-numbly fast analytic process. Yeah, sure. So one of the things that we are finding to be true is that giving our customers control over their data and giving them access to it is fundamentally a really good thing for both for ourselves but also customers. And so what we found is that Amazon actually makes it easier not only for us but also for them to do this sort of analysis. The other thing is, as we're operating on the scale of 120 billion opportunities every single day, there's a lot of data exhaust that's created. So one of the things that we have to be good at but maybe we don't necessarily want to spend all of our time on is moving that data between data centers and analyzing it. And so that's something that we are actually really happy that Amazon has products like EMR and Kinesis and that entire slew of data pipeline tooling and so that's one of the things that we spend a lot of time with. We also have a number of application services that are baked on top of the EC2 stack and are very happy with them. So AWS is a really abstracting way of all that complexity of moving that data around, getting it where you need it so you can analyze it and deliver it to your customers so they can see how their ads are performing and that kind of thing? Exactly. So what about the hard core, the processing, the actual engine that's sending the data to the auction, delivering the ad?  So we do actually have a pretty extensive hardware footprint. Part of that is necessitated by the latency demands that the exchanges have. One example would be between the United States, we have four different data centers. There's one in Brazil. There's three in the Asian Pacific region. But all that data exhaust that comes out of those locations is still funneled through the Amazon Web Services platform. Well that's interesting. So talk a little bit about, so AWS is great for a lot of things but it sounds like in terms of that really low latency requirements, people are still doing that either off-premise, they're doing it on bare metal or they're doing it somewhere else because AWS doesn't really support that kind of latency. Do you see that, is that an accurate perception? Do you see that ever changing? So I'm not actually sure that it is. I think AWS has done a really good job of making HPC type applications possible and some of those boxes have those 10 giga interconnects which should cause that latency to be less of an issue but for us primarily it's distance between our location and the exchange. And so in situations where the exchange is actually in an Amazon data center then that's certainly something that we can do. I know that a lot of companies are fond of doing cost modeling inside of the cloud and then moving those things off-cloud and we've found ourselves sometimes taking that approach and sometimes taking the opposite approach of actually saying that the benefits of managing this entire service outweigh the incremental cost that it might take to run it in Amazon. So I really think it depends on what you're trying to optimize for. We've found that focusing on the service is actually more beneficial to the business and ultimately we are here to solve the business's problems instead of necessarily hardcore technology problems. So talk a little bit about, take a step back and we're here to reinvent. I think I heard 13,000 people up from 8 last year so it's growing like crazy. Let's just take a step back. What is your take on AWS and how they're disrupting some of the more traditional approaches to IT? So that's kind of an interesting question. I think this morning's keynote is a really good example. I think they released, I don't know the exact number, it's six plus new products and personally I'm very interested in all of the software-developable lifecycle tool sets. That's another example of something that I need to be good at but don't necessarily want to spend a lot of time on and so that's something that we'll spend a lot of time looking at deploying our new service that's set into. So if you're a new startup team inside of an enterprise that's working on a service I don't know why you would, with the new tool chain that they've kind of launched today I don't know why you would necessarily look at any other solutions to prop up a brand new service. Just my opinion. Yeah, well I mean it's really amazing that AWS has grown so fast and they're starting to, they're really disrupting traditional markets but they're also really moving quickly. You reference the number of announcements and it's hard to keep track of them all. They keep rolling out new processes, new approaches, new services and they've had that number I think 450 plus new services this year over 280 new ones last year and it keeps growing. It's really remarkable. But you said something interesting and I wonder if you could expand on it. You mentioned business processes, things that you need to be good at but you don't necessarily want to own. How do you determine what fits into that bucket versus what fits into, this is competitive differentiation for us and we need to own this. I mean I think ultimately ends up, it's not a perfect economy but the things that create business value those are the ones that we focus on. So that example earlier of latency, that's the reason why we try to be really great at peering with providers that are right next to us and maybe expanding our footprint globally so that the media that we're buying works for the customer and then the user. But then there are things like configuration management and deployment life cycles and that sort of thing where I could invest an entire team's time for years to solve those problems and as you know it changes. The model changes and you have to keep up in order to be effective and I don't necessarily think that that's the best use of the organization's time at this point. Now if we were the scale of maybe Amazon and we had I don't remember what the number was, 3 million servers, we would definitely take a look at solving some of those problems in-house but at this point I think it makes great business sense not to try to solve them that way. And so it sounds like that approach allows you to focus on the things that differentiate you from your competition. So how do you continue, I mean you've got to continue to innovate. I mean you're in a business where, well let me ask the question, what are the key differentiators for you compared to your competition and where are the areas that you need to keep pushing to stay ahead of them because there's a pretty vibrant ecosystem out there in this market and as I mentioned we've talked to a few of your competitors and a lot of them are here, they're AWS customers as well. What do you do in terms of maintaining that competitive edge? So there's a set of technical differentiators which we look at internally within the engineering team that we try to continue to drive and then there's a set of product differentiators and I'll try to talk to a little bit above them. From the technical perspective we believe that building great technology and having that technology operate at scale more efficiently than the competition is something that will benefit the customer quite heavily. So we focus a lot of time around that. That includes being able to analyze more data, being able to deliver it to the customer more rapidly, driving down the latency on the analytics, those types of things and also making it freely available via our data platform. Those sorts of things are incredibly important to us. We also focus very heavily on the product set which are things like expanding our footprint across multiple channels. It's not enough as a platform like ours to just focus on display. We're expanding into mobile, we're expanding into social, we're expanding into video. We've acquired a couple of companies recently that have focused on that. So those sorts of things are the, that feature set focus is the other half of what we try to do. And I think we've done actually a pretty great job. Year over year, our growth has been incredible. Yeah, I mean there's a lot of opportunity and again I know that you guys are working hard to innovate and continue to deliver more value to your customers. So we only have time for one more question so I want to give you the last word. Again, just any takeaways from this show. What would you say to our audience, people watching who couldn't make it to the show in terms of AWS, where they're going and how they're impacting the enterprise? I think ultimately, if your desire is to be as a business whose business isn't primarily infrastructure, if your desire is to be good at infrastructure then keep doing whatever you're doing. But if your desire is to accelerate the pace of the problems that you're trying to solve with technology then I would take a hard look at not just the infrastructure offering that Amazon provides via the Amazon Web Services platform but also the opinions that they are subtly stating via some of the tools that they're providing. I think Andy Jassy earlier mentioned things like continuous deployment, delivery and integration and all the integrations with that toolset. It's really obvious and it's really clear to, I'm sure most of this crowd here that that's the direction that companies need to go in order to innovate effectively and quickly and if that's interesting, play with it. Good advice. We've got to leave it there. Will Sturberry from Media Math. Thanks so much for joining us on theCUBE. My pleasure. Thank you. Stick around. We'll be right back with our next guest live from Las Vegas at AWS ReInvent. You're watching theCUBE.