This session we are going to be having now is called Embracing Collaborative Chaos and I'm delighted to welcome Linsep River on stage. Great, thanks for the introduction, Krim. Hello, my name is Linse. For the last few years I've been leading a group of platform teams that develop and operate a very large platform as a service for a very large public sector organization based here in the United Kingdom. In this talk I'm going to be going through how we've used Chaos Days as a really useful approach for improving the resilience of our platform and learning more about how it handles failures and helping us improve the kind of approach to handling failures so that failures can be handled more gracefully. Right, let's get my slides up. Let's start off by thinking about what we mean by Chaos Engineering and why it's an important thing to spend time on. Chaos Engineering is really important when you're building IT services, particularly IT services that are really, really important for people to be able to access and they take a lot of traffic. It's further compounded when you're trying to do all this at speed. Here in the United Kingdom our government has been providing various services to try and help people that have been impacted by coronavirus by CV-19. The organization I have been working for has been part of building these services and it's been a very exciting thing to do and we're all privileged to be part of this. These services were only built in about four weeks. They were delivered 10 days early and one of the services processed 140,000 claims on its launch day. It did all that without any major incidents. I think one of the reasons that we were able to achieve this is because as we've evolved the platform that this service is built on, we've paid a lot of attention to understanding how to make more resilience to understand how it failures and to use chaos engineering to help drive those learnings and improve that resilience. So this scheme, this government scheme is just one example of a service that has really benefited from chaos engineering. Let's look at another one that is another good example of why it's important to think about failure. When you're building something like Nest, for instance, it's something that's used around the world and it relies on services that are interconnected around the world. It's very easy now through cloud platforms and through things like agile and lean development techniques to build something that's planned at scale really, really fast and when you're building it, you're dealing with components that are hiding a lot of the complexity from you. But underneath those components have a large number of moving parts and a large number of things that are connected as this diagram shows. I think if you're building something using Google's technology they've probably got resilience covered. So what could possibly go wrong? Well, I'm not sure if people saw this just every year ago. There was an issue with one of Google's, I think, one of their networks and that had a ripple effect. It combined with various other problems and then caused people to use Nest to not be able to use some of their smart home automation. People were tweeting that they couldn't turn their air conditioning on because their Nest wasn't working because it wasn't communicating with the particular Google service that it relied on. They couldn't get into their homes to just go using a smart lock. This is a great example of just how when several failures can happen to combine in a way that's catastrophic. When I see these incidents happening around the world I always find it really interesting to read the incident report that really good organisations like Google tend to make publicly available soon afterwards. What fascinated me about this particular incident report was how it typified the importance of chaos engineering in that it wasn't one single issue that caused this problem but it was actually two normally benign issues and a specific bug. They came together that caused this significant outage that had a significant impact. The systems that each of us built, unless they're incredibly trivial will generally fail in some kind of way. Failures are often happening in small ways the whole time in our systems. It's just that they're not combining in such a way that they cause catastrophic failure. I like to spend a few moments just thinking through a system that you're working on currently or you've recently worked on and work out in your head what are the different component parts that you have control over? How are those component parts connected? Are they all running on the same piece of processor? Are they spread across network connections? Are they spread across the world? How reliable are each of those components? How reliable are the connections between them? What would happen if one component failed or two components failed? What would happen if a component partly failed and there was latency in the connection between the various components? What I'm hoping is that as you're thinking through those things you start to realise that actually the types of systems that we're developing these days have a high number of connecting parts that make them very complex and very difficult to predict exactly what will happen when things fail. This is where chaos engineering can really help. Unless you're dealing with a simple system that is just a very small number of components with a small number of connections. I don't think many of us are working in that space these days. Most of us are dealing with hard systems that have multiple components, multiple connections between them. Many subcomponents, and often those subcomponents are things that are hidden from us. Chaos engineering helps us understand these failure modes by deliberately inducing failure into the parts that we want to understand more about. And then observing the impact of that failure reflecting on what it's teaching us about the way the system behaves on the failure, about the way we can observe that failure and also about how our teams operate when those failures happen. Chaos engineering then helps us to take those lessons and improve the resilience of the systems we're working in. It helps us to build resilience into what we're doing. Chaos engineering also helps us to get used to thinking about production the whole time, particularly about what could go wrong in production. Because it's not a question of if something is going to fail. It's more a question of when is a failure going to occur. Resilience is a word I'm going to use quite a lot in this presentation so let's just agree on a definition. I took this from the online Oxford Dictionary. By resilience, we're focusing on systems that can recover well. We're not talking about systems that are indestructible, systems that will never fail. Because the complexity of our systems these days is so high that failures are happening the whole time. And it's very almost impossible to achieve systems that will never ever fail. It's more important that we consider how can we make our systems more resilient and how can we ensure that when failures happen and when they combine in different ways we can bring normal operation back as quickly as possible. We're trying to design systems that are elastic. There's four different ways I've seen of using chaos engineering to help us achieve this. I'm going to step through a brief overview of them in a minute and then we'll focus on the chaos day approach. The fourth one is something that is not a very popular one to talk about and so I'll get to that in a minute. The manual chaos is where things like chaos days or AWS game days or change specific chaos where the engineers that are working on the system they deliberately take time out to interfere with the system and inject chaos in some kind of way and then observe the response, learn from it and then adapt the system as necessary. This can be done either on your own system through your own chaos days where you can learn more about the concepts of chaos engineering or resilience engineering through going to some of the AWS run called a game day where you get to work as an engineer on a system that they've constructed, an artificial system that they start injecting failures to and that's a really fun thing to do if you've not done it before. A change specific chaos is then when teams are looking at each significant change they're making to the system and trying to, as part of the engineering of that change to kind of test out different failure modes. By automated chaos I'm referring to systems and software that you have running either continuously or we run frequently against your system that cause things to fail in an automated way. So things like Netflix's Chaos Monkey. I'd also count the use of cloud technologies such as AWS spot instances and GCP pre-emptable VMs. So these are compute instances that don't have a kind of guaranteed permanent lifetime. They're instances that will be terminated at some point and the services running them will receive notice but by using these types of compute instances then you're having to design the services running them to handle the fact that they will be terminated at some point and failing frequently and that is one way of making your services more resilient for instance when they run on other compute instances that fail unexpectedly. Another example of automated chaos is the picture shown in the bottom. We ran a hackathon day at this organization and one of the teams came up with this great concept of they took a Nintendo emulator and they wired up the emulator so that when you're playing Super Mario if Mario happened to die in the game then a random Kubernetes pod would be killed and that's what you're going to see in the background there. So that's quite a fun way of trying to semi-automate some of the chaos. In process chaos engineering is what I think teams need to try and evolve to. So this is where just as when we helped achieve continuous delivery in many organizations by thinking about quality and shifting quality to left and building quality into what we're doing the same bit can be achieved I think with chaos engineering but if each person on the team not just the DevOps engineers or platform engineers but the product owner devs, QAs, business analysts they're all thinking about failure and what can happen now should the system react to different failure modes throughout the whole of the engineering process then that will help improve the way a system will perform production and improve its resilience. The fourth way of using chaos that we generally don't talk about that much is unplanned chaos. I'm sure people are kind of watching this talk your systems are rock solid and they never fail you never have production instance.  They can be viewed very negatively and it can cause a lot of stress but I think it's really important that each time you have a production instance you see it as an excellent opportunity to learn. You can improve the way you learn from production instance by structuring your team's reaction to them and over time you can also use things called post-institut reviews there's various templates and literature about how to run post-institut reviews well. This is a growing field by the way it's something that we as an IT industry don't do very well yet so it's still something we need to learn a lot more about but if you have a production instance then getting everyone that were involved in that incident together afterwards and just step through what happened what did people observe how did they know to dig into particular areas stepping through instance in that way it's a really great way of learning more about the system that you're working with and through learning about it that knowledge to engineers and product owners and QAs are gaining can really help improve the resilience of the overall system there's various ways of trying to automate how incidents are managed and the data that's captured through instance there's all sorts of tools that are starting to emerge now in the marketplace those tools are fantastic but I would encourage you to start simple and look at just simple ways that you can prove the way your teams respond to incidents and look at things like your communication channels how do people become aware of it in an instance and then how do people create a timeline and step through it afterwards test engineering isn't just about trying to build things that are more elastic build things that are more resilient I think it's primarily about education it's helping us as an industry us as a group of engineers and product owners and testers to understand more about how our actual systems work it's a bit like we're a medical team and we have this alien in the operating room represents our system and we haven't got a clue what's wrong with it what happens when we do something to it and the RIT system is so complex they're like that so Chaos helps us build up that knowledge of how our system behaves what does normal look like for our systems and what happens when things go wrong it also helps train people's behavior so that when problems do occur they have more of a reflex action to know what to look at, who to talk to and what steps they can take to remediate the problem next Chaos engineering can help with the process it can ensure that the teams have a much clearer set of steps to go through when an incident occurs it also can improve the breadth and the depth of lessons we take from incidents by helping us do incident reviews well finally it can improve our engineering process so through using Chaos engineering we can start thinking more as we're specifying a change and then speccing it out and then actually doing the engineering on that change to think more about fader and think more about how we can make this change more resilient and improve the overall resilience of our system having addressed people in process only then do we start thinking about how Chaos engineering can be applied to a product but it can help in many ways so Chaos engineering can help us develop systems that are simpler to reason about if you have an overly complex system then when an incident is happening it makes it extremely difficult to work out what problem is triggering other problems when you're seeing different signals or different logs happening what they really mean Chaos engineering can help you improve the observability of the system by ensuring that you have the maximum signal to noise ratio it can help with your documentation so if an incident happens and an engineer that's on call and responding is not so familiar then they'd ideally want to run but that can really help guide them how to respond so it can guide or it can improve how we can write our run books the kind of information we put in there okay we're now going to move on to the mechanics of running a Chaos day so firstly how do you decide when to run it and what are the steps you go through a bit of context for the organization that we've been running Chaos days in it's a very large public sector organization it has over 100 million customers the organization has grown significantly over time it's about six years old we started off with just two delivery teams and they were building the platform as they went along we didn't design this layout from day one it's how it's evolved and as it grew we've now reached about 60 teams and those 60 teams are supported by six platform teams that provide the underlying capabilities like building deployment services observability, so lobbying and monitoring, alerting auditing, persistence those are kind of the platform capabilities it's a very very busy platform in terms of the amount of requests that go through it the numbers on the left-hand side were from our busiest day last January that was pre-COVID so I think those numbers you could probably double them now when we have new services that help large numbers of people in the United Kingdom receive benefits related to COVID we get a lot more traffic now we chose a very constrained set of technologies when we started this platform and that really helped onboard new teams and ensure that teams could land really quickly on the platform and develop services very fast our focus when we started was on getting the platform right for growth so optimizing it to get teams on and teams delivering public facing service really quickly we were also thinking about resilience and we helped improve its resilience by going from one cloud provider which wasn't Amazon because Amazon wasn't available in the UK at that time to two cloud providers just to ensure that we had a spread of the cloud platforms that we're working across I wouldn't recommend doing this now by the way I think that platforms such as ADOS and GCP I don't have any experience with Xero because I can't comment on that but certainly I know from ADOS and GCP that just using one of those cloud providers their resilience is certainly high enough that I'd be happy investing in just one of them as a cloud platform but we didn't have that luxury as we chose to go multi-active now through this period of time we were having a lot of production incidents they were a kind of a fact of life and they made people that one called us lives fairly miserable and because of that we knew that we weren't ready to start doing chaos engineering because we had so many like so much free chaos engineering in the form of unplanned production incidents then finally AWS came along in the UK and we migrated to it we moved our entire platform across to AWS we also achieved quite a significant change in our process in that the organisation gave us permission to allow every team that worked on a platform to deploy to any production any environment whenever they wanted and this was quite a big achievement it gave the platform teams who at that point were doing all production deployments for all services a lot more capacity as their capacity so the combination of those two things made us think okay we're on AWS now we know we've got a cloud infrastructure that is much more resilient than what we had previously we have more capacity to think about how we can improve it and how we can measure our resilience let's do a chaos day so that's the timeline that took us to our first chaos day if you're wanting to run your own chaos day who do you involve where do you run it and how do you step through it the approach I'd recommend is that the people on the chaos day itself are actually going to be breaking things we call them the agents of chaos and we made that team a closed secret team people knew who were in the team but they didn't know what they were planning and for our first chaos day we took one person from each of our six platform teams and we asked the team to volunteer their most experienced person who's been on the platform the longest who is the person you always go to when there's a production incident because what we wanted to simulate through doing this is what we call the bus factor that's from the highest bus factor imagine that person had been hit by a bus and then your platform goes down how are you going to cope so for the chaos day itself these people were unavailable to their teams so not only were they planning the experiments to run we had good knowledge on how to plan these things and what were the key areas they wanted to explore they weren't going to be available to the team that was then responding so having identified who these agents of chaos are the next step is to map out the system that you're going to be experimenting on there's just to gather around a whiteboard and when you're doing this process you can involve the whole team you don't need to constrain it to just the chaos the agents of chaos because at this point we're just wanting to understand what types of things do we want to learn from this chaos day what's a big list of possible experiments we'll let the agents of chaos pick the ones they're going to run with and design them in the way that's secret but I think through having the whole team involved in looking at the the system mapping it out understanding what normal looks like in terms of the behavior of your system understanding what components do we not feel confident if they were to fail and also what components do we have no control over so for instance if an AWS availability zone went down then you can't do much about that particular kind of problem but you should hope that the way you've configured AWS should mean that your instances can spread across multiple availability zones so you would cope with that so doing this in front of the whiteboard and just having lots of ideas about things that people are worried about in terms of if they went wrong they wouldn't know how to respond to there's a helpful approach and I found that as a facilitator for this session that my job was to say as little as possible apart from setting people the constraints giving them a clear goal and then leave them to get on with it which happened with Apollo 13 when the engineers had to get together and figure out how to deal with the problem of the air levels in the Apollo 13 module a bit of structure helps and the structure I found useful to provide is this template we used Trello so we had a template Trello card for each experiment or failure that we wanted to run and the agents of chaos and once we had kind of brainstormed a big list of failures the agents of chaos did the detailed thinking on what are we actually simulating here what's the real failure that we want to learn more about what do we think will happen what's our hypothesis on how the system and the team will respond and then how do we think things are going to automatically recover will there be any self healing or do we think an alert will fire and people will notice and fix it somehow how will the failure be rolled back this is a hugely important thing to cover that we forgot about or didn't think about in our first chaos day and when we're running chaos experiments we found it useful to time box how long we run an experiment for so we'll introduce the failure give it maybe 20 minutes half an hour if the team haven't noticed we might give them a nudge if the team notice and then they're struggling we'll probably give them another half hour but it's quite demoralizing if you've got a production and you're just completely stuck so generally to help get us through as many experiments as possible and get kind of learning from a diverse range of experiments we found it useful to have a rollback step which we then invoke so the agents of chaos would invoke if the team is really stuck so we just magically put everything back to normal and normal service resumes because we're generally running several different experiments in our chaos day you need to think about how these experiments might interweave sometimes it's useful to have multiple experiments running at the same time just to simulate what can really happen in production when multiple failures happen sometimes you'll want to serialize your experiments so for instance if you want to explore what will happen if you took out or if you're alerting or if you're monitoring that's a useful experiment to do but you don't want to do that and run lots of experiments other experiments at the same time because your teams that are responding will be pretty blind without that kind of monitoring and alerting stack and so you're not going to get as much learning from doing those two things at the same time quite a different type of chaos experiment you would also want to consider running on a chaos day is having any kind of security engineers or security people interested in security on your team try and run some security attacks during your chaos day this is a great opportunity whilst people are busy trying to fight production instance will they notice if someone uses that all of that chaos and that confusion will they notice if someone was trying to then to the system or steal data or take someone's credentials if you've got people that can add some security experiments then it's a good idea to have those as well so having decided who's going to be doing it and what the experiments are the next thing we need to consider is which environment are we going to run it on and Netflix are quite famous for running their chaos experiments in production and I really strongly emphasise it doesn't have to be production you learn an awful lot from doing things in a pre-production environment what I think is really important is that whichever environment you run it in is as close to production as possible so that you're learning what would happen if it were in production by close production I mean you need to be able to simulate traffic going through that environment in a way that you would see it in production you need enough traffic so that when a failure occurs it will have an impact that the failure is going to then cause alerts to fire for instance that means you need production like alerting and telemetry and logging and monitoring we're really fortunate that on this platform we have cookie cutter environments that have the same logging, alerting and monitoring in each of the production and pre-production environments we just need to tweak for a chaos say tweak the alert thresholds so that a level of traffic cause a problem that an alert would fire you also need to consider then once you know which environment you're going to work on what's the kind of blast radius for instance if you've got other organizations that depend on your pre-production environments wherever you're running your chaos do you want them to be impacted by the chaos that you're running or do you want them protected it's fine either way just long as they're going to be affected that they're aware of what you're doing finally you all need to agree how are we going to simulate a production instant happening from the perspective of communications so what's Slack channel or Teams channel what way of communicating to the people that use that environment and the people that are coordinating our response to an instant where are we going to have these discussions again you want this to be as close to what you do in production as possible I'll show you an example of that in a second okay so you know who's going to do it you know what they're going to do you know where they're going to do it you need to then think about the actual execution of it each time we run a chaos day we ask ourselves is it useful to give people warning or not that this is going to happen we've also moved from having just one chaos day to actually running a week of chaos and this allows us to do a balance so we found that useful perfectly respectful to warn people that use our platform that we are running a chaos day but we've increased the element of surprise by changing it to a chaos week so we give people notice that in this week things might go wrong just be prepared for that so don't plan any super critical work to happen in that week because you may be delayed it's important when you're picking these dates to know if there are any key organizational events or deliveries that need to be done or that we'll use that platform or that environment in that period because you probably want to avoid those because chaos days can be quite disruptive it's important that you agree when you're going to stop wreaking havoc when we did our first chaos day I think it took us two days after the chaos day to put the environment back together following that we all agreed we'll stop wreaking havoc at about half past three half past four in the afternoon or back people are much happier if you do that then the agents of chaos they need their own private way of communicating so if you can't calicate them which is pretty difficult these days then ensure they've got a private Slack with Teams channel ensure they have a Trello board with those cards that are prioritized in terms of the experiments they're going to run so they can see who's running which experiment are people responding to it or do we need to roll it back helps to have someone to facilitate this just so that the team don't get obsessed with just one experiment run it for too long so they can actually move across several experiments super important that you self-document what you're doing and we find this is really useful to do using a mixture of putting comments in the Trello card and also using threads in something like Slack so each time you're running an experiment put into Slack that we've started this and then make comments about what you're seeing documenting chaos in this way makes it super easy to get really valuable lessons as you then later on go back and review what's happened because Slack effectively is creating an automatic timeline for you about what happened when for everyone else that is working on the platform that's going to be impacting the chaos it's very important that they work like it's just another ordinary day at the office or ordinary day working from home they should be busy doing whatever they normally do because you don't know when a production incident is going to happen do you they come when you least expect them they come when you're busy doing other things and that's what people should be on the chaos day however when something does go wrong if it's in a pre-production environment then they need to treat it as though it was production that was on fire itself in terms of communication channels on our platform we have public event then environment and issues so we have event staging issues event QA issues and for our production environment we have event and live issues and we agreed that if we were going to use staging then we would use the event staging issues as the public communication channel where the platform teams that are responding to the production incident or the pretend production incident would be communicating to the rest of the platform they then followed their own processes using other channels to communicate on the details of their response having run the chaos which gives you loads of learning by the way actually stepping through that day people learn a lot about how their systems work we then need to dig in a bit more and draw out more lessons and share those lessons more widely how do we do that when you're running chaos days on large platforms we typically had all well probably 6 plus teams we had about 10 teams involved in chaos day getting 10 teams of 10 together to try and do a retrospective is really difficult I would recommend that you divide up your retrospectives or your post incident reviews into your component teams so each team ask them to go away and run a post incident review or a retro about the chaos day to treat it for each failure that they were dealing with as a real post incident review and step through that process that's a way of improving that process itself they should be focusing primarily on what lessons about resilience can we draw out from the chaos day what we use to improve how people can understand resilience what can prove about our process and then finally what can improve about the product to then secondly consider the actual mechanics of the chaos day itself because it's a good idea to run these things regularly so what would we do differently to try and make the next one even better to try and improve the lessons that we get from the next one having had each team do that themselves then regroup bring someone from each of those teams into a team of teams retro where they can put forward the key lessons that they got from the rain retros have this all documented as well and shared across the organization as wide as you can there's so much good information from these post incident reviews and retrospectives that it's important to distribute it so other engineers that didn't participate can learn from it having done this quite a few times now and in a few different organizations we've learnt that the first time you're doing a chaos day starts more start with just one team and a few services because there's a lot you learn about doing chaos days themselves through doing this if you try and scale up to multiple teams on your first day it can create a lot of pain that isn't very helpful I'm talking about pain don't break too much on your first chaos day or even on subsequent ones you want to run maybe between 5 and 10 experiments and that's enough to get a whole load of learnings because that's a bit like 5 or 10 production instance happening all on the same day you don't need to use a production environment you can just use something that is close to it as possible that should help you consider how production-like are our pre-production environments because it's really important that they are production-like if they're not production-like then you've probably got a bunch of engineering you just need to do on that it'll be very tempting when you're running through chaos days to try and come up with a big list of improvements these are all the new alerts you need to put in these are the run with the dashboards we need to put circuit breakers here and while some of those improvements can be useful hold back on coming up with that list focus first on what lessons can we draw out about our system about resilience from this chaos day and as we're going through those lessons you're doing the retrospective note down somewhere separate what are the things that we might consider about improvements to our product for this then later a few days later go away and review that list because you don't want to have knee-jerk reactions of circuit breakers and alerts that you're going to put in because the chances are the alerts you put in they may never fire again because of the special nature of some of the experiments you put in finally there can be a lot of fun so use the chaos day as a way to help people that perhaps don't normally work together learn more about each other and enjoy being in control of your own chaos having listened to this there's some things that you might want to take away to your organization so I'd encourage you to think about how established are your systems that you're working on are they ready for applying some kind of chaos engineering what's the smallest thing you could do to improve your systems for resilience this is something that we have to help other organizations out with us so feel free to get in touch there'll be links on the slides that I'll share after this session we've also taken things we've learned from running chaos days and put them into publicly available open source playbooks not just on chaos days but also on things like secure delivery working remotely and our share links to those also right okay thank you so much for your listening and we can now move on to some questions so the first question is from Sambhavi it's what has changed in chaos engineering practices over the years could you share a bit more about where we are in the maturity of this practice certainly okay I think chaos engineering is currently a bit like where continuous delivery was maybe 10 years ago 15 years ago I remember seeing a talk by John Osborne who helped introduce continuous delivery at Netflix and he did a presentation about it in a Riley conference about 10 years ago and after talking about why continuous delivery was a useful thing to do some people came up to him and said you're irresponsible trying to encourage your organizations to deliver in this way and I think the same response can be made to chaos engineering some people when you're telling them you should try and break things in your pre-production or in your production environment and see what happens you might think that's irresponsible that was probably a few years ago I think now people are realizing actually failure is such a certainty that it is useful to be deliberately invoking failure and our systems are so complex now that deliberately invoking failure is the only way to be learning about how our systems respond and so I think we now recognize that this is a useful thing to do and there is increasing kind of tool support in terms of particularly things that work with different clouds or automating kind of chaos it's not an established engineering kind of practice as say tester and development or pair programming so it's not something that I think many teams have at the very start of their process yet so I think we're probably another 10 years off that I hope that answers your question and should we move on to the next one do you measure the cost of running chaos days we don't measure and report on it we know how much it costs in terms of how many people are involved because in terms of the cost it's the agents of chaos their time kind of planning things and then actually running the experiments for the rest of the teams they bear in mind that we're encouraging them to work as a more working day so if your organization is set up really really really well then most of the team will be unaffected and just be whoever is on support that day having to deal with things but if you're trying to break things big style and your team is poorly set up to handle these failures then often the whole team might get involved we don't find any kind of benefit in measuring the cost of actually engineering time and reporting on it because we we've kind of asked ourselves a question or you can ask the organization a question what would you rather do would you rather spend time putting in controlled chaos and learning about how to respond to it or would you rather wait for a critical business event a key business event to happen and for things to go wrong at that point because there's the cost that you're incurring there when it's a key business event that takes your platform out is the reputational risk and the reputational cost and that is orders of magnitude more than engineering costs so I don't think measuring the cost of chaos days is such a helpful thing to do because the benefits to me are clear the next question is does the Pareto principle work in chaos testing as well Pareto is at the 80-20 rule when the team is starting off to brainstorm what are the kind of experiments that they want to kind of run typically we get the whole team involved in doing this because through getting the whole team involved then they learn more about the system people will be saying well I know that when this component fails it will ripple out in this kind of way and another engineer may not have known that or one engineer might say well I'm really worried about what will happen if this component fails I can't predict what will happen and there might be another person that has the answer to that so I think when we run this we typically get 30 possible experiments that we want to run and I encourage people to try and prioritize things by risk, stroke, worry what are the things that if they happen tomorrow in production and your busiest day would be most worried about and we go for those and you get a lot of learning I think sometimes regardless of which particular experiment because if a production incident happens and some things are common across a production instance such as how your engineers diagnose a problem, what kind of alerting and logging do they get from it how do they communicate those kind of things transcend different experiments I think just by choosing a small number say between 5 and 10 of experiments then you will kind of learn learn a lot and by prioritizing by risks, stroke, worry that's very effective you don't have to cover massive amounts of ground


Live from New York, it's theCUBE. Covering Big Data New York City 2016. Brought to you by headline sponsors, Cisco, IBM, NVIDIA, and our ecosystem sponsors. Now here are your hosts, Dave Vellante and George Gilbert. Welcome back to New York City everybody. This is theCUBE, the worldwide leader in live tech coverage. We're here as part of Strata plus a dupe world. This is what we call Big Data NYC. This is our seventh year of covering this show in New York. We're seeing Hadoop evolve beyond, it's not the shiny new toy anymore. Riddika Gunnar is here. She's the vice president of data and analytics at IBM. Good to see you again. Good to see you too. So yeah, big week for IBM.  You got a customer event. Must be excited. It's absolutely exciting. We believe that our organizations and clients are using data in so many new ways and they need new ways to put data to work. And so tonight's launch is all about how organizations, how clients, how everyone in society puts data first and how they're building their businesses and the outcomes for their organizations. So you're calling this data works, is that right? Is that the working data? Yeah, so we're gonna announce tonight. I always get to tell you guys before it actually happens. We're gonna announce tonight a IBM Watson data works platform. And the core capabilities of this platform is the ability to be able to access all data and to be able to drive AI-powered decision-making across all organizations. Pretty phenomenal and foundational. So when you think, like we've been, like I said, covering Hadoop world since the early days, 2009, 2010.  How do we add value beyond all this sort of plumbing?  So talk about that a little bit. Where did the vision come from, the strategy? It's starting to all come together now. You know, we see and realized major shifts that were happening in the environment today with the numerous different clients and organizations that we talked to. There's a substantial shift that's happening in terms of how data and analytics is treated within organizations and we intend to lead that shift. Fundamentally, what we saw is it's no longer about the persistent stores. It's no longer about data in Hadoop, data in an operational database, even data in real-time streaming. What's even more important is how that data is accessed with trust and how organizations actually use that data. What's also important is to be able to allow users to be able to leverage machine learning and cognitive capabilities to be able to automatically recommend for them where they should be starting from to gain the most types of insights in their organization and effectively start to help organizations future-proof their investments by leveraging open standards and open source capabilities. One of the complaints that you often hear at events like this, particularly around the complexity of the Hadoop ecosystem, is that complexity? Does this simplify it or am I adding now more tools that add to that complexity? Well, I think there's simplicity in the capabilities that we're able to provide in the platform, especially in how users interact with it. We have user-specific tooling that is fit for purpose for each individual user to interact with the platform, given the skill set that they're most comfortable with and allow that collaboration across the different teams. So that simplicity is absolutely there, but it's an equal balance of that simplicity and that choice and flexibility which we know today's organizations want. Today's organizations want the most innovative set of technologies that are out there in the market. And so having that open ecosystem and an open platform was extremely imperative to the platform that we're announcing tonight. So, Ritika, it sounds like most of the ecosystem still stuck in, well, my persistent store is better than yours and I've got some amount of polyglot and you're reframing the debate. And talk about the life cycle of the data from perhaps the ingest to the semi-organization of a data lake and then the analytics that take place after that and how you are adding value today with AI or cognitive capabilities and what we might expect going forward. That's a great question. We intend to infuse analytics and cognitive capabilities almost at every point imaginable in the platform. And for example, at the point of ingest, we have capabilities that automatically classify the data types of the data that you have, the relationships between that data and how to catalog that data in the metadata store. And then as we access that data, you can actually perform deeper machine learning type of characteristics on that data. You can figure out what type of models that you wanna be able to create from it. We can even recommend for you what models or blend of models you have and you can quickly iterate and evaluate those types of models in a pre-production-like scenario. And then we provide the capabilities for you to easily deploy those into production. That is the power of what we're doing because we're effectively shrinking the time that it takes you to find insights in that discovery phase and then use that same platform to be able to deploy those insights into production. Is that auto-classification capability, is that new, is that... No, you might have heard capabilities from us before called Entity Analytics and we're using our Entity Analytics capabilities on the platform. And of course, we're modernizing it, right? Because we're actually leveraging new metadata techniques even founded on open source capabilities like Apache Atlas to be able to help us with how we store in catalog. But the actual classification of types and finding those relationships is using our Entity Analytics technologies. At the point of creation or use. At the point of creation. So you can scale. Absolutely, so where data finds data. And the thing in what makes it cognitive is as the data changes, so do those relationships. And so that's what is extremely powerful because as you learn more and as you expose to more data as most machine learning projects grow in the amount of data that they operate on, those relationships can also change. So what's the business impact of that? That provides flexibility as things change. You know, one of the number one things I hear from clients is they don't even know what data they have, how to access it, where it's stored and what other data may have some sort of semblance that they want to be able to augment with to be able to find those insights. That's the power of being able to use Entity Analytics because you now know where data is, how to find it, what the relationships are and what else you may want to augment it with to find the right relationships. So if I were to ask you, talk to the chief data officer, even the CIO or even the CFO who's been budgeting money for these data lakes or data as a service or just big data, it sounds like what you're saying is all those stories about talent being really scarce, data scientists and expensive, you're making them far more productive and you're making the likelihood of success far greater. Absolutely, I mean we intend, especially like in the area of data science and the data science profession, to democratize what it means to be a data science professional. If you recall in our June announcement, I remember talking to you guys in June when we launched the Data Science Experience, which is an experience that's built on this platform, we talked about how our number one goal is to be able to help these data science professionals learn their craft, to be able to learn what it means, to be able to code in new languages like R, Python or Scala, to be able to create and build in any language and then to be able to collaborate, that notion of collaboration and learning from each other and that network effect is absolutely important. So we're lowering the barrier to what it means to be able to drive insights from all data, regardless of whether that data actually sits within the platform or it's outside. As a data point, we know that in the next three years, each application will have probably 10 to 15 external data sources associated with that and that means what you know about how you treat data today is gonna change dramatically now and we intend to be able to reinvent how organizations think about that and act on that. So this is, I wanted to bring up the issue of applications. I think, I spoke to Nancy Hensley who works for you, that the estimate today is that, or the current thinking is that roughly 70% of these sorts of pipelines will be to enhance the machine learning within existing apps. How mechanically might that work and then give us maybe some examples of how those applications could change when you have this capability? Well, I talked about one of the greatest, one of the great things that we're doing on the platform is being able to not only in the same model be able to discover and iterate on insights that you have on data but be able to deploy those into production. If those applications, for example, are built in a framework, you can easily take what you have built and operationalize those into applications. We're actually gonna show an example tonight in the demonstration where we take insights that we've developed through a R model and we actually infuse that into an application that has been built within Bluemix seamlessly in the snap of a button and so these type of things are actually real even for existing applications where now that collaboration that needs to happen between a data science professional that is driving and finding those insights and the application developer who is codifying that in their web and mobile applications, that collaboration needs to happen in real time so that you can realize the benefits within your applications quickly. So that's interesting, the relationship, you're affecting the relationship between the data science team and the application development team. Is that a sort of a mesh gray area? Is there a fine line there? Well, it always overlaps and we actually go, we've actually had a lot of internal discussions back and forth on are they tasks or are they personas? Because the reality is in many organizations, the data scientists may be playing an application developer role. But the fact of the matter that remains is that the skill sets for data engineering, for data analysis, for data science and for application development are absolutely necessary to core critical projects in any organization and that collaboration is essential within those skill sets to be able to bring value from that data. And you need to service that independent of the organizational structure or where the skill sets lie in one camp or the other. So would it be fair that I'm being tactical here but the developer, the application developer might take that model if it's our, it might put it on an R server and then the application could call that for predictions. Distributed applications, right? Right, could even be, let's throw in some more terminology, could be a microservice. Well, the platform is built and founded on that core value of microservices and the most comprehensive set of data and analytics services. So absolutely microservice architecture based. Okay, so then it's essentially, we've taken the dev ops culture and processes from the traditional application world or microservice world. Now we're adding an analogous set for the machine learning. That is a wonderful observation. And in fact, we call that insight ops, the ability to be able to take insights that you find in the discovery phase where your data science professionals are actually finding new golden nuggets. When you want to operationalize those into production like environments and you want to do that seamlessly, insight ops. It's about insight operations. Some we call it data ops, we call it insight ops. Insight ops, that's scale. Absolutely. Simple enough that you don't need a rock star data science person to make it happen, okay? One, there's one more step, which is not necessarily in dev ops, which is the data feedback loops that you get from machine learning models where they have to keep getting smarter, preferably automatically. Is that something that you've tooled up for yet? Absolutely.  And that evaluation process is an extremely iterative process before you then deploy in production. And even in production, you're going to continually to evaluate and train those models. All right, we got to go. Last word. We are so excited about what we're announcing with IBM Watson DataWorks, and we're going to talk about a method as well, the data-first method that really helps our clients through this journey, because this is not just a technological journey. This is a journey that involves a transformation in technology, culture, processes, and organizational change. So stay tuned. We're looking forward to the journey that we're on. Love it. We'll see you tonight. We'll be covering that event with theCUBE. So thanks very much, Rilika, for coming on theCUBE. Thank you. All right, keep it right there, everybody. We'll be back with our next guest. It's theCUBE, we're live from New York City. Right back.


Good afternoon everyone and welcome. This is Melissa Armel with a stock swish, and I thought I would review today CMG This is an option trade that I called actually more than a week ago. I guess it was the 11th So two weeks ago, and it's very interesting Yeah, it was Tuesday two weeks ago very very interesting because the stock completely plummeted last week And has just fallen off a cliff ever since and one of the things that this trade is still on for anyone That's in it because I gave option expirations out till August 4th and the following week the 11th I don't know who was still in this though after the drop-off that happened last week But anybody that is it, you know is up a huge amount of money This is more than doubled even since last week's exit Why because I call the 380 and 375 puts and the stock this morning was a low 336 50 something So I mean that is a massive move and in the time that I called it It still has time left to go now the earnings are out. However a report tomorrow night Tuesday evening So we don't you we don't know what's gonna happen on those and we'll have to wait and see So I would suggest to everyone to exit this trade if you did not last week by today you made more money exiting it today and again It's an unknown with the earnings report until they do it But the interesting thing I think to learn about this chart is how you can make money playing institutional moves for follow-through and Continuation and you do not have to wait for pullbacks to get in trades when you know how to play the gap Okay, so what people don't understand about? Charts our gaps number one and number two people do not understand actually that you don't have to do an entry in Something or decide on the directional bias to take something for a pullback or a pull in In this case here, this was a short So if you were looking to get some kind of big rally in this to short it you lost money Or if you were trying to do the reverse and go long the stock Against the sell-off that happened here thinking that it would bounce Which every which has never really bounced at all, but every Smidgen Midsum that you could even construe is about so let me get rid of this when this happened in here after the Fall of here here and then here and then also here I mean if you tried to go on the stock thinking it was going to have a rally back to some area For a quick scalp or whatever people call it you got hurt I mean you got hurt and that is not the reason this continued to drop so drastically The reason is because selling action comes in huge big and fast when it does in stocks panic selling action That's exactly what happened in this all the way back from the gap in here when stock broke 400 on this day in here Fell off a planet on news related something that happened in here and continue to go so This is a great example of plain momentum plain institutional money and how you got to play the gap But you got to know how to do it and so many people don't understand gaps They don't understand how playing with momentum into the momentum into the volatility is the right way to make money as a day trader or swing trader an option trader or anything and actually it's so easy to Make money if you get the direction right with momentum of volatility And I am able to predict that with my golden gap method and I'm able to predict it quite phenomenally Actually, because I did tell people that 350 was a dream target on this In fact, I told people that then when this happened But you know, I didn't give I didn't give as much time in the in the option that I called in this So I called a second one When past it and almost 20 dollars past the dream target and this trade still on if anybody is in it So who knows what it does to you tonight? I Will be watching this and many other things this week big big big week this week for earnings for a big week for the market and Just what a what an amazing show of selling action here in the CMG And I think that people can learn a lot from this chart and people can learn a lot from this chart So congratulations to everyone everyone made thousands and thousands of dollars in this depending on the amount of size you took in it, but what a huge huge trade and You know for depending on what time you got in of the second call that I made which some people have better prices than others But on average if you paid six dollars for one contract and got out here You would have made $18 profit if you were still in it in here today You could have made $33 profit that that's just huge the cost to sell it today was you know around 40 something So that was just phenomenal so congratulations to everyone and I'll look to continue making these awesome calls for everybody on the option letter Email me at Melissa at the stockswish.com if anyone would like more information to sign up for the gap options letter You can reach out to me. Thanks everybody. Have a great day


First question is from C. Greenwood 32. What are the golden rules of training for performance? Well, I mean the the biggest thing I think that I've adjusted with athletes was to really, you know, pay close attention to like how your your joints like like what your range of motion is the integrity like the movement quality in general I think that we we kind of leap past all of the the prerequisites towards training and I think there's just so much attention there that needs to be had because these sports require so much out of our bodies. I mean we're moving all different directions we have to stabilize we have to like provide power when we need it so it's like so much you have to build upon that that if you haven't put the work in there I would immediately drive them back into like quality of movement how we need to address each individual joint specifically I think another golden rule that we have to list because we I think we revisit this every every week when we get a quaw is nothing is going to make you better at playing your sport than playing your sport yeah in fact I would name that rule skill is the most important thing right skill over anything else because it's got to be the most it's a very I mean every week we have a question related to sports and also looking good and people I practice my sport more or should I work out more right and it's all if you want to be good at your sport nothing is going to make you better at that than than actually that's great so I would say definitely skill over anything else is always number one and then when we get to physical performance just instead of good one mobility is very important lack of mobility isn't one of the number one reasons by people hurt themselves and then you can't play your sport at all and another one I would add is that strength for most sports or for most physical pursuits is the foundational physical pursuit meaning what does that mean that means if you get stronger you generally improve your performance and other metrics you can't say this for other metrics like if I make my speed better my endurance better my agility better I don't necessarily get better and everything else but with strength we tend to now this doesn't mean you just focus on strength and you ignore everything else but it does mean that strength is very important and it's more important the younger you are and the newer you are as you get older things get more fine too what I mean my older is more experienced not older like you know I'm in my 40s and 50s but rather I've been playing for years and years then it's less important but when you first get started you want this kind of general strength because it's going to help you a lot a real strong base to work with and that's why you guys brought up skills like it's hand in hand you never lose sight of skills training that that has to evolve as you get stronger as well otherwise you know there's there's going to be a big disparity between the two once and this happened to me even just focusing completely on getting bigger getting stronger but now that affected my actual movement and my performance on the field because I wasn't keeping maintaining up my skill now what about like the said principle right like I mean what you guys what you train is what you're going to get right so if you have somebody who is trying has a very specific sport there's obvious specific movements that carry over that are going to benefit you versus doing real general I know you made the point general strength is the foundation but then after that point if you're a tennis player trying to get good versus a football player the specific exercises that you are doing is going to be completely different alignment doesn't require as much rotational strength let's say like a tennis player is going to right I'd say the the more advanced in competition you are and the more experience the more important that is right if I'm training a a new athlete general strength is going to give them the most bang for their buck but when you're training someone who's been playing for a long time you want to start to focus more on specific types of strength and mobility plays an even bigger role right because as you get bigger faster you're playing against people that are better mobility becomes oftentimes the difference between a winning team and a losing team typically I mean if you're at your peak performance that's what's going to carry you you know and and like provide longevity in your career is to really address those things and I think that's why I brought that up first is because it was something that was completely like breezed over like we'd have like a barely even a warm-up and then we get into like this really intense workout or you know intense practices and and people are dropping like flies and and the reason being is is there just wasn't that quality control and attention that well to that point another golden rule I think is that the the real work is done in the off season so I think there's that that was like the same I mean I used to talk about even competitors on stage that's what I tell them and I think athletes are the same way like the real work is done in the off season as far as how hard you're training and getting after it once you get into season play just trying to prevent yourself from getting hurt that's right like then it's more about protecting you than it is progressing and so and it becomes all skilled mostly at that time and taking care of yourself so the real work is done in the off season


Happy Shohei Otani day everybody Shohei Otani on the slate for tonight in daily fantasy baseball along with some other pretty interesting pictures overall I think it's a pretty fun slate a lot of ways you can play these things you got some decent I would say under the radar picture so I don't think a lot of people will be on so overall a fun slate let's dive on in and get you all set this is the Fanduola live Q&A right here on the Fanduola YouTube Twitch Facebook and Twitter channels my name is Jim Sonnis I am a senior writer and analyst for Numberfire.com here to take your questions live on air for the next half an hour to get you set for a Friday night slate of daily fantasy baseball as always no matter where you are watching you can get your questions in YouTube Twitch Facebook Twitter no matter where it is put those questions in the chat we'll take them live here on air to get you set for tonight's slates next week we've got some NFL stuff coming down the pipe so that's going to be a whole lot of fun we'll still be going live at 4pm each day so the time will not change baseball though once we get an NFL stuff started 430 the NFL first followed by baseball either way we're going to keep this baseball stream going until the end of the regular season so plenty of reason to stick around here on the Numberfire daily fantasy the Numberfire streams and the Fanduola YouTube Twitch Facebook and Twitter pages let's start things off with John over on YouTube John is asking how would you stack Houston tonight John I do think that Houston is a good spot to go facing Jake Arietta Arietta really struggling so far this year especially recent that's why he was releasing the Cubs now coming off the IL for the Padres first start didn't go that well so I think that thing set up pretty well here for Arietta or for the Astros facing Arietta and I do like the the Astros the one problem we have John with regards to filling out a lineup is not really sure what it will look like for tonight because it could be Bradley sitting could be your Don Alvarez sitting my hope is that it's Bradley who sits because I don't want to use him I want to use Alvarez so my hope is that we get a situation where Bradley is the guy sits as they use a DH and I can go to your Don Alvarez and the other really fun guys on this Padres or this Astros team let's pull up the Astros salaries over here I'm probably going to wind up being pretty high on Kyle Tucker for today $3,300 a guy who has a lot of upside because he has both speed and can swipe some bags so Tucker low salary can swipe some bags I'm probably going to be pretty into that let's check out Alex Bregman see what he's been doing here since he came back from the IL it's a situation where I want to you know check out the the numbers he's had recently given he's coming off a pretty extended injury make sure he's hitting the ball well seeing the ball well we can feel good about him because it's not a fragment it's pretty forgiving at $3,600 so if we can justify being into him I'd be pretty jazzed about that okay we got about a six game sample here he's had a couple barrels 50% hard hit rates that's actually looking pretty good so I would say Bregman we can get the green light on him so let's get Bregman in there going to gloss over Yuli because it's a righty I want Altuve in there and I want Alvarez I would say this is probably where I'm looking for a four player stack of the Astros I think that with if you like Nathan Yavaldi for tonight you can make that work it does force you to try to find some value plays but I think that works so this is probably the core four for me John in terms of stacking the Astros if you need to skip over Altuve or Bregman or Alvarez to get there that's fine depending on what value pops in there but that's probably the main four guys I'd want to turn to for tonight let's talk to you Renee on YouTube any interest in the amazing mats yes I think they're actually pretty good for one os for today because they're facing Sean Nolan they just saw Nolan last week and Nolan was pretty good against them did let up a couple of home runs and they've got some righties who can hit lefties pretty well so I would say looking at guys like Kevin Pilar specifically for value if he's in there I've not seen their lineup yet but if Pilar winds up playing he's minimum salary got some power can allow you more flexibility and using the Astros and stuff like that so I'd say Pilar's in play Alonzo's in play and checking out some of the other righties there so I would say yes Renee I think that the Mets more so for one offs than stacks definitely a team want to go to Bayez lindor etc etc but I think that Pilar specifically a guy who I can definitely see myself being on for tonight James says 14 games are a lot and he also likes the express slate over on fangirl James in general I am pro express slate because narrows down your research gives you a lot you know a lot fewer things to look at I think that if you are someone who's you know working all day Friday doesn't have a lot of time to do a lot of research the express slates are a really fun way to get around that personally I feel like I've had more success with smaller slates if you're playing a big slate like this I think that's a good time to go to you know single entry three entry max type tournaments to make it where you can't you're not facing people have like every combination of every stack so yeah James I agree I think the express slates are great I think that single entry three entry max type things those are very good for today would recommend and good luck to you in those let's talk to Julius he did a mid-level picture for tonight I think there are two Julius I would say the primary one to me is Nathan Yavaldi basing off of Cleveland he did just see them as the most recent start so familiarity here is high and I hate high familiarity they just saw him they know what he's doing stuff like that that's very concerning for me always because it does matter we see we tend to see guys fall off in that second start but Yavaldi really good picture he's been using more curveballs over his path seven starts 28% strikeout rate that is the second highest mark on this slate behind Freddy Peralta who is not going to be on a full leash coming out the aisle low walk rate for Yavaldi Cleveland not a bad offense definitely better now than they were earlier in the year but still not one we need to avoid so I'd say Yavaldi's a good mid-range pitcher and the one I wouldn't expect many to be on today is Vladimir Gutierrez for the Reds facing the Tigers the Tigers 89 WRC plus versus Reides and a 26% strikeout rate and now they lose the DH and go into a National League Park so Gutierrez has been leaning more on his changeup recently trying to cancel out what lefties do against him seems to be working in the past six starts the more changeups a 23% strikeout rate still letting up a lot of hard contact so it's not a guarantee to work well because there are some paths to failure for Gutierrez if you let up fly balls and hard contact at great American ballpark bad things can happen so he's risky but I like the upside here so I would say Yavaldi and Gutierrez the two uh mid-range guys I'd want to turn to there Jimmy on Facebook says Altuve crushes Arietta yeah I think that Altuve is someone we should feel pretty good about for today he's good against lefties good against Reides whatever it may be so I think that Altuve definitely a good one there let's talk to James on YouTube how do you feel about an Atlanta Colorado stack I like the Atlanta side of that one James facing off with Antonio since the tail has since the tail is a pretty good pitcher but not gonna lie like he's been pretty solid um he gets ground balls and stuff like that has had really good results but he does live dangerously for a pitcher of course field a lot of balls and play does let up hard contact the Braves got some bangers in there so I'm okay stacking against them I think the Braves are my number three stack for today I'm okay putting you know the uh the the Astros and the Reyes above them but the Braves are number three as far as Colorado I really like Huascar Inoa I think he's a talented pitcher he's looked good since he came off the IL as well so I'm okay avoiding the Rockies pretty weak team versus Reides Inoa a good pitcher so I would say thumbs up on the Braves not as interested in the Rockies for today but I do like the Braves quite a bit Kenneth how do you feel about Otto tonight as a pitching option so this is starting number two for Glenn Otto facing the Angels the Angels not the toughest team to face right now by any means Otto the problem I would say is he went just 73 pitches in his first start so that would be a concern for me we might not see a lot of length out of Otto so I would say probably know for me he's on the road stuff like that not a lot of length so personally an avoid from me I'd rather go to again with Gutierrez or Ivaldi for tonight so Vann best stacks for a single entry for today so I feel like the Braves are probably going to be pretty popular because their apply total is so high so to me I think the Astros and the Reyes may wind up being really good single entry teams just because I wouldn't expect them to be as popular as the Braves for tonight so I would say the Reyes and the Astros great out pretty well maybe get some value there I think for a single entry pitcher I'm probably going to go Ivaldi or Gutierrez over Otani for tonight but so may not need that salary savings but I think we it means we can go with Austin Meadows we can go Brandon Lau there's no wonder Franco for today for the Reyes who's kind of a bummer but I do still think the Reyes great out well Jackie is asking Jean-Carlo Stanton home run today uh facing John Means John Means does let up a lot of fly balls not a big hard contact guy this is eight stars since he came back but a lot of balls in play a lot of fly balls Jackie thumbs up on Jean-Carlo Stanton I think that's a good one for today Maurice is saying CJ Upshaw not sure what that is uh Bastruder is saying um let's see here let's go to um Rich he says he's looking for a 50-50 lineup uh he needs to ice the beer down uh Rich not gonna give out a full lineup for today uh because you know we the goal is to give you the tools to make your own lineup I would say if you're looking to build a cash game lineup I feel like you might just want to go Otani because the safety is pretty high with him I would say though if you're not going to go Otani definitely going to evolve the overall Tani because you know again it's a repeat matchup which is is scary for a a cash game situation he's a good pitcher um Cleveland not the most threatening offense so I'd say you have all the good cash game pitcher for today chill introvert like the name uh Atlanta bats worth the salary or are there lower salary bats in stack I do think Atlanta is worth the salary they did get jacked up in terms of salary between yesterday today which they should have because they were super under salaried I would say they're worth it chill uh but if you have like a single entry stack I think that the Braves are what I would not seem to be a must have I think that they are a team I'd like to get to but they're not a must have for today the one reason I am very accepting of the Braves is that the other teams that I want to stack the Astros and the Rays are not like low salary either so I'm still on board with them for sure let's talk to Kenneth do you like Adloberto Montesey as a one-off today yeah I do like it a lot felt based in Dallas Keichel uh Keichel letting up a lot of hard contact right now no strikeouts uh Montesey look good in his rehab stint hit a dinger stole a base in his first game back I think he's awesome Kenneth uh full thumbs up to Adloberto Montesey and Salvador Perez assuming he plays today as well uh didn't check out their lineup before we started here I assume Perez is in there he is oh Edward Olivares is batting forth that's fun wow okay I might stack the Royals I don't know uh we can go Perez Olivares Montesey and Taylor that's kind of fun for a value stack so yeah let's do it let's let's party let's stack the Royals for today Salvador says reporting lives somewhere in North Philly I also like the Phillies okay so Salvador is on the Phillies and I don't think that's a Homer call saboteur they're facing hey so Lizardo uh Lizardo has obviously struggled since he since he I mean the entire year honestly uh but since he joined Miami this is actually a smaller sample than that this is uh he's been lowering the usage on a sinker in that sample 5.33 skill interactive ERA 20 strikeout rates still a lot of walks not the best contact profile that can be a Philadelphia for today the reason why I am not putting them my top three is Lizardo did look pretty good in his most recent start eight strikeouts against Cincinnati Cincinnati's worst versus lefties in the Phillies are but Lizardo is a guy whose talent I believe in the game is in Miami so that's why they're not higher on the list I am not saying not to use them I think I will get there but that's that's the reason why they're not higher despite the fact Lizardo overall has struggled quite a bit Renee uh picture recommendation on the $555 entry kudos to you Renee uh I would say in that situation I'm not I would play things the same way as I would if it were a $3 entry just depends on the the field size there so to me I would say I'm probably looking at Vladimir Gutierrez like why not live life a little bit dangerously I feel like people probably won't be super on him because he has struggled for the full year but I like the changes he made like the matchup so I think Gutierrez is a guy my turn to there if I'm trying to roll the dice shoot for offside I think that Gutierrez is pretty solid option Salvatore Phillies stack or one-offs so asking if you can stack them or do one-offs I'd probably go for one-offs more so than a full stack because with Reese Hoskins being out it's it's a rougher lineup than it was previously but Freddie Galvis I think is a good one-off Salvatore um I think that I can feel pretty good about that one so Freddie Galvis yeah he's bad lead off there McCutcheon works Harper it's a lefty boo who cares uh he works too so I think more so for one-offs than a full stack uh let's talk to Jackie over on Twitch Otto over four and a half strikeouts is my call for him again I'd worry about pitch count there because he was at 73 last time um I don't have the data right in front of me and where he was in the minors but I'd be a little bit worried pitch count and betting it over for that one uh Salvatore one utility spot uh with $4100 to spare okay let's go to utility probably gonna be Altuve or Alvarez actually let's go Alvarez I love Altuve but like you're done Alvarez if you play see my knocks again they lose a DH for today but if Alvarez is in there he's really good we want to use lefties against Arietta I find it's righty stube like you're preferring lefties so I would say Alvarez followed by Altuve as far as other options probably looking at uh Coors field if we're not going with the Astros um okay if Alvarez plays which he might not sounded like there was some optimism you play yesterday he didn't uh Austin Riley would work there too so I would say from ranking those options Alvarez one Altuve two and then if Alvarez plays three if not Austin Riley number four uh Jackie Jim uh Jim can't give all his secrets we can't handle that much knowledge no there's there's no knowledge for Jackie believe me and it's more so like again my goal is that you don't have to watch this show like if we can like give you the tools do your own research and make this show like obsolete not necessary that'd be awesome like the fewer viewers the better you know we want to make it so uh you can do all this yourself that's the goal at least to to try to give you the tools to beat six all Devon is asking if there are any under the radar stacks for today so let's go over here the way to try to identify under the radar stacks because a lot of people are just going to stack the teams with the highest implied total so what you want to do is look at implied totals try to find some ones that may be a bit more middling that people may gloss over if they're doing a quick land so it won't be the Braves won't be the Yankees won't be the Reds not the Rockies I actually don't not super into Boston Cal Quantra's look pretty good recently so if we're looking for under the radar actually the Rays and the the Astros great are pretty well there uh Rene's Mets look pretty decent there so I think that this middle chunk here in the uh the high fours the implied total actually is pretty good the Rays Mets and Astros all teams I like a lot but also not going to pop in terms of implied total so I think that these are all pretty interesting for me uh Savannah's asking is there any weather to worry about I didn't see any earlier I think we should be good to go there and finally I need help narrowing the slate can you help me decide which pitchers I have to target for today so the pool I would say a pitcher for me at least is Shohei Otani Nathan Diavaldi Vladimir Gutierrez I know Gutierrez is risky so I understand if you don't want to go there I totally get it that's totally fine if there are other options Adam Wainwright's an option I worry because you know the team's not that great uh but he's a good pitcher facing a team that will strike out I think that Wainwright is someone you could talk yourself into I would not push back against that everyone else kind of kind of tough I like the the talent on Manoa but a tougher matchup a lower strikeout matchup so I would say probably those four Wainwright being if he I probably won't get there myself but like I can see it for sure diffuse is asking about the Reds so the Reds aren't very good against lefties that's that's one concern 89 WRC plus a 153 ISO they're a very different team versus lefties and righties Tyler Alexander is a guy I've stacked against a lot this year but he's been making some changes recently been going shorter in games he has been more effective as a result of that so I would bet the Reds are pretty popular I'm okay being a bit underweight on them I will say though that for one-offs they're very interesting because um Tyler Stevenson batting second is a guy with a lot of power versus lefties RAC as Aquino has a lot of power versus lefties Casianos probably gonna make something terrible happen and go deep tonight Jonathan India has been really good so I think for one-offs they work and I'm okay going there for full stacks I'm okay being a bit lower than consensus um just because I do worry a lot about them being very popular the fact they're bad against lefties and that Alexander has been improving recently let's talk to Johnny on Facebook is bumgarner a strong play tonight Seattle goes up and down obviously looking at projected roster tonight Seattle has quite a few left-handed bats in play yeah I like the thought process there for you Johnny the concern that I have is that bumgarner's strikeout numbers have been pretty rough this is in his past seven starts with more change-ups 4.88 skill interactive era 17% strikeout rate letting up a lot of fly balls so the matchup is really really good and if it were strictly matchup based strictly um the roof is also closed tonight at chase field that helps as well I just really worry about the that about what bumgarner is doing right now so I like the matchup totally agree to thoughts there it's harder for me to talk myself into the picture himself given the low strikeout rate recently uh Mike also about bumgarner for tournaments yeah I think again I won't be getting there I just need more strikeouts out of my guy um savannah's asking how would you stack Tampa so again no wonder franco for today he had a headache last night and left the game he said he's feeling better today but not in the lineup so we can look at this one here kevin curemeyer batting sixth is interesting so let's check out the raised numbers versus righties I was hoping Brett Phillips something's healthy was in there but that didn't happen um I'm sad about that but whatever it's okay my feelings are always on the aisle that's why huh okay so cross out these guys here and let's go check out curemeyer see if we can talk ourselves into him batting sixth 140 iso six stolen bases that's that's not bad I think that that's okay given the low salary given that I do have some high salary stacks I want to get to so let's actually be okay let's go meadows first then let's go curemeyer then let's uh go to lawitt second base because I want to feel second then the question is do I go with nelson cruz do I go uh with yondi dias do I go with joy wendell wendell he hasn't been very good recently from a power perspective and I care a lot about power so let's look at the bat of all data because last time I checked which was I think on like monday or so sometime when I was on the raise wanted to talk myself into him couldn't quite do it so it was an effort that was made let's pull up here let's go let's go august 10th for funsies since then three barrels for wendell a 36 hard hit rate not loft in the ball that much it's not ideal not ideal uh check out the iso here see whether the results say 169 iso I guess is not the worst so wendell's fine I guess I know it's the most inspiring endorsement I could have given of him but like you know if you go with this raise stack $3,300 that should get you to the astros could get you to the the braze are kind of tough there but I think that this one works so I think this is a decent stack for the the raise I would probably want to get to Nelson Cruz if I could over wendell but I think this is probably where we're looking to to exploit the fact that Randy Dobnack puts up a lot more fly balls to lefties than to righties so far and smart to snack against their stack against the Phillies pitcher for today and Salvatore also asked about Cal Gibson he said I'm getting to Nathan the Evaldi Cal Gibson at shohio tani so let's talk about Cal Gibson in general he's made some tweaks with the Phillies he's been throwing more cutters I don't know why the cutter is not like a good pitch for him it's the pitch he used a lot in the opener when he led up like 97 runs while getting two outs and for some reason the Phillies pitching staff has said let's use that more hasn't worked he has a 14% strikeout rate with his six starts at Philadelphia he's getting ground balls the matchup is great I just I don't like the strikeout rate so I'm not going to get to Gibson don't want to stack against him because Philip or Miami's offense is pretty bad but I also don't want to use him as my pitcher for today Savon I want to fade the White Sox uh starting pitcher in Stack Kansas City but Kansas City has been putting up a lot of runs recently they're not very good so that's probably why um I think that the good thing is they did get Montesey back he's a pretty big boost that entire offense let's actually build out a royal stack here now that we know Montesey's in there we know our boy our shining boy Edward Olivares is batting cleanup as well I need the one off I'd say Olivares is really good there uh let's put Montesey at shortstop let's put Perez at catcher slash first base let's get our boy here in the outfield Olivares $2,100 oh it's so so sweet I'd love to get to hard hit wit but like I feel like the best utility of the world is giving me sour savings let's go with Michael Taylor Michael A. Taylor should say um sorry to all their Michael Taylor was out there 3450 that's quite a lot of juice to get to the the Astros Braves uh Rays if you want to as well so I think that's a pretty fun stack on the Royals so I do think that although they've been struggling as you said Savon I'm okay getting there uh Savon what teams are you staying away from today um let's see here so I feel like I'm not staying away from the Reds I feel like I'll be underweight on them just because you know I again struggle against lefties Alexander's been getting better recently I don't think that they're a cross-off by any means like they're still you know Stevenson specifically is pretty good Costianus is pretty good uh I'm okay being lower on them though as a uh just because I do think that you know I think that people will probably be pretty high on them so I'm okay being a bit lower on the Reds than Consensus other teams I'm avoiding probably not going to be on the Rockies I know they're implied totals pretty high but I have a lot of respect for Inoa uh the Rockies not very good versus righties so I would say going to be lower on the Rockies probably lower on the Reds than Consensus probably lower on the Blue Jays I know Maniah has been bad but like I have like this fondness in my heart for Sean Maniah so I just like watching Pitch so I would say lower on the Jades and Consensus lower on the Reds and lower than or lower on the Rockies and Consensus it's not to say you shouldn't use them if your process led you there go ahead but I feel like I'll be lower on them than others for today let's talk to Jerry what's different between a pitchers park and a hitters park your definition good question uh it's based on so like based on like data is how I try to form it um so like here you see the park factor um a higher number favors batters whereas a lower number favors pitchers and there are a lot of things that go into that primarily it's temperature which is why you can see here I have the temperatures here like let's say it's April and everywhere else it's 40 degrees but it's 70 degrees in Tampa Bay that time of year Tampa Bay is a hitters park relative to other parks now when it's 70 degrees elsewhere our Tampa Bay is pretty neutral but like in the summer like in July early August Tampa Bay was very much a pitcher friendly park because it was colder than other spots so I would say temperature influences a lot Jerry and we have some places where they can be a pitchers park one time a year hitters park other times in general the rule of thumb is you want to maximize temperature for hitters minimize temperature for pitchers um that's kind of the rule of thumb for defining things but things such as dimension altitude all that do matter as well if you google ESPN park factors or rotor grinders park factors you can find different park factors rotor grinders breaks it down by handedness which I think is pretty helpful so I would check it just google rotor grinders park factors that's a pretty big help to see what handiness helps what gets the benefit at which parks and stuff like that home runs triple stuff like that I would check it out uh Aaron have a good weekend Jim where you at background different I am in Maryland um I'm here for a wedding this weekend so I'm at a hotel I've never done one of these just some hotels so hopefully the neighbors aren't too annoyed sorry uh that I'm talking loud at myself in this hotel room but you know uh hopefully they're not too mad at me we'll see uh Salvatore 3000 left in utility where are we going here okay let's go over here and let's check out the utility slot there are a lot of dudes on this slate it's a lot of scrolling to get down to three thousand dollars okay all right so Akil Badiou I mean he's got a you know he's facing Vladimir Gutierrez so I do like but Badiou's probably under salary I would say at three thousand dollars got a lot of power good hitters park A. O. Henio Suarez could work as they won off there versus Alexander and don't want to use Lopez I can use Wendell I guess but again not super enthusiastic about doing so Ahmed Aguilar only a dumbest facing Wainwrights farmer's not bad I would say ooh Rogers is facing you know what that's kind of tough McCutcheon could work I would say McCutcheon might be one of the better options there $20,000 facing a lefty uh Lazaro has struggled quite a bit so I don't mind Wendell but would really probably not get there just for a one-off for one-off I just want to maximize Dinger potential so Suarez so let's say here Suarez is an option I would say McCutcheon is an option those are probably the two best ones in terms of one-offs around $3,000 Renee any Diamondbacks maybe Marte Renee you changed your profile picture during the stream very impressive I like the the the dual efforts there um any Diamondbacks maybe Marte okay the Diamondbacks are facing looks like Anderson that was not official like they had not officially announced yet whether he would play Marte amazing against lefties so you could definitely get there I would say though that Anderson has done a good job of expressing our contact that's why I'm a bit lower just because Anderson is pretty good at that but Marte is very good against lefties so he's not out of play but the the Diamondbacks to me not as interesting Aaron says at least you brought that fancy camera and Mike I did I had to pack in my suitcase I brought the tripod too which you can't see because the camera is on it tripods here didn't bring the lights which is why I look kind of weird but like you know we made an effort trying to get the people their info on a Friday afternoon final one Sivan finally got a DFS lineup optimizer I hope it was from Number Fire Sivan I hope I hope um by draft dashboard trying to see it out what's up any suggestions or tips and lose using these again make sure you do not use default settings like try to alter stuff um because you don't want the default for anything because like someone else is going to have that default too you want to make sure you have a unique lineup so alter things try to find teams you're higher on uh try to lock into picture and stuff like that tweak stacking settings if you can as well alter things to make sure you're not getting the default that is not the way you want to use an optimizer but again check out numberfire.com uh check out the sharp stack over there Sivan that one might be free I can't recall either way I guess one more from Jerry do you like stacking the Mets in the Washington game mix in the Atlanta bats I don't mind some Washington one-offs against Rich Hill and I don't mind some Nets one-offs so Jerry no objections I think that sounds good and Jerry likes a Bayez home run call I hope so I'll be Bayez I know it's been a weird week for him but I'm always pro I'll be Bayez and I'm always pro all of you because you turned in for today for this fan to a live Q&A we are back once again next week on Tuesday not a Monday because it is Labor Day so I will not be here I suggest you're not here either because that'd be kind of boring uh we'll be back once again on Tuesday though no PGA because no PGA event next week we'll have baseball though at 4 p.m. YouTube, Twitch, Facebook and Twitter make sure you are subscribed there if you've got more questions for me I am on Twitter at Jim Sonnis J-I-M-S-A-N-N-E-S big thank you to Joy Affleck our video producer for running the video side of things here today thank you Joy as always and thank you to everyone for tuning in good luck tonight have a great weekend go cats and we'll talk to you once again on Tuesday this has been the fan to a live Q&A