So combining fuzzy logic with this ECG game, that's essentially a bot that will be playing the game for us or in addition. So we could like compete with the robot. That's now a noise option. And that's mainly to see how well the robot is doing when the noise is increased. So currently it's just generating all this false alarms. We will be fixing this before publishing the tool. And yes, you can still play yourself. So you should not click on that one, which was a normal ECG and this one missing a bit at the front. So we are not clicking on it, we're getting correct rejection. And when clicking on an abnormal ECG waveform, we get a hit. And that's what the bot does as well. We had a tuning of it before that just works 100% of the time, but it was relying on some magic numbers that we now removed. The system now performing much more. So it's essentially clicking on everything. It's making a decision that everything is abnormal. And we will be fixing that. We're looking at the, what's it for, parameters, positive peaks, the number of positive peaks, the number of negative peaks, amplitude, and the R peak sharpness. So obviously now it can win the bot, but we pretty sure can easily make it work much better to the point where human can sit back, relax and watch the robot in action. And so we have this four files, obviously this HTML that you see on the screen. So the waveforms are being generated using simple math equations. This is all happening in JavaScript. So front end and the fuzzy logic, which I'll get to in a sec, is working back end. So the fuzzy logic, the robot doesn't know what the current waveform label is. Oh, the robot is cheating somehow. No, it doesn't have the labels. Now we have this game logic, doesn't do much, or might as well get the co-pilot to work just by explaining the code quickly. And we'll try this one shot prompt, single shot, because it tends to work better to begin with and then performance drops with consecutive prompts. Here we do constants, waveform functions, variations. Yeah, we have a bunch of variations as you can see as you saw in the game itself. Generation and noise addition is also in JavaScript. And now the big reveal, the Python code. So this one has the, it's a flask application and it does the fuzzy logic business. Game logic does quite a bit, initialization, drawing, the game itself, so it does the scoring in the game, calculating the scores. Yeah, probably should break it up a bit, but it's, yeah, 200 lines of code, it's not too bad. It does the waveform analysis, noise level adjustment and starting the game. Now the Python code, this message is too long. Okay, so the Python code, yes, it's 400 lines of code. Yeah, the port currently as it is, it just keeps saying that everything is abnormal. You can still get some points out of it. Well, because the algorithm generation, I think is biased, I think it has something around there. The other thing with the core pilot, the core thing about it is we can use this workspace and actually ask how, what's the ratio between normal and abnormal ECG waveforms. So it will try and find the relevant bit supposedly in all the files that are currently open. Should tell you the reference as well. So it should tell you what it was looking at, but yeah, it's taking some time. So it looked at the waveforms.js and the HTML file and it didn't produce the result, which is unfortunate because I think it's 60, 40. Check, it was looking at the wrong file. I vaguely remember something about the 60, 40, but I can't confirm where is the generate waveform noise level. Yeah, this one, 0.6. Yeah, so it is 60, 40. Well, that's why by clicking on every waveform, you are able to overall over longer periods of time gain some points, but you won't be doing it very well because you're producing all these false alarms. So yes, there's the 52 to 38 ratio. So yeah, this is the kind of stuff core pilot is not being able to handle even though it's fairly simple. We are developing this Flask application that we would like to publish today, but I need your help. We need to update the fuzzy logic algorithm parameters as in the rules and the membership functions. We also got some examples of the ad output that I'm getting, can you look at the fuzzy logic analysis result and obviously we also having an issue with the number of peaks being calculated. I assume this is to do with the noise. Last time we were trying to add the basic filter but if we do do that, we'll have to show it to the human playing the game as well to keep things fair. Can you suggest the specific code changes or prompts for GitHub code? So if you could generate code with placeholders, that GitHub core pilot could fill out, that will be great. Please go over all of the code provided and the images uploaded. For the fuzzy logic part, you need a balance of specificity and sensitivity. You don't want to tag normal variations as abnormal, false positives, nor do you want to miss actual abnormalities, false negatives. Adjust your membership functions to reflect the degree of certainty you have about a segment's abnormality based on the input variables. GPT-4, yes, single, single shot prompt.  However, the code is not quite there. So it's suggesting, it suggests to rename the sharpness and that should be fine. We don't have to do it. Oh, it's actually changed it from triangular to trapezoidal with a triangle, triangle in the middle. Now, we already have this code. I'm not sure why it's generating it again. I don't think this will make much difference or a comment that out, but in a comment that that was a GPT-4 suggestion, and noticed that it's fine. It's doing the same for amplitude. And yes, using trapezoidal membership functions instead of triangular ones, which, yeah, I think it will make much difference. I used trapezoidal ones before to kind of make more sense, but the triangle ones should work just fine. Let me show what this side of it do, make these suggestions. If we can say co-pilot can do the right kind of thing. Right, now the rules. The rules is something we do have to go over. Now, yeah, when noise is removed, so that's actually what we are trying to show is that when there is no noise, the algorithm, the machine, and the fuzzy logic algorithm is doing much better. Let's say playing this as abnormal, which is correct. Yeah, it's now started moving much slower because we, the human went in negative score. Let's go, that's cause I wasn't paying attention. It's a normal issue, so I should get 10 points. Yeah, I'll need 12 correct responses to get back to zero. The machine is doing okay-ish. So that's after lowering the noise level all the way to zero. So the false alarms and misses are at 57 and 21. Ideal, it will just stay that way. Yeah, so if no noise, the fuzzy logic is doing pretty good. I think it might not be making any mistakes whatsoever. So for the demo, we might just add the low amount of noise where it does mistakes sometimes. No, it just didn't miss. So it was an abnormal waveform and it didn't click on it. So it's making mistakes even when there is no noise. There's some sort of basic adjustments that we need to make the positive peaks, three and one. There's some working okay. The amplitude, it doesn't look like row amplitude. The amplitude doesn't seem to be changing at all. So it was 0.83Y, now it changed to 0.5. Right, when you don't have the negative peak, the amplitude is changing. 0.58 seem to be the maximum amplitude, which is a bit odd, it should go to one or as close to one as possible. Yeah, so that the amplitude is probably not, oops, the false detection of a hand. Yeah, sometimes falsely detecting a hand on my ear or something like that, the music bit, I don't know if you call it music or not. I do, when the false alarm stayed at 57, the misses went from 21 to 28. So I think it's missing a very specific waveform. The thing about it is that we don't want to tune it up. We don't have any, we don't want any magic numbers that make the algorithm work better. We want it just to just give it the general instructions and for it to just do its thing. Let's go over those suggested rules. I need to explain each one because I forgot what they do. Don't fit the screen as well, which is not good. Yes, those are, again, GPT-4 suggestion. Don't think we will be testing this because that's not what the problem is. Write the peak detection. It's a suggestion to change the prominence value. And yes, we do have a prominence value. They currently, yeah, every time there's something funny on your face, it's a prompt for me to have a break and listen to some music. Hopefully this music is not copyrighted. Assuming if you create a musical instrument then the music is yours. I don't know, what do you think? Just a quick recap. We tried this single, what's called one-shot prompt of GPT-4 to try and fix some things with this phasologic algorithm that is playing this ECG scoring game. That's currently hallucinating a bit. I'm pretty sure one of the waveforms that never detects correctly is why they miss us. So one of the, it's always missing one abnormal ECG. And yes, I know which one it is. So this is kind of unfair for me to do this because I know exactly what the waveforms are. Later, at the later stage, we also try to add because this synthetic ECG, we can get add like an infinite number of different waveforms and see how the algorithm responds. So the rules should just be logical, nothing fancy. So we'll get the error in GPT-4 was an error generating a response. So obviously if I click regenerate, it will start again. It was a long prompt and a long response. What if I just refresh the page and I haven't finished when it's in this continue generating position that actually gives me an option to interact with it again, but don't have to continue. If I do continue, I suspect it will give me the same error again. Yes, that's right. So it's behaving in an odd way. I don't know, it's just for me. Is it because I'm on using the Chrome browser in Linux? I was thinking actually doing a tool where it's using an API and my Python just prompts the GPT-4, whichever model just continuously non-stop, obviously monitoring the cost. Maybe it will be even cheaper. I mean, you can always cap it at whatever, $20 a month that GPT-4 costs. Sorry for that, that's how drinking sounds like. This is the other hand. I like the other hand much better. I was saying I have to change the hands because I can do both at the same time. That was the idea, but essentially I'm playing two musical instruments at the same time. I wonder if there's any musicians in the crowd who would like to test this and provide the feedback. Okay, I don't know, GPT-4 just got stuck on me. Yeah, there's no point asking you anything about the working, the behind the scenes of how this terminal actually works is there. So I think we will just have to leave with it. So you generated pretty generic responses. I'm trying to get to the specifics of the code. Now it seems that the moment in the past we removed the frequency metric. We were looking at the frequency as a parameter and now the result of it is that we always get a miss for a certain type of wave forms, synthetic wave forms that have the R pick too close to the T wave. This must be because we removed the frequency component. How would you suggest fixing it? I would like a specific code. Yeah, I'm thinking should we bring the frequency back? The reason we removed frequency to begin with is because we are not looking at the continuous ECG wave form. We're looking at each QRS complex one at a time. So it kind of didn't make sense to look at frequency. But obviously now it seems like that might have been a mistake. Because if the R pick is moving around, coming too quickly or coming too late, we don't have a detection mechanism for that. We could of course look at distances between peaks but then we'll have to make an assumption. Assumptions as to which peak is which. And also if we have a missing peak that would not work as well. Any suggestions? Welcome. One of the peaks might be missing. So how do we deal with that? When calculating intervals, how would you make an assumption which peak is which and which one is missing? Suggesting to add these four intervals that yes, we have similar stuff obviously in the wave form generation. We have these two QRS variations. So one of them will have a gap between, so the R peak comes late. However, the phasological algorithm, the port, the machine that is playing the game should not know about how the wave forms were generated. So I'm suggesting adding three more parameters. I don't think this will work because again some of the wave forms might be missing some of the peaks. Not might. They are missing some of the peaks. So we're implementing a phasologic system to analyze some data. Yeah, it doesn't know what data we are looking at. Even though it could have looked at the names, likely related to ECG readings. Okay, it did do that. To actually read the whole thing, don't you? Phasologic is the form of many-valued logic in which the true values of variables may be any real number between 0 and 1. Yeah, so all the numbers have to be normalized, I don't think that's the case at the moment, especially not when we have noise, then the number of positive peaks just shoots up through the roof. Yeah, when we remove the noise, it just gives the correct number of positive and negative peaks. It might first sort out the no-noise situation that will kind of make sense. We have the frequency component, sorry, I keep repeating the same thing over and over again, but that's important. That's why our system is not working at the moment. Let's suggest that looking at calculating intervals. The thing is our Python code is already 400 lines of code, so I'm not sure if we should add extra parameters or not. I think it's suggesting three parameters. If we measure, if we are calculating the QRS detection, do we still need the R-peak sharpness to be calculated? Yeah, so the intervals are nice. Then we asked what happens if one of the peaks is missing. I was suggesting to calculate intervals with missing data and additional function. Obviously, every time you ask for something, it generates an additional function. So can I make sure I understand what you mean? If I have, do I still need, but by looking at the inputs of these functions, the answer would be yes, that I do still need them both. It just inflates my code. So after the suggested changes, how many parameters we will be looking at. So we're adding another function. Summing waveform is number array. Calculate the intervals between peaks. For example, intervals between R-peak and the ECG. No, we do not have intervals between R-peak because we're looking at individual PQRST. So that's no good. As the peak detection logic, do we have that function? Why do you stuff up the name? Yeah, we have peaks mentioned 174 times. Which is the variable, is it? Yeah, we just have them in processed data. Can we have it in a separate function? Yeah, it's a placeholder. This is not being co-piled to anything useful. I will just remind you all the tools we make are available on bionicaos.com. Go check them out and don't forget to leave your comments. Okay, a couple of things with the code that we currently have. We need to fix it and integrate it properly. Can you help with that? Ideally, just regenerate the whole code if it's too much regenerate functions. If that's too much, give instructions for GitHub co-pilot to do it. And we want to make sure that with noise level set to zero, the quasi-logical algorithm doesn't make any mistakes. I think for that, we need to sort out that frequency component or which... Sorry for that noise. So for which we have... I think instead of frequency, we now have the distance, the time between the different peaks, but it's not currently being fully integrated. We need to add the membership functions for it and the rules for the additional parameters. Can you do it yourself or provide prompts for GitHub co-pilot, whichever way you prefer? By the way, if you feel like being funny or something, go for it. But probably not too funny. It's something really wrong with the text window, with the window we insert the prompt. Yep. I would expect it to scroll automatically to the bottom or something, but it doesn't do that. Generating might take a while. We have been working on this tool for quite some time. Yeah, that's the one that's always being missed. So that's the one we always get the miss for currently. Yeah, we're not plotting anything in Python. So we can comment the mudplotlib, piplot out. Everything else is correct. Yes, we have this flash application. We have the analyze waveform. We have data processing. Quite safe. A copy of this quickly. Git is not properly working.  So essentially what you see on the website is trying to deal with that mess. There's a lot of tools that are actually not being listed that are under development. You can go to them if you know the URL. Like the board currently, this one would not be working because we haven't added it as a flash application yet. So we don't do frequency anymore and processing the data we should calculate intervals as well after existing peak detection. Yeah, we have it for debugging at the moment, plotting stuff that I'm not... I think it will actually work at the moment. Or maybe it will have to bring the plot function back. Yeah, so this is how the current detection works. We just keep overlaying stuff. This is the type of waveforms that we have. Yeah, so this noise level is found at 35 positive peaks, which is obviously not correct. Okay, just keep popping up. Okay, so finding the A peaks and doing the absolute calculation, printing it out as well. We are not normalizing it because this is actually dividing by 600. We'll do the normalization. We are not returning processed data. Are we doing it for any other parameters? And no. Are you doing extra hours? Yeah, that's a plot. It's trying to plot still. That's okay now. Yes, we're running this application only locally at the moment. Yeah, we better not do this again because it's plotting stuff in the back end. It breaks the flow of the software. Yeah, it's good to bring the intervals. We need to add them in here. What does it get the frequency from? It should be intervals. Yeah, data is for whatever reason. Being generated at the DC level of 300. So we actually reduce it back to zero. That's instead of detrending. We could use a fancy detrending algorithm as well. Do we have frequency currently? Yeah, amplitude is not normal as any more can get rid of that. We don't have frequency. Abnormality is now called abnormality score and we also produce a decision. So it's a yes, no normal, abnormal decision. I wasn't sure about this auto generation of membership functions. I don't think we need this. Here's a process that we have the intervals. The intervals should be returned. Yes, intervals. That's right. Yeah, don't need that line. Let's do some music and recap. We have the calculate intervals function. Is it giving a different function every time? This is better. This is covering a case where there is no R peak, for example. Now we need to modify, get rid of that. Yeah, fuzzy logic analysis. That's the tricky business. We don't have our interval because it's not continuous waveform. Can we rewrite this function taking into account the newly added parameters to do with the time distance interval between the different peaks? Okay, getting confused here. Again, just to make sure we are clear, we are reviewing every individual PQRSD waveform at a time. So doing an RR interval would not make any sense. Instead, we would like to be able to detect if the R peak is too close to the P wave or the T wave within the one complex. Why the RT and PR intervals, antecedent variables do not go all the way to 1, but 0.21 instead? Also, if the rules, instead of adding four new rules, the way combining them all, considering the output for all of them, for the abnormality, is set to be likely. I shared both the current Python code that we have and the game logic JavaScript as well. Probably the JavaScript needs some adjustment, considering we added more parameters to the mix. Can you review the whole code and explain why potentially we're getting this error? Yeah, we're getting a bit confused now. Yeah, the more prompt we are doing, the less bang for the bug we are getting. There's something wrong now. We can actually get the GitHub co-pilot to do this instead. Might use the workspace function because we don't actually know where we need to make the change. Essentially, we added more parameters, more features to look at and process data. Sometimes it provides these links that are very helpful, so that we don't have to search for the code. So we're calculating the intervals here somewhere. Yes, and we were not doing this correctly. This is still not correct, but this is a bit... Yeah, get rid of the print statement. We're going to play it on the front page eventually, so we'll need to change the HTML code as well. I'll share it with you in a second. It should be just called NPData, and we don't need that. Can we just update? It's updating. It should be looking at the previous code for any changes we made. No, it's not doing that. Let's well get rid of it. Oops. Print interval. Sorry, PR interval. Yes, that's what we want then. RT interval. This is correct, and we're returning all that. So that's fine. We're going to calculate intervals. It's not a hypothetical function. It's an actual function. So we're adding them to the dictionary before passing on. Let's do the same thing again with a workspace function. Update the fuzzy logic analysis function to use these new intervals as inputs to the fuzzy system. Let's see if we can find the relevant code. We'll give GPT-4 a bit of a break just so we also don't hit the 40 messages per three hour limit. So it's still searching for stuff. We'll ask it eventually to give an overview for what we have so far. I think it's getting stuck. We get it to work twice. Request timeout. What? So we add the server error. Now this one is not doing anything in fuzzy logic analysis. That's where the function is. We can do that to select the whole code. No, we cannot. We're not doing the calculation in this function. We have this funny default rule that I thought we could read off. We're not actually, no, we are using it. So we have those six rules, a default rule and rule four, abnormal intervals. So essentially, if they're too short or too long, the abnormality of the signal is highly likely. Let's test this again. We're still getting error. Calculate intervals should take one position argument, but two were given. That's not good. I think this might be something for a copilot to sort out because we don't know where it is in the code, we just use the workspace function. So it tries to find it itself. I mean, yeah, GPT-4 is so much better with these things. Well, pretty much anything. Calculate intervals. We have the... It's not in Calculate intervals. I mean this one. No, this is just initializing an array. I want to do an interval type if it's okay. We're not using interval type. Okay, that should be fine. Now we actually need to use the interval type. Interval type is not being used. I think we need to rewrite the whole thing. Returning intervals. Yeah, now it's doing the interval between r and d wave and p and r wave instead of r, r, which we don't have because we're looking at each p, q, r, s, d complex individually. So this should be better. Still getting an error. I'd pop it into a GPT-4. If I select the whole code, it might tell me that the code is too long. No, it's dealt with 500 lines in analyzed waveform. Now it's not getting the links. It's pretty annoying. So I have the analyzed waveform. It actually did this for me. It's only taking one line of code. That's not good. I need to make sure that the p, r interval is in the JSON response. And it's not. Yeah, interval should be there. Actually, interval should be there. It responds data. This one's a bit odd. Yeah, now we're having another problem. Every time you fix something, you break something else, don't you? I'm not entirely sure how to prompt a co-pilot in terms of giving it the code. What code does it need? I'm comparing it into converting the something in the dictionary format. What's being written here? Yeah, I think it's a job for GPT. Okay, I'll be sharing the whole code that we currently have for the Python Flask application that does the fuzzy logic analysis. We also get this error of maybe you could quickly sort it for us. Hopefully, it's nothing to do with the JavaScript. No, it shouldn't be. So I'll give the entire code, and could you also provide a summary for the whole thing and the changes that we have made today? That should be, the previous code should be still commented out in the script. If you wish to be funny or something, make general comments, suggestions. They go for it. It might be thinking for a while. I'm not sure how much progress we made today. Currently, the application is not running properly. The robot is not generating any output. We're getting a silly error. Can we summarize the whole application so far? Give it a broader context. If you feel like being funny, go for it. But not too funny. Definitely do not use any emojis. No, every time I say this, it's actually picking up on the actual word and starts will start to actually generate emojis. Give a general context for the application how it might be useful or not. We already have the game published on our website, but this one will obviously provide extra features where you play against the machine. Imagine this, you've got a game, not just any game, but one where humans and machines face off in the grand arena of ECG Waveform analysis. Picture it as a, whose line is it anyway? But for ECG Waveforms, where the points do matter, players, likely healthcare professionals or enthusiasts, get to test their skills and identifying abnormalities in ECG Waveforms. It's like playing a medical version of, where's Waldo? But instead of Waldo, you're looking for sneaky little changes in those squiggly lines. This way your code steps in, flexing its fuzzy logic muscles. It's like having a robot judge in a talent show, but instead of judging singing, it's judging the peaks and valleys of heartbeats. The machine uses sophisticated algorithms to analyze these Waveforms, considering factors like peak sharpness, interval lengths, and amplitudes. Basically, it's looking for the heartbeats rhythm and style. Players are presented with a Waveform and have to decide whether it's normal or abnormal. Each decision is a gamble, like choosing door number one or two in a game show, but with more medical credibility at stake. Meanwhile, the machine, your Python backend, does its own analysis. It's like having Sherlock Holmes, but instead of solving crimes, he's deducing heart conditions. The machine's verdict is based on fuzzy logic, which is a fancy way of saying it makes decisions in a very human-like, somewhat uncertain way. It's not just about zeros and ones. It's about, maybe this peak is too sharp, or perhaps this interval is too long. Or application is not just a simple game. It's an educational tool, a training simulator, and a way to introduce people to the complexities of ECG interpretation. It's like combining a game of chess, a medical seminar, and a detective novel all into one. You've got peak detection, interval calculations, and amplitude analysis. It's like the machine is putting on its detective hat and magnifying glass to scrutinize every heartbeat. The integration of fuzzy logic adds a layer of sophistication, like adding a plot twist to a mystery novel. The JavaScript front-end makes it interactive and engaging, because what's the point of having a cool game if it's not fun to play? Dealing with errors and making sure the data passed around is in the right format. It's like making sure all the actors in a play know their lines and cues. Your application is a bridge between gaming and medical education, turning the daunting task of ECG analysis into an interactive learning experience. It's like if Grey's Anatomy met who wants to be a millionaire. Equal parts educational, thrilling, and potentially life-saving. So, while there might be some hiccups along the way, like our current era situation, remember, every great invention had its share of tuning. Once this is all ironed out, you're not just launching a game. You're launching a new way to learn, engage, and possibly save lives. No pressure, right? Winking Face


Live from San Francisco, it's theCUBE. Covering Google Cloud Next 19, brought to you by Google Cloud and its ecosystem partners. Hello everyone, welcome back everyone. Live coverage here in San Francisco for theCUBE. Google Cloud Next 2019 is the show around cloud, Google Cloud, I'm John Furrier, Stu Miniman and Dave Vellante. We've been here all week, three days of wall-to-wall coverage here. On the floor with all the exhibitors right in the main, all the action. We talked to all the thought leaders, Google executives, entrepreneurs, experts are in the cloud and around the ecosystem. Dave, Stu, wrapping up, this is the wrap-up segment to kind of put the show to rest and look to next year and possibly Google summits. There's one in New York and some other shows we're looking to also cover. But if you encapsulate the show, I want to get your guys' reaction to what the main themes have been. We're seeing obviously Anthos was the big news. That's the big deal. That's their platform. They want to bring all the connective tissue around data security and really on-prem hybrid cloud, multi-cloud, application modernization, clearly during my open source and enterprise developers, plus the ability to do hybrid and multi-cloud. Stu, your thoughts on the show? Yeah, so John, when I first saw Anthos, I was like, well, this is CSP they announced last year. We were excited about that. We talk about things like Azure Stack and AWS Outposts, but the more I learn about it, the more I understand it, it's more than just kind of GKE and a little bit of packaging here. Eric Burr or Dave and I just interviewed a Google Fellow and you expect the Google Fellow to really be able to articulate the history of Google and the distributed architecture they're doing is like we're going to enable Cloud Native. Of course, we always have that in the Google Cloud, but now we're going to make that easier for you to do that in your own environment. So when you're thinking about modernizing your applications, you know, I was a little bit tough on Google when I said, oh, I hear a lot about lift and shift. Well, most customers, I can't lift and shift and not change because then I'll pull it back. It's too expensive. But if I can modernize wherever it makes the most sense. I talked to some customers here that said, look, I need to kick the team and get it into the cloud and then I can modernize it and start pulling it apart. But for some end customers, I can't move that and they need to modernize it here. And that's Anthos is the key enabler and therefore it's a good message. It's extension of what they've done with Kubernetes. There's a lot of other pieces here, but I'm pretty impressed. Dave, I want to get your thoughts because one of the things I'm seeing and in sports, when a team plays a game and wins, they call it a statement game. I think Google Cloud Next 2019 is a statement by Google saying, we're into the enterprise. We are not going to waver. We got hired Thomas Curry in a mid-savory and they're going to keep all the great talent. No one's leaving. It's not like a new regime change came in, they're pivoting. They know there's no pivot here. They put a stake in the ground saying, we are going to invest in the cloud. Sundar Pakai, the CEO of Google said that on stage at day one, they're clearly putting all the window dressing around enterprise with all the great phrases that we love, digital transformation, data-centric architecture, multi-cloud hybrid modernization applications. They're invested Dave. They are in it to play. They recognize that they're not going to win right away because it's a long game. So Google clearly is playing the cards properly. They're saying, look it, we're going to bring a lot to the table in this long-time table, but we're in it to play, and we're going to play well, we're going to invest. Yeah, I think it took a while for me to get there, Stu, too, because I heard a lot of the what, right? We do global distributed infrastructure. We're doing the applications for digital transformation. We got industry-specific solutions. This is what we do. Okay, great. And I heard a lot of differentiators. This is our unique value proposition and so forth. What I would have liked to hear right up front was, okay, we know that 80% of your workloads are on-prem. Well, guess what? And we're investing in scale and all that stuff, but we're the best at Cloud Native. And we're going to take, and we have the tools and expertise. We're going to bring those to you on your premises and show you how to get there. And then when you're ready, come to the cloud. If you're never ready, that's fine, but we're going to earn the right for your future business. They said that right way, the things that we want to earn in your business, but I don't think they can yet say we're the best at Cloud Native. And then I think that's a good self-awareness, Stu, for Google. I think they could say it. Now, maybe it's debatable, but... I would debate that. I do not think that Google is the best Cloud Native cloud at this point. I don't think they have the breadth and depth that Amazon has, but I don't think that that's the hard core stake on the ground because Cloud Native is early, CNCF, they're investing heavily in, open source is a big bet that they're talking about. They got a lot more work to do, but Cloud Native is still early, because you said the workloads are still on-premise for most of the enterprises, so they got plenty of time. The point is, if they had overplayed that card, I would have been more cautious. Well, okay, fine, let's talk about that a little bit, because it's nuanced. Would you disagree that internally, Google's got the most sophisticated, the best Cloud in the world, internally, globally? Can they make that claim, right, start there? We get the best Cloud in the world, yeah. Well, I think Amazon's got a great Cloud too. They've got all that stuff on there. I mean, they've at least got some credibility there, so I would have come from that position of strength. Now, the other criticism I heard was, where are the numbers? Now, that doesn't bother me so much. How long are they going to take Amazon to show us the numbers? Nine years, I think, so Google will get there. It's clear it's growing. You look around here, there's what, 30, 35,000 people. I don't know, what was it last year, 20, 25,000? It's growing, it's growing nicely, and the quality of the people is good, so. Dave, here's what I'd say about Google Cloud. Stu, I'd love to get your reaction. Sudhir Hasbay said this, he's the director of product management, he's talking about Cloud Fusion. He said, just from a customer quote, Google's Cloud is like an awesome highway, but I can't get my car on the road. So, and that's the on-ramp aspect. I can't, what, get my- Get the car on the road.  Yeah, yeah, yeah, yeah, yeah. So, just note about, if you look at the technology, from Spanner, Kubernetes, which was founded inside Google, and they did that right, big query is amazing. They have fricking amazing tech because they had to do it for Google. So I think that is a key strategy, and unlike other clouds that have come in and then died away, didn't have a lot of tech chops. So cultural shift is one of the big themes, but on-ramping, getting people on board, and the bed on open source, I think there's a gestation period that gives Google some time. I don't think they got to have it overnight. There's some table stakes, but they're checking the boxes. They just got to grind it out.  You know, we love reading the papers and we're really impressed with the technology, but the term you heard over and over again this week is we're going to meet customers where they are. And I almost feel they dialed it down a little too much here because I didn't have anything that I'm like, wow, blown away. They had Urz up on stage and it's like, I'm used to seeing him flying out of a plane with a Google glass on his head. That was Sergey, by the way. That was Google IO like five years ago. Sure, Sergey, but you know, that's what you expect from a Google is some of those pieces. And there wasn't a gee wow amazing moment for me, but the messaging solid, they absolutely understand and are solving some real customer problems today and solid business. Well, and 100% of the cloud providers now have a coherent and explainable hybrid on-prem strategy. You know, frankly, it's about time. I mean, they were denying that for a long time and I think it's clear that's where the business is. Well, to me, the big criteria on the cloud game is do they have the global footprint? They do. Do they have the software at scale check? Do they have the connective tissue to bring these disparate data services together? Check, working on it, continue to improve. And are they on the API philosophy side of things? Meaning, one of the things that made Amazon really great was they from day one were API centric. Google always has been part of web services, so they have that API DNA. I think Apigee is going to be the secret little dark horse in all of this and it's going to be a tell sign because as APIs become programmable, you saw Cisco and VMware on stage, can they build an ecosystem? Can they work with multiple vendors? Because the fact is from our data, and we've been reporting on this on SiliconANGLE and Wikibon, is that big enterprises and governments, whether it's a DOD or a big bank, are going to have hundreds of cloud projects, hundreds of workloads that are going to require unique cloud selection criteria because you cannot separate real-time data from software. And that's just the fact. So the databases are moving all over the place. If I got a workload, I got to need data, I got to be agile with the data, but I then need a data plane to connect across other workloads. So the workload conversation, I don't think was front and center enough where workloads are the key criteria. And still, some of the message on where Google fits in that hybrid and multi-cloud world is a little bit muddy to me. So how do they get, you know, on those in your data center? Well, it's a deep partnership with VMware.  It's like, I don't see VMware and Amazon separating. That's a deep, deep, deep engineering partnership. We've heard directly from Andy Jassy's, talked about it on theCUBE, how important that relationship is.  They're going to partner with all the open source companies and you know, you're going to own your data. We're going to make sure the privacy is there. So as Dave said, the numbers and the business of how Google can start scaling and really growing the enterprise business beyond, you know, G Suite's now part of it and we saw some of the Android for enterprise and they have lots of pieces, but the cloud revenue gets a little bit muddy like a Microsoft. So, you know, from the cloud piece itself, I'm not sure where, you know, they start gaining on a Microsoft or an Amazon today. Well, I think that they could gain ground and take territory, as I said on day one. Jennifer Lin's demo of no code modification, migration of workloads. If that actually happens, that's going to be a critical piece of the pie. That's going to move the needle very quickly for at Google. But I want to get you guys take on surprises. What surprised you here at the show? What was something that you didn't expect happened that was a surprise on a good way? To me, the big surprise is that the word customer was used a lot more here than ever before. Customer is the key to success in the enterprise. Listening to customer and customer choice. That's a playbook from Amazon. You don't hear Andy Jassy or any other executive Amazon go a three words without saying the word customer. If you had a tag cloud and it'd be like customer is the biggest font, here we've heard customer choice. That's been a big one for me, surprises. I was going to say, when you were asking that question, to me it was customer related as well. You know, clearly when you, in an Amazon show it's just customer, just you get inundated with a Kool-Aid injection of customers. It's very impressive and you don't have that scale here. However, what it did see is a lot of Fortune 1000 companies senior people were here. Yeah, still kicking the tires, but learning. And I think that usually leads to something. So I think Google's developing a lot of pipeline at this show that I think next year is going to translate. And we had conversations John with companies that we can't mention on air, but they are seriously, substantively looking at moving workloads into Google's cloud. Number one, number two is, if you look around here, Deloitte, Accenture, Atos. You know, some of the big guys, I'd like to see more of those global SIs, and I think you will. And that's where you're going to really start to see customer traction. So Dave took the customer and I'll say partner. So we said in one of our analysis segments that the logo slide's good, but you know, compared to a Microsoft or Amazon, it needs to quadruple where it is today. But in the conversations that I had from startups through some of those big logos on here, partnering with Google is good for them and they are excited by it. And that's not necessarily the case for every one of the big cloud providers out there. All right, so a lot of multi-cloud talk. I've said multi-cloud's all the rage, but it's really more a symptom of sort of multi-vendor, people going best of breed with different departments. Big news last night on Jedi, John, I want to get your take. Google really wasn't, I don't think, ever in the running, but certainly, you know, Amazon was the lead, Oracle, IBM, Microsoft, share the news and your analysis of that news. Well, yesterday there was news that the Department of Defense, this Jedi contract, the Joint Defense Initiative that's going on, Joint Enterprise Defense Initiative, such as the military cloud, $10 billion contract, was under a lot of, it's the biggest story in tech and DC in generations. It's the confluence of procurement being outdated, cloud selection, one sole cloud for that workload, multi-cloud across the department, and a lot of lost business potentially for Oracle and IBM. So Amazon, Microsoft, Amazon Web Services, Microsoft, Oracle and IBM are all fighting for this business. The incumbents, IBM and Oracle, were potentially at risk of losing billions of dollars. So there's been a lot of dirty pool, so to speak, a lot of dirty politics, a lot of dirty smear campaigns going on from Oracle to Amazon to try to discredit them. So the DoD, Oracle sued the DoD, saying it's unfair process, conflict of interest. The DoD made a final selection, Amazon Web Services and Microsoft are the final selections, and basically kicking out Oracle and IBM out of the process. So Oracle and IBM are out. Oracle's lawsuit is still pending, that'll probably be dismissed, because Oracle tried three different times to claim conflict of interest. They tried to claim conflict of interest in, where it has three, I got my notes here. July 2018, November 2018, and April 2019, all three times, incumbents have been not proven, and Oracle and IBM are out. The analysis here is that this proves what we've been saying on theCUBE, and that is that you can have one cloud, sole cloud, for a workload. So the Department of Defense has hundreds of projects, but for the military project, the $10 billion one, Amazon or Microsoft, probably going to be Amazon, they're the front runner, can serve that cloud, and that's the best architecture. That means that Microsoft will probably win the $8 billion contract of the DOES contract for collaboration. Again, sole cloud, sole workload. This is the trend, Dave. My analysis is that Oracle and IBM, mainly Oracle, knew that they were going to lose, they tried to do whatever it takes to kill the deal, and now the DOD has fought forward and they're modernizing the application. And all these lawsuits about procurement rules from 1985, all this tripwires, all these little nuances, this is a great win for the Department of Defense, and I think it is a tell sign for large enterprises because you can be multiple, you can have multiple clouds, but you can have one cloud work on one workload. It could be a big monster workload, like a $10 billion workload, or it could be a small workload. All the tech vendors want to eat at the government trough. We know that, and so why is this relevant? It's relevant to me because I think you're absolutely right. For a particular set of workloads, mission critical workloads especially, a single cloud is going to be more cost effective, more secure, a higher availability, less complex, and that's really what the debate is here now. Is multi-cloud going to happen? Of course, for different workloads, it's going to be horses for courses, so multi-cloud is a huge opportunity. Everybody's going after it. Stu, Google threw its hat in the ring in a big way. We seem to have a couple of camps lining up, and Red Hat, interesting news in both camps. Kind of got the IBM Red Hat camp, kind of VMware with, now with Google, really interesting sort of chessboard matches going on. Yeah, absolutely, every customer we talk to here, there's no like, oh, you know, I might be moving most of my stuff or even all of my stuff to the public cloud, but it is workload dependent, and that's how I'm choosing it. Google has some key strengths. A guy took us a little while to get the data and AI and ML pieces that we know Google has some strength here. One of the questions I had coming into it, can they reclaim kind of that thought leadership in the space, I'd love to hear whether you guys think that was the case, but you know, messaging point on good speed, you know, TK has them talking to the enterprise in a way that won't scare them away as to, oh geez, I'm not smart enough to work with Google. Well, I mean, I think obviously Google has to get enterprise compatible, and they've been working really hard to do that, and they got to just grind it out. I said this on Tuesday, it's a grinding out game, they got to fight into the trenches, they got to get the checkboxes, and this is what Amazon did early on and helped them a lot. Google has been working hard. I think the security angle from a device, the Android phone, and the onboard security at the edge is huge, I think data and BigQuery and those kinds of onboarding tools is going to be a great accelerant. I think cloud code, cloud run, cloud build is a phenomenal construct. I think that's absolutely developer friendly. If they can continue to serve the developer for the enterprise and make it easy to build and stand up applications that hit that sweet spot of the trend, which is the modernization of enterprise apps, not developer apps, not like a startup. I mean startups are different, startups are cloud, born in the cloud. Enterprise apps got to deal with legacy and all these compliance and all this risk, but they can make that easy and make it DevOps like that's a great checkbox. Just a quick note on that, because there was a lot of enterprise talk there, there's a nice group inside of Google working with a lot of the startups, got to talk to a couple of the startups there, and Google's definitely a company they're looking to partner with. All right, guys, let's wrap this up. Google really leaning into the enterprise heavily, obviously they're not blinking, they're going to continue power forward. And again, I like the Mojo they have here, they got a new CEO, we interviewed George Curry and Thomas's brother, Thomas couldn't make it on theCUBE, he was super busy talking to customers, we'll get him on theCUBE soon. But you got a culture here at Google and the culture is innovation and the culture's DevOps, the culture's developer, the culture's APIs. Dave, that puts them in a good position. Your thoughts? I mean, I've been saying for a decade I feel like a broken record. I said it so much, I stopped saying it, but the marginal economics of the cloud service providers who have scale are driving towards zero, in other words, the more volume they do, the cost of adding an extra customer goes down to zero just like software. There's three companies in the United States who have that scale, Google, Amazon and Microsoft, you know, obviously some guys outside the U.S. And you just look at the capex numbers, 47 billion over the last three years by Google, 13 and a half billion year-to-date U.S. data centers alone, it would take IBM three and a half years to spend that much on capex. It would take Oracle six years. Okay, they just do not have the marginal economics to compete, they'll compete in other ways. But these three are in it to win it. There's a big market there, it's a trillion dollar market, there's enough room for each to carve out an opportunity and continue to grow for quite some time. Yeah, let's do it. And Google lining up their ecosystem of partners to help them get deep into the enterprise. Absolutely, there's good opportunity for Google to do a number of acquisitions they have, you know, a big bank could spend a lot of money not just on infrastructure, but all the partner engagements and definitely some acquisition to help them get there wouldn't be surprised if they, you know, made some nice acquisition to help them grow that enterprise. IBM. In a modern way. Well, you know, now this one that was mentioned too, it was the Curian twins could, you know, be back together, but sure. Awesome stuff, guys. I think my final take is I've always said Google's the dark horse in the cloud game. They don't have a lot of baggage. They've got a lot of work to do and they're working hard and they really bring in tech to the table. They're bringing that culture of innovation they're behind, but there's opportunities for them to move the ball down the field in a big way. I think they can take territory and gain share quickly if a lot of things fall into place. If those bets come home, this dark horse will be right up on number two really quickly. So great job. Want to thank Google, Google's team, Google comms team, Google's CMO and executive new Thomas Curian for letting us come to the cube, bring the cube here. Google's very co-creation oriented. We appreciate the location. Want to thank Google. Want to thank some of our sponsors. Without our sponsors, we wouldn't be here. Cohesity, Signal FX. We got NetApp. We got Sada. We got some great clients here supporting us. Actifio. Actifio, thanks to our sponsors. They signal to the community that they care and they support our programs. Our 10th year of cube coverage at events. Want to thank everyone for watching, listening, sharing, hit us up on Twitter at cube and also siliconangle.com. We now are adding on a new feature to our cube which is on siliconangle.com. Special reports where we flow as many stories as it takes to get the truth out there, get the stories right, of course. Use the cube and stream the data with you here on the cube. We're here at Google Next in San Francisco. I'm John Furrier with Stu Miniman and Dave Vellante. Thanks for watching.