So my name's Demo, and you're probably wondering who is this guy? I'll get to that in a second. But first off, who came just because he had ducks in the title? A few people. I mean, I get it. I get it. So there will be some talks about ducks, and there'll be some talks about AB testing. This is not your normal talk. The slides are pretty bare. It's a discussion. So I encourage people to interrupt me, ask questions, yell at me, death threats, whatever you want to do. So the last time I gave this talk was actually over in Europe, so I'm having to edit all my nice political jokes, now that reality hit. So, but yeah, but actually the campaigns are actually a great source of AB testing because they did some very cool optimization techniques for their fundraising efforts on both sides. And I'll be mentioning some of those just to talk about it conceptually, how they targeted the markets. But first, a little bit about me. So I'm an evangelist at BoldGrid. Everyone's like, well, what's an evangelist? I'm like, it's like the revival religious thing, except I talk about web technologies instead of talking about God. So that's my full-time job is every week I'm at a different word camp, hosting a event or something like that for a BoldGrid, talking about the web and all the cool stuff out there. If you haven't yet gotten your BoldGrid shirt, please stop by a booth out there, take it because I do not want to take them home. So please grab them, they're organic cotton, we spare no expense. So I'm also the treasurer of Open Source Matters Inc. Does anyone know what Open Source Matters Inc. is? Hopefully they don't get thrown at, but it's actually the legal and financial body of the JUMLA CMS project. And a lot of people are like, why are you? That makes no sense. I'm an open source fan. I love open source technologies. I give back to all open source communities. That's why I am an open source evangelist, as you see in my logo up there. I think that you begin to learn things from every event. That's why JUMLA invited Matt Mulingwijk, who we all know and love, out to the JUMLA conference. And it's good to know the other tools and see what's out there. And I just fost and free an open source software. It's good for everybody. Whatever tool you use is great and WordPress is obviously my day job. And I love WordPress, but I got my start 10 years ago in JUMLA. So I try to get back to that community too. Now, I'm also the sponsorship manager of a couple of conferences. And I am a former Disney cast member, father and two, a little fur baby. And of course, a husband to my wife who sees me about once a week, maybe two times a week if she's lucky. It works out really well. Tipped to stay married, just travel all the time. It makes coming home a lot happier. But yeah, we can talk about A.B. testing. So first of all, when I talk about A.B. testing, I talk about control tests. So it needs to be controlled. What I mean by that is a lot of people, they don't really know how to start this A.B. testing thing. And when I first of all, what is A.B. testing? A lot of people don't know. They know it's a way to help you make more money. I'm a realist and everything I do is based on data and we'll talk about that. And all of this is from very real examples of agencies I used to work at. I used to be the digital manager of a pretty large advertising agency in downtown Minneapolis with clients such as 3M. They used to have an entire digital department for a company that did hundreds and hundreds of banking websites. And generally these are all just tried and true examples and ways to think about kind of A.B. testing. Has anyone here have done any A.B. testing before yet? Okay. Keep those hands up. You're not done yet? Okay. So does someone want to volunteer, want to tell me what that test was and what you were testing and what the results were? Sure. For a health supplement retailer on the site, he wasn't getting enough checkouts that he wanted. So my proposed test was he wasn't building up enough trust for the check out. He had a ton of Facebook fans. So when we put a Facebook like box that showed the greater fans by the final check out button and we tested it and increased sales by 70%. Okay. So I'm just, when you guys talk, I'm going to repeat it back not because I'm deaf, but so that the camera can actually hear it.  The conversion wasn't as hot as you may be able to like it. You had a lot of social proof. In this case, Facebook likes and you decided to put a Facebook like button on the checkout process. Okay. So yeah. So you would see, okay. So if I was checking out for the first time, I would see all the social proof that all these people were liking it and then had improved the conversions by 17%. Excellent. One more example that someone would want to share. Sure. A lot of it is social media that we do or a marketing agency. And the first thing is sizes. You know, one size is not enough at all. So we have to work with that. But it's always the message, the reach, you know, based on what platform.  Okay. So again, they do a lot of the market agency, digital marketing. And they do little pilot tests. Have I heard you correctly? See which one is most successful for that specific campaign? Because not one size does not fit all for every client at a campaign. And then decide where to kind of put that dollar on there. That's a good, that's a good point. Because I like to say that A.B. testing in websites are a magic money machine. And our job as developers is to make the magic money machines work as well for our clients. And I get in trouble a little bit with these talks. Because a lot of developers are like, some of the stuff you said and some of those examples that you gave, which you'll see in a minute, it was just, they were just ugly. And I'm like, yeah, but the client made 300% more money. Do you think the client gave a shit? No. It's our job to do our expertise and knowledge and the best things we can to make our client's project successful. Every client has different goals. Maybe their goal is more conversions or more revenue, like we heard from the supplement site. Maybe it's just more likes or visits or more stickiness on it. But whatever that is, we should make sure that we try to do the best for that. And with that, I believe A.B. testing is one of the best things you can do.  Okay. I have a website. I made a bunch of changes and then the website made more money. And I don't know why. Yeah. So that's usually what happens.  So first of all, A.B. testing needs to be controlled. Make sure, you know, you need to test one element at a time. You can do a multivariant testing and we'll talk about that in a little bit. But unless you have a lot of traffic, it's going to be really hard to do multivariant testing. So I always like to do one change, one thing at a time, one tagline, one color, one font, whatever the case is, and test it side by side. I use a lot of different tools. There are some great WordPress plugins. I like BWO as a paid subscription for A.B. testing. There's other ones out there like Optimize. Lee or whatever. And you want to use a tool, whatever you do, use a tool so you can actually see if it's statistically relevant. And that's the other thing that I learned the hard way is that people, they get excited by movement, but they don't really know what any of that movement means. I had a client that was getting thousands and thousands of visits a day. And they're like, oh my God, we did this one thing and I made $5,000 in one day. And I'm like, OK, but if you look at compared to A.B. testing as we tested multiple things, it actually wasn't statistically relevant. It was within the margin of error. So you need to make sure that you know what you're doing because I can't tell who's had something like where your client asks you to do something, like update a website or and then the site broke and then they blamed you for it. Like, you know, like, well, hey, you did it. You're the last person to touch my site. Well, correlation is not causation. And I think that we as people just try to be like, well, this thing happened and then this thing happened. And obviously they're linked because I want them to be linked. So I'm just going to say that they're linked. But you need to make sure they're statistically relevant. And I did this for an insurance company, you know, we did a bunch of different testing and they were it was a insurance dental insurance company. And we did all these tests. We were testing tag lines and pictures and testimonials and things. And we were able to move in their number, their needle by 80 percent. More quote submissions. By just changing the border radius of the fricking button.  I had psychologically done testimonials and I had all of this beautiful graphics and all of this stuff and we ended up just changing the border radius at the after doing 80 separate tests and the clients happy because they make millions. They're making millions more money, millions more dollars on just a little change. And we like to just over complicate things as designers and developers. I find we love to do all of this cool stuff.  I try not to, but it happens. I'll also give you a checklist that we'll talk about. So statistically relevant, micro testing, that's kind of why I said, test little things at a time. Don't test multivariant testing. Just test a couple of different things one at a time. Test a color, test a tagline, test a picture, things like that. And there's some really good proof to all of this. That's kind of going out there. So can someone give me some ideas on what things you can test? Just shout them out. Border radius. What could what could would that do?  Border radius, other things you could test the color of the call to action. Cool. OK, button labels, text on the button, the placement, OK, of the of the button. We went from a two-page checkout to a one-page checkout. OK, simplifying the checkout process, reducing number of steps, things like that. OK, what font weight, OK, cool. Yeah, and we're going to talk about all that. Excuse me. First of all, we're going to talk about buttons. So, yeah, all these things you can do about buttons, you can test the button position, the button color, the button font, if you have an icon on the button. Something that's worked really well for me in the past is having multiple lines of text in the button, like a main button called action with like a subtest, like sign up, you know, sign up for three and then a little subtext, you know, we won't sell your information or whatever the case may be, depending on what you are doing the thing of. So, but in sizing colors, if you have icons, the call to action, all these different things, and you can very easily do this with some of these tools, depending on how much time we have, I may actually jump into VWO and do some live testing on someone's site. That's a cool thing about VWO is you can type in any domain and test. It just doesn't go live unless the JavaScript's on there. So I've like done this on cnn.com and whitehouse.com and things like that. It's kind of fun, but if we don't have time, I can definitely sit down with you over in the happiness bar, which they would like to let you know, speakers after their talks will be in the happiness bar if you'd like to talk to us.  Text is a very big thing that a lot of people overlook. Images, and we'll talk about that in more detail when we get to the checklist. Images are funny. Stock images, who uses stock images? Raise your hand. Stock images. Who refuses to use stock images? One person. So it's kind of funny.  You're like, I never use stock images because, well, they're stock images, or I only use stock images because the client has no money and whatever the case is. So images are kind of funny because I've tested stock versus not stock.  All these different things and images are really those things that actually kind of work. And it will kind of beat you up inside, like when we talk about some of the examples, some of these things as designers, and it beats me up inside, but I test it. I do have to stress that when you have a client, please make sure you approve all tests with them before you start doing them, otherwise they might get mad at you. And please make sure that you explain why. You say, we're just going to test a small sample. We're going to test, I personally test, 5% of all traffic to my client's site. If they have enough traffic to support it, that's the problem, right? Is that if your client doesn't have a high traffic site and you only test 5% of the traffic, it might take four months to get enough statistically relevant traffic to actually make a decision. So you need to kind of decide, I don't like going over 25% out of any circumstances because I don't like everybody seeing my A-B testing and I don't want to have the A-B test being permanent until I get the statistically relevant data. So images, definitely look at the images and everything basically has to all point to this. It's usually at this time they talk about ducks. So HubSpot did a study. Believers HubSpot have to relook up the way paper, but they do so many, it's hard to keep track. They did a study. They had two landing pages. One had a picture of a duck facing to the left. One had a picture of a duck facing to the right. And the right facing duck converted three times more than the left facing duck. Can anyone guess why the duck facing to the right converted more? Okay, we read left to right. Okay, if you're looking to the right, look to the future. Okay, so if it's to the left of the page just like the duck is saying the text, okay. Did the right facing duck face the CTA? Did it face a paragraph copy, not the CTA, but... It should face the CTA. Okay, I've heard, oh, yes. Something to do with right handed more than left handed people. Okay, right handed versus more than left handed people. I'll get answers. I've asked this question all over the world and I get all these different answers. And you know what I tell them?  I don't care. I don't, and neither should you. You need to stop trying to think of why things happen because it'll drive you crazy in A-B testing. Have the data drive your decisions. If your data is statistically relevant and you have enough of a test to make it statistically relevant and it's working, by all means, just keep pressing the button for the magic money machine and keep taking your client's money and being everyone's happy. If you're trying to think of all these psychological reasons, you're going to drive yourself crazy. And yes, I know there's lots of people out there with psychology degrees that do very expensive testing and consulting and eye pupil things and all this expensive stuff. At the end of the day, just do whatever moves the needle. If it's working for your goal at that site, because every site has a goal. It might be revenue, it might be staying on there, it might be liking the Facebook button. It's something. Every project has a goal and it's our job to try to help that client realize that goal. And if you do a good job and you can prove that, you know, you're making the magic money machine spit out more money on the other side, you know, you know how many clients have said, okay, Mr. Client, you know, I'm really sorry we only made, we only increased your profit by 35% this month instead of the 14 it was last month. Would you like to go again and try to get 35% again like we have the last six months? So do you want to cancel? Nobody's ever told me they want to cancel. And that's the point is we want to service our clients, but we also need to make a living. So my checklist, my actual checklist, I took this from a book that took it from another website. It's sourced up here. And I'll show you how to get this book at the end. And I'll show you how to get it without spending money. And this is my actual checklist and it's funny. Like it says, don't laugh till you try it. So professional versus unprofessional. And yes, I've done this. I've used Comic Sans and AB tests. But guess what? Sometimes that sells. And this is where I talk about one, the dangerous possible campaign thing. So one of the candidates who ended up being successful, he did his fundraiser a week before the election. He said, hey, donate and your name will be on my Facebook page. Okay, it's kind of a cool social way to do it. Donate, your name will be on the page. Can anyone guess? And if you know it, don't say it. Can anyone guess how he did that? Like what are the technical methods that if you said, hey, you know, I would like to have a thing. So when people donate their name flashes on my page, can anyone think of any technical ways that you could achieve that? Or the website if you don't want to do Facebook page. JavaScript, live video editing software where it comes in with a nice fade. There's lots of polished ways. You can make that look nice and polished and professional and dare say presidential. But you know what the Trump campaign did? I couldn't believe it. It was on my screen all day. I couldn't believe this is what major political candidate with supposedly all the money in the world was doing. And it raised more money than any other fundraising effort he did. He had a live webcam of a laser printer printing out the names. For 14 hours, 20 names on a sheet. And not only that, they had a very high tech method to get you to donate. Every once in every about 20 minutes, a hand would pull down with a sign. A hand written sign. That said, go to this website. Donate blah, blah, blah. Hashtag drainage, drainage sample, whatever it is. And it worked. And I was watching the numbers. He was having hundreds of thousands of views all day on this so low tech example. But guess what? He probably would not have made anywhere close to that if it was a high production polished value because he knew his target market. Yeah. Now, it's funny to think about. But if you think about it, think about the target market of the Trump campaign. Think about who we connected with. A lot of it was middle America. Lou Coller. And it actually resonated with them. He tried this once before, not nearly as successful. It was before the RNC. You could donate and you would have your name on the big jumbotron during the RNC. And that was produced in value with all this production and lens flare. And all this after effect stuff because they paid $800 for the software. So we got to use every feature. And it raised not nearly as much based on the numbers. They don't release the exact campaign success, but based on what I saw when witnessing it. So yeah, it's very interesting. So photo versus illustration, pretty easy. Test of photo versus illustration. There's lots of great stock illustrations out there. One of my favorite sites is actually Fiverr for just AB testing illustrations. If you're an artist or know an artist that can do it custom, great. Please do, but not everyone does. So Fiverr, you can get a custom illustration of five bucks. It's not perfect, but it's enough for a test. One color versus another color. Pretty easy test. Low brightness versus contrast. High brightness versus contrast. Border versus no border where it blends in with the site background. Clear image versus blurry image. The blurry image stuff is actually pretty easy to do and worked quite successfully. Static versus animation.   And it works sometimes. Sometimes it does. And it's just, it's funny. Like we all wonder in today's day and age, because anyone landed on one of those DR or direct response long form pages that just go on and on forever. And then like have the testimonials and the screenshots of the actual PayPal account with all the money that I actually made with this tool. There's a reason why those exist. It's because they work for a certain person.  They work for a reason. So static versus interactive. The Clinton campaign was really good about this one actually. Their donation page. The home page had like these buddings you could choose to donate on the home page like $5, $10, $25, whatever. It wasn't actually a button. It was just a picture of a button. And no matter what you click, it like defaulted to like $100 or something. But it's just the idea. It's the idea of choice. And I guarantee you, they do AP testing. One of the best things you can try if you want to look at some really good A.B. testing live is go to puckovin.com. It's Wolfgang Pucks, like pressure oven thing. And do it for like different IPs. You'll see so many different layouts. And it's just interesting how they, and the price will change like a $100 swing. So it's pretty interesting how they do that. Professional stock photo versus amateur informal photo. We all can take amateur informal photos pretty easily. We all have these really expensive cameras in our pockets. So definitely hang out with that. One layout versus another layout. Another again, pretty easy to do. One called action versus another called action. Try different things. The dental insurance example, hiding things like, they used to say a 30 second quote. And we would say, we tried no obligation quote. And we tried, get an instant quote. We tried all of these different things. Get a quote, have, starting at $9.99 or whatever the rate plan was. And of course the border radius is what did it. Not better. See what happens when you fixate on why stuff happens? You never get over it. Images versus text only. You might think text only is crazy. Why would I want to do text only? But again, it depends who your target market is and doesn't hurt to test. And if it works, it works. Okay. Butting versus blue underline text. I've done this quite a bit. Works pretty well, depends on it. Regular border versus a false border in drop shadow because, why not? A custom ad design with familiar UI elements. I like doing mock Facebook layouts a lot, quite a bit simply because it's familiar and people like to know how to interact with it. Now it's at this time in the talk everyone says, okay demo, that's your real name. Which may not seem democular, sort of a demo. There's too many mics in this world. I'm trying to help everybody so just don't think about it too much. But yeah, they're like, this might be fine for some back county website. But I have a big client like CNN or something. And remember, I've done work for national brands. I've done work for 3M. And all this works, you just have to make sure you follow their brand book and their brand guidelines. All these concepts can be done with any brand under the sun. You just need to massage it towards that direction and of course get approval ahead of time.  But you can have it look nice, but still be unprofessional-ish in terms of how this goes. So original image versus the mirror image. It's our old duck thing going over again. I accidentally answered that. One font size style case versus another font size style or case. Standard ad shape versus a custom ad shape with CSS. It's very easy to do 100% border radius. Not that kind of fun to do. Although, someone looked at the web recently, I think a square is the custom ad shape now and the circle is like the standard shape. I see it everywhere. Positive versus negative. This is always fun.  I had a client that it was a device kind of like life alert.  Do you notice they sell those things in two different methods? They sell them because the client isn't the senior citizen living independent. It's the kids. That's the client. That's who they're targeting with these ads. So one example is they'll say, do you want the peace of mind that your mom and dad can live independent and not have to go to an assisted facility and all this wonderful stuff, all these nice wonderful feelings. That's one way to do it.  It's going to be your fault. I mean, you've seen these ads. I'm not making this stuff up. You see it out there. There's a reason why this stuff works. I did this with a device that did, that stopped stoves. It was a device that if you walked away from your stove, it shut off because stoves are the number one cause of house fires. One example, it's like, you know, would be like, you know, for like a lot of college kids, we were selling it to other parents for the college kids or whatever.  But we also tried things like, you know, we reported, you know, we had like statistics this week. There's been this many stove fires that burned down houses because of unintended stove fires, blah, blah, you know, get this product, whatever the case is. So definitely try it again. Approve anything by your client first. Generic versus relevant to time period. You probably see a lot of these ads, especially like for like auto loans or whatever, like, want a girlfriend? You can meet single women here or home alone on a Friday night when it happens to be Friday. How did they know? Or, you know, like single women in Wichita looking for you. I live in Wichita. How? Oh my goodness. It's my lucky day. There are single women in Wichita. Or want a girlfriend for Christmas, assuming it's around the holiday time. Mention prices and discounts. Regular $20 to 90% off. Or, you know, leave it a mystery. Good for your feet. Add to cart to see final price. There's two reasons why company retailers do that. One, sometimes they're obligated to based on the MSRP requirements that they can't tell you the price until you're in the checkout process. But a lot of times it's just they want to get you to add it to your cart. So, because there's all sorts of data around that. It's amazing. No trust logos versus trust logos. Ascena, ABC, NBC, CNN.  You need to be careful with that. Has anyone here heard of my pillow? The pillow? Okay. Has anyone heard? I live in the Trans City. So, he's like a local rock star in there. Like, so silly. I have the pillow personally, full disclosure. I like it, but the guy's silly. Anyway, they would always say, better business bureau A plus rated for years and years. Anyone hear what happened with the better business bureau with him? You heard about it? Yeah. So, the better business bureau took him from an A plus to an F overnight. And now he's like being like all surprised and blah, blah, blah. There's lots of reasons why and has nothing to do with the product. It had to do with the sales practices or whatever, but and not honoring the code of ethics of that organization, which you sign when you pay the membership fee, whatever. Anyway, but now it's hurting him because he's been using this as his sales crush this whole time. And now he can't. So, now he has to find another sales method. Now, I personally love the pillows. I think they're great. I think the sales are really frustrating because I never know when they're on sale or not, which was the main complaint to the better business bureau. So, whatever the case is. That being said, you can also use your weapons of influence. These are from influence by Robert Kennedy. And I, of course, have just pronounced that. You look at scarcity, you know, offer ends at midnight. Why? So, wall supplies, last authority. More doctors recommend this five out of six dentists agree. Consistency, current visitors who take small steps, asking for a major commitment, you know, get all those little guesses, whatever the case is. A lot of, like, digital content providers will do this to get it down with a free e-book and then, like, do little steps, like, you know, along the way to try to get to the big yes of the big sale, liking using photos of real people versus models, similar to what we heard from our supplement example earlier.  She uses our product to keep her baby happy. Of course, that product is obviously an iPad. Social proof. I don't have kids, by the way, if you can't tell. So, I'm never home. I mean, we may have kids, but they might not be nine, but yeah. Social proof, one million satisfied customers since 2010. Testimonials, I lost 56 pounds in this program. I didn't, obviously, but that example did. We're giving you a free trial, or my favorite thing, a no-risk trial, but they charge you, but they still refund you later. I like that term, no-risk trial. But again, you can test all this. It's all about avi-testing. The point of this is to give examples. I want to make sure we have time for questions. Give examples of ways you can use this in your business. And we'll ask questions at the end. Appeal to the eight universal desires. Survival in German to life. Life extension, you know, if you can do that. Enjoyment of food and beverages. Freedom from fear, pain and danger. Sexual companionship is also something you can use for selling. Comfortable living conditions to be superior. Warming, keeping up with the Joneses. Caring protections of love, ones or social approval. Fear and pain. If he died tomorrow, who would take care of your family? Get a free life insurance quote today. Because nothing takes care of your family more than cold hard cash. Benefits, not features, three times warmer than other brands. Maybe with Maverick Gustav, ask a question. Be controversial. Use short words. Because short words are easier to read. Like Trump or Clinton. Vote for the census. Five senses. Sight, hearing, touch, smell, taste. Example of using touch. Talk about a tell a story. People buy, I used to sell TimeShare years ago by the way, which might be why I'm jaded. People buy in a motion to justify with logic. Seriously, and it works on the web the same way. And again, it's our job to try to paint that picture to help our client move the needle and make the magic money machine work. Because if you can tell a client, and I do this with clients, if I have a machine, and I put $3 in a machine and $5 came out the other end, how many dollars would you give me to put in that machine? Well, the answer is on the limited dollars. And that's great for us, because those mean billable hours. So, plain have fun. That's the thing I want to make sure that you remember is that A.B. testing is some of the most fun you can have for the clients money that you'll ever do. And you definitely want to make sure you take advantage of that, while also ensuring your loans are being later. So, I'm going to show you how to get a cool book about A.B. testing. The slides are on slide sharing linked on the website. Now, the reason I have these steps outlined is because although I like this company's products, if you don't follow these steps exactly, you'll spend anywhere between like $30,000 to like $30,000. So, it's an exaggeration, but he does have some expensive products. So, you can go to .comsecretslabs.com slash free dash book. I need to zoom in, that is awful. Okay, you'll see this, and you'll see this form. Fill out the form. Easy. You will see an offer for something. If you want to, by all means, buy it, but just decline the little checkbox, which will be automatically checked. Then you'll see a video with an upgrade offer, which you say no thanks, I'm good at the bottom of this. Now, I want to say I've paid for a lot of his products in some of his clubs. For a while, I was a member of his monthly newsletter subscription for like $75 a month. I don't know the exact price.  But you can use his philosophies. He's one of the largest experts out there kind of teaching this stuff to just everyday people for any client out there. He was actually featured on The Profit, the CNBC show. He helped make a site for the Neth Watch episode. So anyway, I like his stuff, but not everyone likes checkout processes like this, which is why sometimes companies adapt like GoDaddy. You say no thanks at the bottom. There'll be another offer. You say no thanks. You'll see this page. Close it. If you click anything, you would have bought something. You get an email and then unsubscribe from the email only if you don't want beyond this newsletter, which will offer you more services. Again, I like his stuff. It's just not everyone wants that sort of information. So I've got about 10 minutes for questions. So I'm not going to be able to do a live demo, but if you have questions, I want me to look at your site with ideas. Please see me in the happiness bar. I'm happy to sit down and talk with you. I'll talk with you longer if you tell me you have picked up a bold good shirt ahead of time, simply because I don't want to take them home. I'm kidding. I'll talk to you just as long if you even if you don't get a bold good shirt. But so questions, who has questions, comments, the threats, anything? Yeah? What's the reasonable sample size or at least the minimal sample size I try to get 5000 users through my tests personally. Everyone has different numbers, but what I like about paid tools like VWO is that I'll tell you when the ratio is enough to be statistically relevant and actually give you a recommendation like kill the test, extend the test or implement the variable or whatever the case. I try to hit 5000. That's my personal rule, but I'm usually dealing with sites that have a lot of traffic. Then let's let a lot of brands and sites out there. So other questions? Really? No. I have another one real quick. Sure. You said the other thing works better. Yeah. Anything else that seems to be Yeah. It kind of depends on like how like the social consensus is going. You also kind of kind of look at like who the target market is and are they educated or are they not? What the persona is is a great tool. Makemypersona.com which helps you walk through how to make a persona. It's a free tool powered by that helps by offers. And you kind of look at it because if they're educated or uneducated, have kids not have kids whatever the case may be. It might definitely can definitely change what you might want to do. Other things that I've seen that work well are a lot of like micro steps like get a free ebook to try to opt into something or whatever the case. I'm always a fan of trying to make it statistically relevant. There's a great tool out there that they can call JL Mailer. It is an email newsletter system that will personalize the website based on the person that you send the email to. So you can send the email to Joe and Joe clicks the link. It's going to, it'll say Joe's name in the text content. I've had that work really well. I've also had a mad lib form. So instead of a form it says hi my name is form field and I'm interested in drop down. So it's kind of you know more conversational. Other questions or things that you've experienced? For the email or? It's a service. It's called VWL. Victor, visual website optimizer. They have a free tier I think or a low cost tier and it works with any CMS out there. There's some things on WordPress out there on the repo. I don't use those as much because I don't like changing my database permanently. And unless it's a permanent change. So I don't want to have like two pages and have people go to actually two separate pager post types to try to test and try to like deal with different URLs and things like that. I want every element to be the same except that single change. Which is why a JavaScript solution like optimized via VWO kind of works for that situation. I played with it. What I like about VWO is you don't need to know any code to do any of the changes. You can literally click a button and drag it across the screen and it will do the change for you even if you don't know how to do the code change. Now if it works, you'll have to figure out how to do it on the final product. But it's a very easy drag and drop editor. Other comments, questions. We've got a couple more minutes. Three. No? Yeah. What's your greatest discovery that I've often like? I guess my greatest discovery was to stop trying to figure stuff out and stop caring. But seriously, I've learned to embrace data and do database decisions. I learned that the small stuff makes the biggest impact. I've done the big huge site we do and not knowing why stuff happens. I mean, Victor, over there, we used to serve on the G1 project together and they did a huge update to their equivalent of the plugin directory and traffic went down, like 80% less or something, way less. And it was a huge redo and maybe that wasn't the best thing for those users. Maybe it should have been small tweaks. But, and I also learned to test along the way and that has to do with kind of accessibility and things like that too, that accessibility is like becoming kind of a passion of mine because one in six adults has a disability that is covered under WCAG 2.0. WCAG 2.0 is a new ADA accessibility requirement that's going to be a Department of Justice mandated 2018, they've announced. There's already been lawsuits going out and have been lost and settled by companies like Target and like 12 of my clients have been sued after this new requirement. The Department of Justice is saying you have to follow WCAG 2.0 AA or AAA. It's a very complicated accessibility law. Please make sure you're looking to it because it's going to become standard. If you would like to make a lot of money become certified and have a test of accessibility because it's very hard and very expensive to do so. But I like to test things that are related to accessibility like color contrast ratios and things to make so that people with certain types of color blindness can still read the text and stuff like that. So, yeah. Could you think that the psychology side that makes sense when you have low traffic and low budget and you have to pick small number of tests and you can't do all the variables even if you do them individually? Sure. You still have to... But you don't have enough traffic, yeah. Yeah, definitely. I mean, if you need to look at what who's coming to your site and make the best decision for that site. So, yeah, definitely prioritize what you think might work the best. For example, in my insurance company example we wanted to test having... We thought, okay, maybe if people knew what the average price was that might help things like that. So, yeah, definitely look at... You know, try to look at what might make sense to your users but if you can, test as much as possible. But if you have a low traffic site even doing one at a time it might take years to go through that list. So, and what I like about VWO2 is that there's all these cool ideas. All these like AB testing ideas that they give you and show you like how long they are to implement things like that. And just because a test you read a case study and I worked good for one person it doesn't mean it's going to work well for you. Every user base is different. Anyone else? One once? Going twice? Cool. Well, that's exactly time. So, thanks again. I really appreciate coming here. My treat I handed was MP Mike. Thanks.


Thank you everybody for joining us for another all things data on the OpenShift Commons briefing channel. This is a new briefing series and we're excited for everybody to join us. And if you have any ideas or things that you would like to see in the future, please let me know. Today we have Jakob Schultz from Red Hat as well as Karan Singh from Red Hat and thank you both for joining us. They're here to talk about AMQ using OpenShift container storage. So Jakob, please take it away. Okay, let me share my screen. So in this presentation it will be me presenting together with Karan. So I am principal software engineer in the Red Hat middleware engineering team and Karan is senior architect in the Red Hat storage unit. And today we will give you some introduction to AMQ streams and to the storage requirements which AMQ streams has. And then we will talk a bit more about the importance of OpenShift container storage for AMQ streams. How does it fit together and what are the advantages which you can get from using these two together. So let me first do a quick introduction into what AMQ streams is. And it's Red Hat's distribution of Apache Kafka. It has two versions. One of them is for Red Hat Enterprise Linux. So that's suitable if you want to run Apache Kafka on VMs or bare metal directly without any containers and OpenShift or Kubernetes orchestration. And the second version is for OCP or OpenShift container platform, which obviously for this talk is the relevant one. And on OCP we are using the power of operators from the StreamC project. StreamC is a CNCF project which makes sure that Kafka runs natively on Kubernetes and OpenShift environments. And with the operators, AMQ streams can run basically anywhere where OCP is running. So it can be on VMs, on bare metal, on private clouds or public clouds, wherever OpenShift is supported. You can run AMQ streams as well. And with AMQ streams on OCP we support OCP 3.11 and higher. So if you haven't jumped on the OpenShift 4 train yet, we have you covered as well. But OpenShift 4 is of course great with a lot of improvements. So ideally you should run it there. And for those who don't know what Apache Kafka is, let's give a quick introduction of how it is, what makes it special and how it works. And there are actually many different definitions of Apache Kafka. One of them is streaming data platform. What does that mean? What that means is that Apache Kafka basically provides capabilities for storing, delivering and processing data, which makes it kind of ideal platform for doing streaming data, even driven architecture, stream processing and so on. But it is also a published subscribe messaging system. So it can be used as an alternative to let's say the traditional JMS style messaging system and can be used for the traditional integration use cases. And then the third definition I have here is a distributed horizontally scalable fault or a land commit lock. I love this one because it's full of very nice buzzwords and it sounds very sophisticated. But don't get misled by that. It's not really just a random collection of some fancy words. It's actually pretty accurate definition of how Kafka works because if you look at the Kafka broker in detail, how it is implemented, it really is distributed horizontally scalable fault or land commit lock. So it's pretty accurate. Kafka was originally created at LinkedIn, but later it was open sourced and now it's as the name suggests part of the Apache software foundation. And it was from the beginning designed to be fast scalable, durable and available. And it achieves that by being distributed by nature. So it does basically everything it does is based around data partitioning or sharding if you want. So all the data are split into some smaller shards, which are handled individually. And that's how basically Kafka is able to achieve the high throughput with fairly low latency for delivering the data. And that's what gives it ability to handle a huge number of consumers. What I think is always important to mention is that while in this talk we will mostly focus on the Kafka broker, which is where the storage comes into play. Kafka itself is ecosystem rather than a single project or a single component. So apart from the broker, it has its own stream processing library in Java. It has its own clients for different languages. It has its own integration framework called Kafka Connect. And even beyond Apache Kafka project itself, there are integration with many different third party tools. All the big data tooling stream processing frameworks such as Spark, Storm and so on. All of them have great support for Kafka. So I always like to point out that it's not just the messaging broker itself, but it's really ecosystem of many different components. So I already said that Kafka was designed to be distributed and it does so by partitioning the data. So when sending in or receiving messages, we always send to or receive from a topic. But every topic is always split into one or more partitions. So it can be that topic is just a single partition, but in most cases it's many partitions for each topic. And these partitions act as a shard. So most of the actual work is always done on the partition level. And the topic is really more a virtual object where you send or receive the messages from, but which gets kind of internally pointed to the right partition. And each message is always written only to single partition for a given topic.  So the broker is really just the commit lock as I mentioned before. And it's the clients which have a lot of the logic of distributing the messages into the partitions. So when you try to send some message, then the client decides into which partition it should be sent based on the message key or based on some round robbing distribution if there is no message key. And then the consumers will similarly decide which consumer should consume from which partition and so on. So that's what allows it to scale so nicely because there's no single point through which all the messages need to flow in the brokers or anything like that. So where the partitions were doing the scaling and the sharding of the data. Replication is a concept which provides availability and durability guarantees. And basically every partition can consist out of one or more replicas. And the replicas have always two roles. One of the replicas will be a leader. That's kind of the main replica which is used by the producers to send the messages to. And then the other replicas will be called followers and they will act as a backup to kind of provide the redundancy for the data. And the followers will really just connect to the leader and will copy all the messages which the leader receives. And they will try to stay as much in sync as possible so that when the leader is lost that they can take over the role and serve the messages without any data loss. And of course that means that the roles can be changed during the Kafka cluster lifecycle. So in this picture you can see that we have three brokers with single topic which has three partitions with three replicas each. And you can see the dark blue partitions or replicas they are the leaders. And you can see how they are feeding the data to the followers so that the followers can keep the copy of the data. And now of course what can happen for example the broker see goes down because of some hardware failure or whatever. And basically immediately the one of the follower replicas on the broker B was able to take over becomes the new leader. And the clients can just reconnect there and keep working without any significant disruptions. Now the replication is really important for Kafka and cannot be fully replaced by mirroring on the storage level. Because having multiple active replicas available with the same data means that the availability is really great and they can very quickly take over if the leader fails. But it's also important for the reliability that the data are written to multiple different disks on a multiple different brokers when they get put into the disk buffers and then later written to the disks. So the replication is not something what you can fully replace with storage mirroring or to the storage mirroring can help with other things. Now because we want to talk about how OpenShift container storage fits with the MQStreams let's look a bit more about what kind of storage MQStream supports and what are the requirements for storage which we have. So on OpenShift we currently support three types of storage. The first one is the FMRO storage which is using the empty dear volumes. So if you don't know what the empty dear volume is on Kubernetes or OpenShift you can imagine it as a temp dear somewhere on the host. So it works quite well for things like development or CI pipelines but it doesn't really provide any data reliability so it's not supported for production. Then the second storage type which we support is persistent volume claims storage. So what we do here is we just use the storage classes and persistent volume claims as a Kubernetes based mechanism to dynamically provision the storage and that's really great because it means we do not need to kind of write a separate support for each kind of storage type which Kubernetes or OpenShift support but once the support exists in OpenShift or Kubernetes we can automatically take it and use it. And then the third type is something called J-Bot storage, just a bunch of disks which really what it does is it allows you to specify that the broker should use multiple volumes of one of the two types I talked about before and that's useful to increase the capacity and performance of the brokers in some cases. Now Kafka was basically written to not require any sophisticated storage features or appliances so it doesn't really need storage mirroring, it doesn't really need rate arrays to make bigger disks instead it doesn't need read write many disks so you can easily use even local storage with it and the actual requirement which we have for the storage is that it needs to be block storage because file storage things such as NFS does not really work well with Kafka and is not supported so what is really the main requirement is to have block storage and ideally with XFS or X4 file systems. Now the OCS block storage is of course one of the supported storage types and while it might have some features which I said before are not required the fact that they are not the minimum requirement for Kafka doesn't mean that they are not useful and in the second part of this presentation Karan will talk a bit more about where OCS can add the value over some other storage types. And so I'm really from the engineering not really from the field but I get involved in talking with our field people and with some of the customers and OpenShift Container Storage actually comes up quite regularly in the AMQ Streams discussion and while in the next slides we will show some of the numbers and examples mainly from Amazon AWS because that's easy to reproduce for everyone. It actually comes up quite often in discussions with the customers or users who want to run AMQ Streams on-premise because they often start up with some questions like does Kafka support NFS can we use that and the answer is no but quite often the users don't have anything else on-premise and that's why how kind of the discussion quickly turns to OpenShift Container Storage because that's an interesting alternative for them. And just remember that with OCS you always need to use the block storage with AMQ Streams and not the file storage. And with this I will hand over to Karan who will talk a bit more about the specific details. So Jakub have gave you a pretty nice introduction to Kafka and he also painted like what storage requirements Kafka imposes on you, right? So in the second half of the presentation we're going to transition to how OpenShift Container Storage adds value to your AMQ Stream or Kafka deployments. So if you're running AMQ Streams on top of OpenShift then by introducing OCS for your persistence storage layer for Kafka it gives you additional performance and additional resiliency. We're going to talk in detail about these two parameters. So as Jakub mentioned Kafka is natively designed with fault tolerance and you know it's inherently distributed and provides better. It's pretty good in doing that but OCS can add additional resiliency features and could provide you additional improved service availabilities during infrastructure failures like a node goes away or a storage volume destroyed itself. So these are the benefits that you can get from by using OCS. So let's quickly cover performance as the first thing. So in our lab we have tested various different combinations of storage vacants. So we have tested this on AWS. So standard OCS OpenShift Container deployment with three masters and three workers for Kafka. We have tested different versions of OCS just to make sure whatever we are testing is consistent and just to have various different data points. So we have tested OCS 4.2 the current release and we have also tested the future release of OCS 4.3 which is coming in a few months I would say. We have also tested different storage classes which Amazon offers. So EBS, GP2 and provision IOPS storage classes. We have done a comparison just to see how different storage vacants performs once you fly Kafka alone on top of it. The architecture looks like this. Everything is running inside OpenShift and storage is provisioned using OpenShift Container storage. Each of the Kafka broker have their own independent block volumes because Kafka works nicely on block and the supporting component which is ZooCuber has also got some tiny bit of block storage so that it can store its metadata information. On the load side we have tested, we have used Open Messaging Framework which is again a CNCF project. We applied Kafka, so a Kafka producer app, eight of them were writing to the topics of A&Q and then on the other side another eight consumer apps were reading the same topics from the Kafka topic. So this is some pretty standard Kafka architecture. So let's look at the performance. There are a lot of information on this graph. So this is performance characterization across different storage options. So before we go into the numbers let's talk about the config real quick. So three Kafka broker pods and with each pod having three... So three storage nodes for OpenShift Container storage because storage has to run on a node so which is why we have to use a physical or a virtual instance on AWS for OCS. Standard small message size and one PV for every broker pod in Kafka. On the y-axis we have tested produced messages per second, the metric that we were looking. In the next slide we have messages consumed per second. So on the column side here we have tested OCS on OCS consuming EBS for its storage needs which is the current design. So for OCS you need to use EBS for storage layer. We have also tested the upcoming new feature in OCS which makes use of local instance storage just to make sure just to compare how performance looks like. And then the third one is using AMQ on top of EBS without using any OCS layer. The last one is ephemeral storage which is not something you want to do in production so I'll just keep it side right now. It's mostly for Dev and QA testing purpose. So as you can see this from red and golden lines, OCS on local storage delivers the best performance across the workloads that you've tested here. And OCS when running on EBS imposes some performance stacks. The reason is that we are running a distributed storage OCS on top of EBS which is also again another distributed storage. So which is why we have to pay a performance tax that you can compare between golden and blue. So which is why there's a performance tax. So some ways that OCS on I3 instances delivers you the best performance that you are looking for. Same story for the second metric which is the messages consumed per second. Again OCS on top of I3 instances delivers the best performance. One thing we don't have in this graph is we have not tested Kafka on direct I3 instances which Yako mentioned that Kafka, usually Kafka people deploy Kafka on physical machines and they just use JBords or local storage. So we don't have that metric so we accept that. So this one was standard performance on a steady state Kafka cluster. We thought let's induce some failure in the Kafka system. So we have triggered, we have destroyed a part and tried to inject some artificial failure scenarios into the situation and tried to measure how much does it take for Kafka running on OCS versus Kafka running on EBS. Hit on the recovery. So you can see this, the red line is Kafka on OCS and the blue one is Kafka on EBS. So the smaller the line, this line is, the better it performs. So you can see this, the red line quickly goes back and the partial recovery is pretty much fast on OCS-backed Kafka clusters compared to EBS because Kafka has to do, you know, some regeneration of data because EBS does not provide, you know, storage level redundancy, which Kafka requires or not requires but Kafka would enjoy if storage by default provides that. So yeah, OCS helps in faster recovery of the ANQ cluster in case of infrafilers like node goes down or storage goes down. We're going to talk about that in a few slides. So this is the additional resiliency part which OCS brings to the table for Kafka. Kafka is no doubt inherently resilient but adding OCS gives it resiliency plus plus. You can think of this. So this is the performance metrics from a different cluster that we have tested because we have been getting some questions from the feed like, I need both performance and resiliency which is multi-AZHA. I'm going to talk about this word in the next few slides. For Kafka, so which storage should I use? Should I choose OCS or should I use the more expensive storage on Amazon which is provisioned IOPS? So we did a comparison between these two, one of these two storage classes. So the first one is AMQ on OCS and the last one is AMQ on EBS, provisioned IOPS. We found that because eventually at the end both these storage classes are basically using NVMEs underneath but we found that OCS performed slightly better compared to EBS provisioned IOPS which is kind of nice. And for both messages produced per second as well as messages consumed per second. So yeah, if you're an architect, if you're designing clusters of Kafka for a customer or even for your companies then probably this makes sense to like OCS or IT instances provides you the highest performance and the resiliency that you're looking for multi-AZHA. And because EBS does not provide multi-availability zone, high availability kind of, okay? Because it's a zonal service. I'm going to again, I'm going to talk about this in a few slides but you will understand that. The second question that we have got from the field is like, hey, I love to run, let's say I love to run AMQ streams on OpenShift container storage but you have like two storage classes over there. You have block storage, you also have a file system storage, FFS. So which one should I use for my Kafka clusters? So Jakub has already pointed out that NFS or ReadWrite many volumes are not, I mean Kafka works on that, you could run Kafka clusters using that but in the longer run there are stabilities problem if you're using file systems underneath Kafka cluster because of the locking and all these things which are there in file systems. So hence, if you are designing this on a Kubernetes native environment you should be using a block storage provided by OCS or even in general you should use block storage for the Kafka needs and file system causes stabilities and performance issues. So yeah, so this is what we have on the performance side. The second part of the equation is resiliency like how could OCS helps in improved resiliency for Kafka clusters. So for that, we first of all need to understand the problem. What is the resiliency problem do we have? Let's suppose I'm running my Kubernetes and Kafka environment on public clouds because this holds true for most of the public clouds, every public cloud has their own az. So in this diagram we have like we have OpenShift cluster running on, let's say running on AWS and you have your Kafka clusters spread across multiple availability zones. Your Kafka cluster is consuming PVCs which are being backed by EBS volumes, all good standard architecture. You're using your system, Kafka takes care of the data application and but if anything goes down, let's suppose any of the EC2 instance in any of your az goes down or let's say your EBS of that az goes down or the entire availability zone goes down because it's all silicon, things can go wrong, right? So if a mishap happens in this environment, Kafka by its nature, given that you have enough nodes in your architecture, it will spawn up another part because it's all Kubernetes and it will request for a PVC and EBS will give it another block storage however this new block storage would be empty because it's a fresh PV which EBS to this class has given it. So Kafka has to reconstruct all the data in the failed EBS volumes and it could take minutes or even days depending on the size of the data that Kafka needs to rebalance. So during this time you're taking your resiliency of the system. So think another mishap on the same az or a different az happens, you are in a problem. You're losing your availability of the service. So that's the problem. Let's now understand how this problem could be solved using OpenShift Container Storage. So exact same architecture but this time we have an OpenShift Container Storage layer running on top of EBS which is OpenShift Container for storage 4.2 thing. You can create an RBD storage class out of EBS storage class and this time your storage would be across availability zones. So again standard data application taken care by Kafka, everything is good and storage level replication is taken care by OCS. So all the data is tripled replicated across the availability zones. In this case let's suppose same thing happens if the az goes down, EBS goes down or the entire availability goes down. Kafka will spawn up another pod and it will request for storage but since it is backed by now it's backed by OpenShift Container Storage. OCS will give Kafka the same PV or PVC which was originally assigned to your first pod and Kafka just need to sync up between the changes. It doesn't need to replicate or regenerate all the failed data because the data is still there because the application was taking care by OCS. So this is the additional resiliency you can get out of OCS in your clusters. It can handle az failures in case of public clouds. But wait a minute, you can definitely debate on here like hey, okay, so we are going through the performance text problem which I've explained in my performance testing. We are running a distributed storage on top of a distributed storage. So whether I'm going to lose some performance? So the answer is yes, you're going to lose some performance because it's storage on storage. So how can I solve this at the same time having the same resiliency? So you can deploy an OCS on top of AWS or VMware or any public cloud of IT type instances which are equivalent of Azure and Google. So these physical instances provide NVMe volumes exposed to the OS and OCS can use local storage operator which is a new feature it's going to come with in coming releases which means there is no performance tax layer of EBS. So we have completely isolated or removed the storage layer provided by the public clouds and you can simply have an OCS simply have Kafka requesting storage directly to OCS and if anything goes down, anything goes bad in your environment Kafka will take care of the data because it doesn't need to regenerate all the data the data is already there and it's fast because it's packed by all flash storage. So this is how OCS adds performance and resiliency benefits to existing resilient architecture of Kafka. Real quick going through deployment. So it's a standard ML file which you could just use. You can go and take a look at streamz.io. It has a fantastic documentation. Yakuban esteem is maintaining that. It will give you enough guidance how to deploy your first AQ clusters on top of Kubernetes or OpenShift. With respect to storage thing, you need to make sure that you have an OpenShift storage cluster on top of it, top of your OpenShift platform and then you just need to add these like four lines in the respective sections of Kafka and ZooKeeper so that your Kafka and ZooKeeper clusters can request PVs from OCS. So just six lines of code and you will get additional resiliency and performance from the system. If you want to go give it a try, you can have this URL and which I've created like you can have a step-by-step instructions to deploy AMQ streams backed by OpenShift container storage on top of OpenShift platform. So this guide will give you step-by-step instructions how to do that. All right. So this is all we have today for you guys. If you have any questions, you can ask in here. Drop into the chat. We can answer otherwise. Yeah. So back to you, Karina. Thank you so much, Pran. And Jakob, that was excellent. Thank you, everybody, for joining us today for all things data and our special guests, Jakob Schultz and Pran Singh from Red Hat. And join us next time. Again, same time next Tuesday at 8 a.m. Pacific, 9 a.m. Mountain, and we will all see you soon. Thank you again for joining us.


Going on everyone Sam here with late sports cards We are ripping 20 23 tops tier 1 baseball hobby 12 box case number 7 picker team style read off Lucifer names in our teams got Alex and with the Cardinals Andre say blue jays Bill H. Orioles Chris V. Diamondbacks Chris e-braves David O. Reds Astros Dennis V pirates Marlins Eric W. With the athletics George S. Guardians Jeff T. Has the twins and that's Jeff M. Cubs Tigers Padres Jordan then with the Rockies Joseph S. With the White Sox Kirk B. Yanks Matt P. Rangers Matthew C. Brewers Michael C. Angels Mitchell H. Rays Robert V. With the Phillies and the Royals Robert F. Giants Shane C. Mariners Terry B. Dodgers Red Sox Zach T. With the Mets good luck here everybody 11 to go on number 8 two more PYTs left on the day 1 2 3 4 5 6 7 8 9 10 11 to go on number 8 everybody 11 to go on number 8 Number 8 There's your link 11 to go Let's get after it. Good luck guys. Greg. What up, buddy? Greg got a long time. No, see is right. He's saying them now All right, we've got Buster Posey to 99 for the Giants Higantes and Raffy Devers to 400 for the Red Sox nice one there in Boston There again with the athletics in eight. We are down to 10 almost in the single digits everybody That throw does make for great rebounds very very true ghost nice one here Jersey Otto 99 is Larry Walker Nice one there. Congrats on that Larry Walker. You know, it's funny ghost is from the town over for me Where I grew up and yeah, Caleb Killian to 299 rookie Otto for the Cubbies. Nice one there congrats Cubs Caleb Killian Phil Necrow to 400 going to the Braves nice one there Atlanta. That's cool. I like that a lot Like that's a lot Martin Z with the Orioles and Cardinals in eight William M with a mega dual case spot We are down to eight to go now. Thank you guys Appreciate that. There he is. Mr. Palmer Jim Palmer going to Baltimore to 175 Congrats on that Baltimore Orioles Mr. Palmer and Davis Martin to 299 rookie Otto going to the White Sox. Congrats on that Chicago To 299 Also have a case of tier 1 divisions for to go on that one that is coming up very soon as well Of course our mega dual case is what we're all really excited for that is gonna be a George Brett nice one there. Congrats on that Royals. Let's go Dennis Dennis how about a little last spot mojo? That's going to the Marlins. That's gonna be Marlins Pudge Ivan Rodriguez 60 of 149 tier one talent. There we go. Dennis. Why you don't let the last team sit right there everybody Very nice. Congrats on that Dennis take that all day Rookie Otto for the potential World Series winning Rangers Bubba Thompson to 299 Congrats on that. Very nice. Very nice. Greg with a mega dual case spot. Let's go Let's go Greg. Thank you, buddy. Caleb Killian to 299 rookie Otto for the cubbies as well. That might be sold out now actually That is gonna be very close to being sold out. It is sold out mega dual case is sold out everybody Thank you all again. That is a case of this tier one and a case of gilded Boston cream cooking up in that tier one oven. Oh boy flavor Torres the 400 nice one there yanks. Congrats New York New York All right, Reds Tigers Astros Dodgers Brewers Mets Pirates Padres all available in the next one eight to go Dan's be Dan's be swanson to 175 going to the Cubs. Nice one there on the auto World Series champ with Atlanta a few years back And then rookie Otto for the D-Vax buddy Kennedy you 138 out of 299 Appreciate that to me appreciate that to me. What up, buddy Patrick wisdom to 400 Congrats on that cubbies Greg I'm with the Mets Dodgers and Reds in number eight. Let's go Greg. Thank you, buddy As always appreciate that Greg That's somebody pulled an albis barrel out of this has pine tar a little all over it. That's pretty nasty That's pretty sick to 149 Barry Larkin. I swung there for the Reds since he Uh Tigers Astros Brew Crew Pirates Padres five five to go in the next PYT David VR rookie Otto going to the Higate Congrats Giants to 299 Boston Wade Boggs to 400. There you go, Terry That's buddy Greg I'm also with the Cardinals Rangers Red Sox and Orioles and number nine. Let's go Greg. Come on Greg Let's go, baby. Thank you, brother to 299 James Altman this card Far out man You're gonna be honey Tim Hudson to 299. I swung there. He's talking about the Purdy Greg I swung there some honey One of my buddies from Connecticut. He's a huge be Tim Hudson collector. Oh, really unbelievable He was super underrated that that rotation of him Tim Zito and the other one older was it No, let's go Greg. Hell yeah, dude Max Fried to 400 Nice one there Greg. Let's go Greg 14k for a Purdy.  Let's go Tom Tom with one two three four five six seven eight nine ten personals. Let's go Tom Hell yeah, that's Kerry bonds to 25 Kerry Carpenter nice one there for the Tigers Of course, I call him Kerry bonds because Spencer officially ruined this guy for me The bot Cannon Smith and shake, but it's 299 for the Pirates rookie auto I know I'm kind of surprised it didn't that and the Bryce young Nebula didn't make it either It's kind of surprised about that Not gonna lie, but oh well, what can you do? Braves legend Tim Hudson Candler's blocked on my end apparently That's somebody else who hasn't made the trash list. That is a easy potential member Jose Conceco for the Oakland A's nice one there to 99 Got to write a letter to HR George with the reds in nine. Let's go George. Good stuff, buddy So we're gonna teach you something nice there man Kyle Stowers rookie auto going to the Orioles to 299 rats on that one Baltimore and Lee Smith tier one talent to 249 go to the Cubbies This one there at Chicago 177 at 249 Moving here everybody glad we have that though Timmy. That's awesome, man There he is mark McGuire to 400 now now I can see you. I don't know why I didn't see you when you said I Know chairs are safe now. I know rest in peace Bobby Knight is right Yeah, I swan here ATL Michael ATL Michael Michael Harris to 299 congrats on that Atlanta Here we go. Bravo's capturing Bravo We lost Bravo Chrissy congrats Chris and nice one here. I think this is going to Terry. I believe her Yes, it is Terry. How about a one-of-one Max Muncie? Congrats on that. Let's go dude.  Yeah, Terry. Congrats brother. I swan there with the Los Angeles Dodgers one-of-one uno de uno Good stuff buddy. Good stuff Get that into a mag for you. I know I can't wait for cod dude. I cannot wait for cod Javi bias to 400 I swan there for the Tigers the Braves Killed at the last two boxes to 50 chipper Jones. I swan there.   Let's go Terry. Good stuff buddy. Good stuff man. Hell yeah, Terry. Oh man Torrey Hunter to 400 for the twins Tom with a spot in divisions in a spot in each of the flux. Thank you Tom We're down to three to go on the visions everybody Halo jump from a play cod. I tried to get a drop shotting for a bit, but I wasn't good enough Chris Morel to 299 You about that? This one there. I Can't wait Andrew for the OG fortnight on Friday. We're all amped about that. That's a word about so And nice one here nine of ten Ryan Aguilar nice one there for the angels. Congrats. He lose again everybody Tigers Astros Brewers Pirates Padres all available the next one five to go Five to go in the next whoa next Here one quick scope.  Yeah, that is gonna be that's nasty right there, dude George springer 400 Nice one there for the blue jays. We got a Randy Johnson to 200 for Arizona Nice one there. Let's go Greg with a division. Thank you Greg Terry with the D-backs in 12 and 13. Thank you Terry Let's go man. How about 38 of 49 Alex Rodriguez as a Seattle Mariner? Congrats Seattle. That's awesome little a rod action riches boy right there What's up Ramirez, how are you doing buddy and Corey Lee to 299 rookie Otto for the Stroh's Congrats Houston H town stand up Very nice. All right quick recap here everybody start with our relics. We've got Randy Johnson George springer Tori Hunter Javi Baez mark McGuire, Jose Canseco max freed Wade Boggs Patrick Wisdom labor Torres George Brett Phil Negro Raffi Devers and Buster Posey He's the reason I'm like Corey Lee Alex Rodriguez Was that raw? Yeah, Ryan Aguilar Christopher morale Charles LeBlanc Chipper Jones Michael Harris Lee Smith Kyle Stowers Jackson Smith Oh no, not Jackson cannon Smith and jigba Kerry Carpenter Tim Hudson James Outman David VR Barry Larkin buddy Kennedy Dan to be Swanson Caleb Killian Bubba Thompson Pudge Rodriguez Davis Martin Jim Palmer Caleb Killian Larry Walker and then a one-of-one Max Muncie for the Dodgers That will do it for the break guys. Thanks again for the fill Next up is number eight


From around the globe, it's theCUBE with digital coverage of Pegaworld Inspire brought to you by Pegasystems. Everybody, welcome back to Pegaworld Inspire 2020. This is theCUBE, and I'm Dave Vellante. We're here with Adam Field, who's the head of innovation and experience at Pegasystems. Adam, thanks for coming on. How you doing, man? It's my pleasure, Dave. I'm doing well. How are you? Good, thank you. I'm excited. We're talking innovation. Innovation hub, but start with your role. I love the title. What do you do? Give us the background story. Yeah, I get that question quite a bit. So I've been with Pegaworld a little over 15 years now, and I've held many roles, but currently as head of innovation and experience, we have a team. I like to call them creative misfits, if you will. We sort of bridge that gap between technology and creative. We do research on emerging tech and try to understand how our clients might use it, how it's going to change the future of work. That's the innovation side. On the experience side, we do things like for these Pegaworld events, we match where art meets tech, and we build these experiential things that people come and see at our events. We build all the demos and all the production that you see on the main stage. So we kind of touch a lot of different things around the future of where technology is going. Well, I can see, obviously you're innovative. You've got the awesome setup there, the great mic sound, fantastic. You look good. So now you've been involved in previous Pegaworlds, both from behind the scenes and out front speaking. Obviously this is completely different. How did you prep differently for Pegaworld 2020 virtual versus what you normally do? Yeah, right. So this will be my, well, 16th, I guess, Pegaworld. And obviously this one stands out as the most different. Normally we'd be in Boston today. We would have been working on our stage production and on a floor that's 170,000 square feet big with dozens of booths and hundreds of demos. And obviously this was completely different. But as far as prep goes, I remember the day we learned that early March, this was going virtual. And after a few moments of sadness, the team really came together. And I remember the first thing we talked about is, we're not gonna take a three day event and try to put it all online. Let's, we know people's time is valuable. Let's figure out how to take just what's important and get it out to people so that they're inspired to move forward and engage with Pegaworld. So I think that's really been the biggest change in how we've prepped. Well, you know, I think that's a great point because obviously theCUBE has been very much involved in these virtual events and people send out the note, hey, we've made the tough decision to go virtual. That's easy decision. You really had no choice. The tough decision is what you want to preserve from the physical and understanding that you can't just pop physical into virtual and you got to create a whole new content program.  Oh, you can't afford to Beyonce for Beyonce. Well, you better make your content interesting. So to me, Adam, that's the tough part. Help us understand how you thought that through and what the outcome actually is. That's right. We didn't have Beyonce, but we did have the Dropkick Murphy. So that was pretty cool. And they did a concert for us. So that's been great. But, you know, again, a lot of people talk about all this free time that they have. And I know I have two young kids who are schooling at home now, the job that's busier than it's ever been. I've tried to join a lot of these virtual events and frankly, I've gotten overwhelmed. So we took two days and we boiled it down into two and a half hours. And what we decided to do is we looked at all the areas which we go to market and how people design and deliver their apps and some of the tech like PegaCloud that they use. And we went to our, I went to my extended team and I said, normally you have 75 booths. We're going to boil that down to 25. Let's work together to figure that out. Normally your demos might be 20 minutes when someone walks up. We want to make them seven. But I think the biggest thing that we did, we said what we don't want to lose is that interactivity. And so we had online dozens of Pega experts we could ask questions live. Alan was online doing answering questions live. We made sure that we included live components. Our host Don Sherman was live from his house. We didn't just prerecord everything because then why would anyone come join when they could just go watch it 30 minutes later on your YouTube channel? See, that's innovation to me is having that combination of live. Obviously, you got to do some stuff prerecorded but having a live component adds a dimension. It's challenging but that's pushing the envelope and I love it. Adam is roles. The roles are different in a virtual event. Are they, you're not doing site inspections. Like you said, you're not dealing with 170,000 square feet. How did you guys rethink the roles for a virtual? Yeah, so there were some teams whose world was completely upended when this all went virtual. The people that do exactly what you were just talking about dealing with hotels and vendors and things like that. And I got to tell you one of the most events called Pega World Inspire and not to sound too cheesy about it but one of the things that was really inspiring was to see how everyone stepped up and said truly, how can I help? And what was really neat about it is we saw different skill sets come out of people that maybe they hadn't had the opportunity to flex before where they might have worked on one thing that was no longer needed because of the change in the format and they jumped in to become copywriters or liaisons between, because now we have new vendors in this tech world that we didn't have that we turned around in just a matter of weeks. We had people like on my team who normally last year built this massive physical exhibit containing mirrors and lights that became video producers to produce some of these live videos that we did. And one of the things that was really impressive, you asked earlier about how did we prep differently and what changed? We looked in the marketplace for different tech and how to bring our CEO and our host and our head of product and everyone together live and split screen. And when you're a big studio and you have that equipment ready to go that's easy, but when you're just getting average people in their homes and you want to put all that together, we're finding some of the tech in the marketplace just wasn't there. My team built some new video chat technologies that they actually used to produce this in real time. So that was really impressive to me how we turned that around and really innovated not only the things that everyone sees but all the stuff behind the scenes too. See, again, I think this is, what's amazing to me is as I learned more and more about Pega interviewed Allen earlier. Pega is all about being able to adapt to these changes. So a lot of the processes that we're using in virtual events, they're unknown. You know, normally software, right? Through the history of software is okay. Here's how the software works. Figure out how to fit your process into it. And very rigid. Today, you know, the last three months with this lockdown and this coronavirus have been completely unknown. And so that's sort of one of the hallmarks of your company, isn't it? That's right. I mean, we've had the tagline build for change for a really long time. And I will tell you, I remember in that first meeting again when we learned this was going virtual and someone stood up and they said, guys, we're about to live our tagline. And people really do believe in that because we go to our clients every single day and say, change is what's gonna make you special. Change is what's gonna make you different. Now is your opportunity. Seize that change and run with it. And so we said, look, we can't change the world right now. We know we gotta go virtual. All we can do is change the type of event that we do. We're not gonna do the standard event that we think everyone else is gonna do. Let's do it differently. And, you know, today was a pretty good example. I think we achieved that. I think a couple of things from a challenge standpoint, you mentioned the chat, how do you get people to engage? You had to sort of invent something and really think it through for virtual. And I think the other is tech. People come to these events, they want to touch the tech.  You got to take us through how you thought through that and what the outcome is. Yeah, so that is the toughest part. And I got to tell you, you know, all of this being said, I'm looking forward to someday being able to get back and meeting my clients in person. And I'm the type, you know, when I see you on the floor of the innovation hub, I run by your booth and high five you for all the great weeks of hard work, you know, and I love to see people's face as they see the demos. And that's tough not being able to see them smile and get that moment of wow. But what was interesting was it really helped us hone our messages. I think we really realized when I went to everybody and said, you don't have 20 minutes, you have seven minutes. Here's a template, you know, to follow to be able to tell your story better. And people started thinking in that mode of storytelling. And what was interesting was a lot of people came back to me and said, actually, you know what? I can tell that story in a much more crisp way and really show people what they need to see in a much faster timeframe. And what it really allowed us to do was find those bits that we thought were most important, find those demos that we think are most important and just, and, you know, bubble those up. One of the things we also did too, we took the opportunity to say, you know what? We're gonna be online. I watch my kids, my kids are avid gamers, whether I like it or not. And they, you know, watch these Twitch streams. And we thought, well, we should be able to do that with even corporate software. So we had these live build sessions where we took some of our developers and I said, you're gonna be put on the hot seat for 15 minutes on script and we're gonna let people just guide and direct you. And they were a little nervous at first, but they went off great. And it was a new format we had never tried before. So if we keep doing these types of different things and we just embrace the moment that we're in, I think people will really, really come to it and get some value out of it. I mean, that's awesome. You've got to keep your audience engaged. And so you do lose, you don't have a captive audience.  You got to take 20 minutes down to seven minutes. But so you do lose some of that. But what do you gain with virtual? Well, I think one of the things that you obviously gain is you can be more widespread. So, you know, this event reached tens of thousands of people in dozens of countries. I did an event first week of April. So you can imagine, you know, we had two weeks of Turner and I was supposed to be in London and Amsterdam presenting in soccer stadiums. And instead, we made that a one hour virtual event. And we thought, well, we're just gonna get people from the London market and from the Netherlands market. And it turned out we got people from all over the world to join. So one of the benefits to this is the reach. So we're able to reach a lot more people. I'd say one of the other just things that we realized after too is we're creating a lot of content. We filmed all of this as we were rehearsing and we're going to put it up online later. So now we have all this great content that anyone can use and go view later. So that was sort of a, you know, unexpected outcome as well. Right, yeah, you lose the airline miles, but you gain those, I was going to say, you gain the post. I don't mind not traveling as well. Yeah, I hear you, hear, hear. But you do gain that post. And I think with physical events, people always at the end of it, it's like, I've never given birth, but I've witnessed it many times. But people feel like, okay, I got to just chill out now for a couple of weeks. And then when they come back, now they're swamped, they've got to catch up. And I think people are realizing, wow, there's a real opportunity to maximize the post event here, post nurturing, keep streaming out content and continue that engagement. That is a plus of these virtual events. Oh, for sure. And, you know, we started early on deciding, how are we going to do, what are we going to do as follow-ups? You know, that European event that I talked about, once again, instead of taking all these different markets and trying to replicate it, we did one one-hour event. But then because we were in the early days of COVID and some of our clients weren't able to get recorded and speak, we did subsequent webinars in the weeks following them and the attendance was fantastic. So it allowed us to plan ahead and have a lot of follow-up activities that were starting to launch right now because as soon as the event ended. How do you feel about the outcome for Pega? Do you think it was better, worse, or the same or just different? I'm going to go with different, you know, like I said, I get energy. I love being up on stage in front of 5,000 people. I love meeting my clients in person. I love the energy of being with my colleagues. But, you know, it is what it is that we had to do it. And I think what we really embraced it. So I'll say it's just a different way of doing things. But, you know, I do look forward to the day that I'm able to go meet my clients again and get back on stage and produce some really great things. And once again, being able to physically see our attendees go, oh, when they actually see the software in person, that's the most rewarding thing for me. It's going to be interesting as we come out of this. I mean, very clearly things are going to be different. Probably going to have hybrid for some time. Maybe even indefinitely. But I'm interested in some of the learnings, some of the things that you think will be permanent. Some of the advice, and one of the things I always say to people is don't start with what software we're going to use. End there for your software platform. Think about, you know, the experience that you want to work backwards from there. But what other advice would you give folks given your experiences? You're so right about that point. I remember interviewing a lot of vendors that we were going to use to bring this online. And we were telling them what we wanted to do. And some of them said, no one's ever asked about that before. We can't do that. So you're 100% right about that. The advice I will say, and the thing I do worry about a little bit is at first, people were a little bit more accepting if maybe the video quality wasn't as good or the content was like any old webinar. As months go on, expectations are going to be higher. People are going to have attended a lot of these things. So you're going to have to keep upping the game. And I think the advice I would give is try to take what's great about an in-person event and put it online. But don't try to replicate the event and put it online. And some of the best things about in-person events are just the live nature of it. Take the risks. Do some live stuff. People will really appreciate that. You'll get a lot of credit for that. The interactivity is what's important about a live event. So as best you can, figure out how to make sure there's some interactivity. Now, in the early days, I think it's going to be some live Q&A. As we move on, it'll be real private rooms with experts that you're able to have one-on-one chats and go through and bounce around and be able to talk to people, just like you would accept between two cameras instead of in-person. So I think everyone as months go on are just going to have to up their game. I think that's great advice. You're absolutely right. Up your game, up your brand. Get a good camera. Get good sound. And it's going to just help your personal brand and your company's brand. Oh, we learned what it was like to try to ship microphone and camera equipment around the world overnight. So we're experts at that. If you've got any questions. Well, I mean, what a difference it made. So Adam, thanks so much for coming on theCUBE and sharing your experiences. You guys have one of the best that we've seen at the virtual event platform. So congratulations on that and really appreciate your contribution to the community. Thanks, it's my pleasure. Great to talk to you today, Dave. All right, keep it right there, everybody. This is theCUBE's coverage of Pegaworld Inspire 2020, the virtual event. Right back for a short break.


Hey guys This video notice I was a bit on the edge and I bet that every low cost rider will understand When you're always counting the pennies because you don't have that much money to spend on your bike or on your gear And you feel that you cannot push yourself to the limits or find your new limits in order to improve that ends up being a bit frustrating and On the video I ruined completely my boots and I went kaboom Just gave me a huge full


It's also great to be here with Premier Kanu with Manitoba's housing minister Bernadette Smith as well. Wab, I know how important it is for you that we continue to fight every day for an economy that is fair for everyone. That's exactly what we're doing today with a significant announcement that will protect and expand affordable housing. The affordable housing complex where we are today is multi-generational, meaning some units are for younger families and some are for seniors. I had an opportunity to sit down with Jocelyn and her girls a little bit earlier but also with a pair of seniors, Marlene and Lou, who are talking all of them about how good a place this is to live, how affordable it is, but also the sense of community that happens here. Rents are affordable and a good portion of them are based on people's income and the intergenerational connections create a strong sense of community. For families like Jocelyn's, this is a good place to raise kids with peace of mind and stability. Now this place is run by a non-profit called the Winnipeg Housing Rehabilitation Corporation and we need more affordable housing buildings just like this one right across the country. Unfortunately, too many of those places are under constant threat of being demolished to build condos or sold to speculators and large corporations that will increase rents at turnover. People are being priced out of their communities and that's not okay. So we have to help non-profits and community partners acquire units and preserve rents at a stable level. That's why we're here to announce the one and a half billion dollar Canada rental protection. The Canada Rental Protection Fund will aim to mobilize investments in financing from the charitable sector to provide affordable housing for non-profit families. This is a very important part of the plan. The Canada Rental Protection Fund will aim to mobilize investments in financing from the charitable sector to provide affordable housing for non-profit families. A good proof of concept of this initiative is the BC Rental Protection Fund in British Columbia. They recognize that for every new affordable rental home that is built in their province, four more are lost to investors, to conversions, to demolition, and to the public. They recognize that for every new affordable rental home that is built in their province, four more are lost to investors, to conversions, to demolition, and to rent increases. And this is happening in communities right across the country, which is why the federal government is acting today. We're here to build more housing more quickly and to build the necessary infrastructure. We're here to help build apartments with reasonable and affordable landlords, and we're here to protect the affordable housing that already exists with models that work well for low-income people. We work every day to give each generation an equitable chance. We're fighting every day to build an economy that helps every generation get ahead, including Gen Z and Millennials. Because for Canada to succeed, we need everyone to succeed.