Hello, everyone, and welcome back to the state of the web. My guest is Colin Bendell. He's part of the CTO office at Cloudinary and co-author of the book High Performance Images. And today, we're talking about the state of images. Let's get started. [?]. Colin, thank you for being here. I want to start by just talking about some of the things that developers have probably already heard about with image optimization. Let's start with layout stability. What are some of the things developers could do to eliminate that jankiness of image loading? Yeah, so I mean, that's one of the many challenges we have with images and video on the web, right? Is that experience where the user is engaging in the page and all of a sudden they go to click on a link or scroll, all of a sudden everything moves and it's gone. And we talk about jank a lot of times because it's about JavaScript, filling that buffer in the time loop and be able to do too much and you can't do that.  And so they come in later. And as they're coming in, if the browser didn't already have a box predefined, then when the image finally comes in, it realizes, oh, this is a six by nine image and then has to reflow everything and you've got the real layout. And so there's a number of strategies that we've used in the web because ultimately we're trying to stop the user from having a poor experience, right? And we don't want to distract the user from that experience. So we've used in the past things like defining the height and width of an image, which is great in a fixed layout kind of style. And this becomes one of those many different strategies of, well, it kind of worked in the past, but now that I'm using max width and min width, and I'm using responsive layouts, now how do I also define that height? Because if I can't define the height in absolute terms, then we still have the same problem. So some of the more advanced strategies now are using techniques like low quality placeholders, where you have an image that maybe is encoded in base 64. So it's like a very small image or maybe even an SVG that gives enough information to the browser early. While it's doing layout, to be able to already define what the height and width of that element is. These are all challenges with the browser. And later on we can talk about some of the new enhancements like intrinsic sizes and aspect ratio, which are also trying to provide us new techniques to give that layout piece. But ultimately the biggest boom for your bang for your buck is to make sure you have defined those widths and low quality placeholders are a good way to start with that. And if we can start transition to intrinsic sizes and things like that. Another age old battle developers have been fighting with images has been the balance between quality and file size. And the image format has been one way of ensuring that you minimize the file size. So what is the conventional wisdom for using the right file format? Also every day there's a new file format that somebody's talking about. And we've got a lot of great standby formats. JPEG, PNG, GIF. I mean if you've been on the web and you search around for image optimizations, you probably have seen a lot of these talks talking about these standard formats. And even some new formats like WebP or FLIF or other standards out there. Now the challenge is that each of these formats have a different lineage, a different history. JPEG's been around for 25 years, 26 years. And it's a great format because it does a lot of great things to reduce the bytes per pixel and give you a really nice small image. But it's designed for photographic need. So inside of that photographic you are expecting to have gradients of colors, that sunset picture or the clothing. And that's not always the case that you're using. So you have different formats that actually excel in different ways. So you have GIFs and PNGs, which are great for probably more illustrations or what you're doing with palettes or solid colors and large swaths. So that way you can, in the optimization of that format, they get looking for those patterns and basically reducing the bytes down. And so then you've got a smaller image that way. And that's what PNG. But because of where they find those patterns varies dramatically between these different formats.  What are you trying to do? What kind of content? Are you trying to create icons, illustrations? Are you trying to do photographic? Are you taking pictures for a news event or for dinner in the picture of your plate? So each format has strengths and weaknesses to how you address it. And then you also can talk about formats like SVGs, which are great for vector-based imagery that can scale based on the display area. But you don't have an exact pixel that needs to be stretched or shrunk. So those are the formats we have. But the challenge always comes down to, what does the end user support? The browser or the app that is consuming this content, fortunately, here we are in 2019, JPEG, GIF, PNG, supported everywhere. They've been around for a long time. And these libraries are very old. They're very robust. So when we talk about new formats, that's the first hurdle is, how do we make sure that they can be supported? And then how do they compare performance-wise, both in decode but also bytes savings? Yeah, so the stand-bys are always the first place to start. And some of the features that you may want out of your images like transparency just aren't even supported in some formats. Oh, exactly. So JPEG is great for a photographic, but it doesn't do transparency. Whereas GIFs can do transparency, but single bit, whereas you can do much better transparency with PNGs. You can do that anti-aliasing with PNGs. So then you're also trapped with these trade-offs of what am I trying to accomplish and with the best format to get the byte savings down. It's good to note that this paradigm that we're evolving into, we're in this Instagram age, right? We've got lots of images everywhere to grab our attention.  We should have solved this a long time ago, but yet it's a new thing, right? We have this tension because we have content creators who are trying to create beautiful experiences and then us developers are trying to get a good score on Lighthouse and get our page performance down. And we have to balance these two objectives with just the format selection. Exactly. So once you've chosen a format, what are some of the knobs that you can fine-tune to optimize image quality? Yeah, OK, so the standard ones that you'll probably see a lot on the web for JPEG quality index, right? So you change the quality factor and you'll probably hear use quality 80 as a good. Now it's probably a good spot to talk about, when we say quality, we're not talking about percentages. A lot of people think quality 80% on JPEG. Well, actually, it's a unit base of 100. And each of those 100 map to a quantization matrix. It's a matrix, an 8 by 8 matrix that's applied in JPEG world to this image to try to reduce the number of colors, if you will, in this block. So if we can make those more whites the same kind of white, then we can easily compress those series of pixels out and we can get better savings and the more aggressive that we can take the range of colors down. So quality indexes in JPEG is the one most people talk about. You start with quality 80, quality 90. You get a lot of bang for your buck on average. Now there's other strategies, and we talk about other ones like using chroma subsampling, which is a strategy that comes from the TV industry, where you can separate the colors because your eye sees, you've got cones and rods, and you are more sensitive to brightness than you are about the color itself. So in, say, JPEG, you can express the bitstream of color in, instead of just RGB, you can do it in YCBCR. That's the chroma, which is the color, and luminance is the black and white or the brightness. So your chroma subsampling is a technique where you basically keep the luminance channel. And the chroma, you can share the color in, say, a 2 by 2 grid, share the same colors, put one pixel of color. And blending that with the black and white or the luminance channel, it perceives like different colors, but they're actually the same. This is how all TV generally works, is how we got color TVs back in the 20s, is by doing this trickery, playing with our eyes. So JPEG can support that. PNGs, GIFs, not so much. Another one is progressive displays. So how you progressively provide that content. So if you do one quarter of the image and then one half and then full general terms, those scan layers, as they apply, they bring in the resolution of the image. And by doing that, you're actually able to save some bytes in those layers in the JPEG world. So if you were to start from somewhere, I would say start with these three levers. Quality, probably 80 or 90. Use chroma subsampling. And again, there's footnotes there. And enable progressive on those images. So we should also talk about mobile and how not only are the screen sizes getting smaller, but also the data plans are getting more expensive. So in what role have images played in adapting to the mobile world? I've been doing a lot of research on my own about the consumption habits by users and looking at how the different users in different environments consume the web, what kind of formats are available, and how is that in different experiences. And I always had this mental model of mobile and desktop, and that there is these two paradigms. And if I track it by time of day, I expected to see desktop traffic go up in the morning, people go to work, and then it goes to Peter off in the afternoon, evening, as I go home, and then mobile traffic carries on. It turns out that I was completely wrong. That in today's day and age, what we see is yes, the morning, 8, 9 o'clock, you see traffic of desktop. But mobile's always there. And then in the afternoon, you have this decline. Five o'clock slight, slight decline, but it continues on all the way. And mobile continues on and bumps up. So the first distinction I wanted to make is that there really isn't a mobile versus desktop world. Desktop is there is not just a great box at our desk anymore. A mobile device includes our laptops, which we classify as desktop, but is really, we carry that around on the subway, and we do work. So the minor correction I want to make is that a desktop is actually a mobile device. So to your question, the real question you asked about is what should we do for mobile strategies? Well, the first place we start is, of course, we should be dealing with responsive layouts, right? Using flexible grids and all the great stuff to give that consistent experience, regardless of your form factor, whether you're viewing it on a tablet or a phone or a desktop. But this has a number of other challenges because you now have a very wide screen and there are very narrow portrait and landscape challenges. So where do I start? So the good news is that we've got for a number of years, we've had support for responsive images. So you can use source set and source. These two attributes on the image tag with source set allow you to specify to the browser, hey, I have a number of different versions of the same image that allow me to define different break points. So if you have a wide image and you want to say on a giant screen versus a smaller screen, it's the same image but I've got different pixel volume. So source set allows you to say, I've got a 200 pixel wide version, a 300, a 500, a 1,000 pixel, et cetera. But it's the same image just being resized. The other attribute we have, it's not an attribute, it's an element, is the source element. So in the picture tag, you can now have a source that says I have not just one image here but I have actually different cuts of that image. So I have the image that is square, perhaps in this layout, but maybe I've got a wider version as well if I've got a desktop view. So what can I do with that? So you want to specify your media queries and then provide that different context so you can have that very fluid experience of the same image, but instead of just resizing and making it squishy and small but you've got now maybe different cuts from portrait to square to letter box, whatever, based on the experience of the web. You make a good point about you can never really be sure is a desktop device purely the gray box sitting at home. But like if I use, for example, I use my mobile phone at home on my home Wi-Fi where the network isn't really affecting me. So you can't really make too many assumptions about like, well, I need to serve low quality images. They're on a small form factor device. It's not always true. So do client hints help at all here to understand the context of the user's experience? Yeah, yeah. So client hints is a strategy that allows the browser to send additional information along the wire to the content servers. And now this is at the HTTP layer. So you enable it through markup or through HTTP headers saying, you know, enable the client hints. And the different client hints that are available today are with viewport, with and DPR. So DPR, sorry, start with width. Width is the relative to the viewport. So the viewport is the screen. Width is the, how wide is this specific image? So as I said, viewport, width is the actual screen size. And then DPR is that the density of the pixels. You have a 2x or 3x. These hints, when they're sent across the wire and the content negotiation, allow the content server to say, oh, I see you're making a request for this image. I see that your layout is 500 pixels wide for this image. I will send you an image that is appropriately sized. The objective here is it'll help save you with your markup so you don't have to have, all the verbose markup that says, hey, here's all of the different source sets and all the different source attribute elements. You can now simplify that down and let the content negotiation define, you know, what should I send to that client? Now there's a, I'll put a footnote here because client hints has been around for a number of years. Some great work that was done to bring it to fruition, but it also has flagged a number of security concerns around fingerprinting because those client hints were also being sent to everybody. So we've had some actually some really good developments in the future policy spec and where we're evolving client hints now to be much more security aware. In fact, it's gonna hopefully bring us a spot where we can be a lot more security first conscious and actually help us lock down the user agent because the user agent's such a terrible HTTP header. So this way we are able, enabling the developer to say these are the sites that I trust to do content negotiation where I want my images server or my video server to actually be able to do that negotiation for width and height and DPR, but I don't want, I don't want those ad networks doing that and I got control to make that decision of what I trust because the opportunity is not just about providing the right content, but also using other details about the environment that might give a hint of how do we get content to the eyeballs as fast as possible. So if you're using say network RTT or just looking at congestion window, these are very low level TCP moments that allow the server perhaps to recognize that hey, you're in a Wi-Fi, but you're in a hotel Wi-Fi and it's not the greatest experience. It's really slow. So perhaps a server can then say, hey, I wanna give you actually a half DPR image. So the pixels are gonna be half the volume of pixels. So I can send you across the wire a much smaller file so I know it's gonna get there and I'll let the browser do that upsizing for you. So yeah, clients can help you in that regard, help that content negotiation back and forth. Images also play an important role in accessibility and according to the HTTP Archive and Lighthouse also, only about 50% of mobile images have alt attributes set. So this is a drum we've been beating for a very long time and it's kind of surprising to hear that only half of images are properly set up for this. So why do you think this is still an issue? We started off by talking about, hey, there is this tension between content creators and us developers. The reality is that when I'm building something, the first user is me. And so it comes from my experiences that I start from. And the alt tag is a good example of accessibility. You know, you've got image, source, blah, blah, blah, and then alt. The alt tag is used by anybody using a screen reader. And if you're using a screen reader and you don't have that alt tag, you're gonna have the person with, has visual difficulties. They're literally going to be listening to the URL of that string, right? Terrible. Now the alt tag gives that person the opportunity to know that this is a dress with sunflowers on it. This is information that I care about when I'm looking at it. But the challenge is that I as a developer, well, I'm fortunate enough to have good sighted vision. So I don't suffer from this. So it's hard for me to remember to do, to add the alt tags of the description attributes for the rest, and so unless you've got that direct tangible experience, it's easy to forget. So fortunately we've got things like Lighthouse, they have the accessibility scores in there that help remind us, but it's a challenge. Now there's actually some cool stuff. There's a Canary version of Chrome that's got, that has machine learning generating some of that old alt tags. I think we're seeing some really cool stuff coming out on that angle to help with the accessibility, but even a machine learning based system that's gonna generate tag, you'll say that it's a dinosaur on a skateboard. Well, is that really what it is? It's still best to have that content for you. It's a good safety net, but developers should still be providing that alt text if they can. Absolutely. And so when we talk about images and video, there's actually a whole spectrum of that experience, right? So it's not just about the alt text, like why we're doing that all, because we wanna feel warm and fuzzy inside. Well, no, we're trying to do that for accessibility. And actually it also helps with SEO and discoverability. So there should be a lot of people championing for this. But there's, you also should be thinking about the experience as a whole, because we're dealing with eyes that are making decisions. It's our lizard brain, if you will, right? Like we've got these cones and rods that go straight to our visual cortex in the back of our head.  So we're creating visual content and experiences on the web to attract you, to get that hook, right? To get above the noise of the web. But our eyes aren't all the same, right? Some people have, you know, colorblind, right? They have, we're taught in school that we've got three cones. Some people only have two cones, or have two and the third cones are a little bit less, right? So that's where you get the gradients of red, green, color confusion. But there's also a percentage of people, particularly women that have four cones that can actually see more colors. Now the interesting bit about this is that all of our physiology research is all, particularly around vision, is all based in the 1940s, 50s, 60s. And it's predominantly dudes that did all this research. And depending on who you're looking at, there's different people that will say, well, maybe only 12% of women have this fourth chroma. Some will say that there's up to 50% of women. So we have this other aspect here of visual experiences that are different based on some possibly gender. And there's just no research really to really say how much that really makes a difference. But it does make a difference in when we make some of our decisions. So I said, you know, use 80% quality factor in JPEG. I take that back because that's almost a terrible choice because if I'm saying quality 80, I'm assuming some editorial control.  And so I can very easily make a poor choice by saying quality 80 that will actually wash out those reds that a person who has better apparatus than I have would actually be able to see the difference between those reds and say, no, no, no, that's not a good red. And if you look at, it's actually, there's very few textile websites out there that offer feedback. There's a few out there that do. Lululema is an example where if you look at the comments, the most common recurring comment is that I love these clothing but the color wasn't what I thought it was. Or, and it's always about the color. It's not exactly what I expected it to be. And there's this downstream impact, the returning products and things like that. But we expect that the image to be a representative reality. But if we've got people making decisions and saying quality that waters down those colors, then that experience could be diminished. So we have to be conscious of it. So the fortunate thing is that we also have a lot of algorithms now that can help us define the right combination of chroma subsampling or whether it be quality to equality factors and decide how do we preserve that experience using structural similarity SSIM based algorithms that try to make it more of an equation based and more math driven that look at the experience. And the emerging ones are also looking at the colors as a part of that input so that you can have confidence that I guess I want to reduce bites but I wanna make sure that it meets the expectations of the content creator. So the accessibility is one aspect of it but also the emotional experience. We're trying to grab that user and bring them in and get that experience consistent. That's such a great example about the shopping cart thing because a user may look at a color in an image and not expect to see it come arriving at their house a totally separate color. So the way that developers actually fine tune their image quality doesn't just save bites on a wire in dollars and egress costs but it actually could incur costs in return fees and bad reviews and there's a real monetary cost to that. Absolutely and I would love to be able to do some research on this as one of these areas I think we haven't explored enough is like when we, when the real world so many times we walk down the street and we walk by this fruit cart and also we stop and we're like, oh yeah, actually I would like to have that apple. It's really, I like to make that choice. I like to buy that apple and I was actually hungry. Yeah, I didn't know about it but I was hungry. We're making these decisions because of that lizard brain, right? Seeing the environment. When we have the right combination of colors and experiences, those decisions now alert are, so if I have the web without images, it'd be very fast but we probably wouldn't see anybody actually engaging in your content because the hook isn't there. And our content creation teams have spent their entire career learning how to get that hook, that emotional, that matches reality. I wanna ask you about WebP. It's been around since 2010 but I'm ashamed to admit I haven't quite adapted it yet. So what do you think is the current advice you would give to somebody about WebP? Should I be using it? It's been around since 2010 or so. It's a great format that has hit the web in the absence of other formats getting large traction. And as I mentioned earlier, there's a lot of different competing formats out there. But WebP is supported by Chrome predominantly and more recently now we've got Firefox and Edge that also support Chrome. I'm sorry, I'm gonna support WebP. Now, WebP is trying to learn from the past. It'll learn the techniques that we've developed over the last 20 years and be able to apply new algorithms that can save more bytes. And so it's a really good mechanism. I've got data that suggests that with JPEG, we've got techniques and with, you know, say Moz JPEG we can save another 10% of bytes. With WebP, depending on the size of the image, we can save anywhere from 10% up to 30% bytes. And it does depend, right? So small images actually, we say you see that 30% byte savings as you get to a larger image it gets watered down because it's about 10%. But over what you could do with JPEG. So that's some significant byte savings across the wire. If you're constrained on that network, cellular network, then getting a visual hook in front of the person, that's a great win. But it comes with some cons as well, right? So JPEG's been around for a long time, as I said. It's really fast decoder. So when we're dealing with WebP we have to deal with modern browsers.  The reality is that not everybody does. You know, I still see a lot of Lotus Notes traffic. The people getting web content in the browser and it's from Lotus Notes or it's from an old version of Internet Explorer or old versions of Firefox. The internet's weird. There's a lot of stuff out there. So the one part is adoption. The second challenge is, of course, the use cases.  So we've learned those lessons from the JPEG days. But we are also constrained in two big ways. One is it's forcing Chroma sub-sampling. That's what I was mentioning about the technique that we've learned from the video industry of the Luma and the Chroma and be able to, reduce an amount of Chroma, you're hacking the eye, you're removing detail but you're able to get away with it. But what WebP does is it forces a 420 Chroma sub-sampling which means that there are many cases where it's not a good choice to use WebP. For instance, in Asian markets, in most times you'll have an image of whatever commerce that you're selling and you'll have the lexographical characters bonded to that image over top, maybe the red outline. Well, those hard lines are very problematic for Chroma sub-sampling, they become blurry and they're not great when you have that blurry effect. So anytime you have hard lines or illustrations, these are cases where WebP just falls down and doesn't give us the flexibility that we need. And the other gap is color palette, color space. So in the web, we've built the web based on 1996 standards around S-R-G-B, if we looked at the amount of colors that the typical, I'd say typical quote unquote, three-coned human can experience, if we plot those reddish, greenish and bluish cones, we get this, you probably see it as a horseshoe shape, the XYZ color space, all possible colors. Now it's not two-dimensional, it's actually a three-dimensional group, but you get the idea. S-R-G-B was a standard established in the late 90s that said we can represent these sets of colors, the triangle of colors, and we can represent that on the web and in LCD displays and so forth. So all of the web is based around this S-R-G-B limits to eight bits per channel, so you get this 24-bit color space that is really a fraction of the full colors that you can actually see. So with JPEG, there's some hacks you can do to kind of get over that if you can beg, borrow the pixels in the index, but you're still limited, with WebP you're limited to this S-R-G-B. We need to move beyond that and to be able to support more colors because if we're trying to get that hook of that user, we need to represent reality and S-R-G-B is just a small percentage of that overall. Now we've got devices if you've got a modern Android device or a modern iPhone, you've got devices that can do P3. P3 is an even wider set of colors both in the overall sets of color but also the discrete colors between and if you've got a 4K TV at home, you've got probably Rec 2020 with high DPR, which is even larger. We're closing in on what we could do with analog film back in the 70s of the colors that we can reproduce. Still not 100% of reality. So we're talking about formats, these are some of the gaps. We've solved some and WebP is great and available in a lot of browsers but we have these other upcoming challenges that we need to also solve. So yeah, it's, the internet is weird. I always say that too. So you had mentioned that WebP supports animations and that brings me back to the days of like those blinking under construction gifts that you'd see on the web everywhere. So we've obviously come a long way but what is the state of animations today and what would you recommend for people who wanna use them? Yeah, so animations is one of the spots where it feels like it's been stagnant for a long time but surprisingly it's not died out. Everybody assumed like animate gifts are terrible. Just put it out there. Like they're terrible because they're large for a very short clip of say a TV show and you have the sequence of that that you can be rocking up megabytes of data really quickly but they're really good at expressing meaning and intent. That's why they're so popular in social media platforms because you're able to express an idea really quickly and you've got the, it's like a video but it's muted, it loops, it's got a whole different creative aesthetic. But gifts are really the only option out there. Now we've done a lot of hacks to get you higher color palettes. It's not just 256, be able to get the effectiveness of a larger color palette even though it's not there but we still have now there's challenge of there's still really large images. With WebP, like learning from the past obviously brought in the ability to support animation. Say that 1.7 megabyte animated GIF in a WebP is maybe about 300K. That's pretty awesome. But you also, that only supports inside the Chrome environment. Last year Apple and WebKit in the Safari 12 brought in support for image source equals MP4. So you can actually ship an MP4 to an image tag and get an animated GIF experience where you can have a HevC or an H264 payload inside that in MP4. It loads like an image tag but it's muted and it's looped. So then you can take that, the advantage is there if you've got animations that are more picture based because you know, HevC and H264, those are really good codecs for real life. For video, movies, TVs, that kind of stuff. So if you've got animations that have that kind of content then you've got a great vehicle that brings that 300K and WebP down to maybe 50K in HevC. In the contrast though you still have these gaps of pallet based animations if you're trying to illustrate the ionic bonds for textbooks and things like that. Those cases actually, GIFs might be still among the better and WebP are good for that. With these video codecs they become a little bit more blurry. So maybe not as good. So there's still lots of room for us to figure out how to best deal with these different composites. Just one little quick anecdote though. It's not just those young kids with their animated GIFs. Talking with a number of, even the ad tech industry. They're trying to figure out how to get past people putting on ad blockers because well it's a crappy experience. And up the game and the quality of the ads so that they're relevant and people aren't less like in. And there's emerging research showing that yeah, animated GIFs are actually really good for conversions but they can't be too noisy. Like too noisy really disturbs people. But cinematographs, the subtle movements create great engagement for the ad tech. In fact it's working great in even commerce. You probably see those like the girl with the dress moving the dress so you can see the color, the light reflecting a little bit. So I think as we can start solving some of these technology bits we're gonna see the content creation kind of explode looking for animations. We're talking about animation as an image tag. The age old of course recommendation is to use video tags. If you've got an animation just convert it to MP4 and use a video tag. I mean that's probably actually the best way to get the mix of all worlds. You don't have to depend on Safari just to get the image as an MP4 in an image tag. So if you can change your markup, change your templates to use video, that's great. The challenge is that not everybody can modify the markup, right? If you've got a CMS system or if you've got some other platform then you're gonna have to, sometimes you have to live with the tools you have. So this is where these, there's different strategies we can play. If you can play with the markup, then great, use the video, make it looping and muted. But if you are bound by the content or it's gonna be six months a year before you can get the next code release out there that can do that conversion. You still have lots of other tools available to you. One more HD3 archive stat that you've probably heard. About two thirds of the average webpage is made up of image bytes. So practically speaking, if developers do optimize their images, what types of impact on the user experience could they expect to see, maybe web performance wise? Yeah, so this is one of those areas that I say yes but no. We see the stats and we say, where am I gonna get my byte savings to get my user engagement up and how am I gonna get my lighthouse score up? And it's easy to say, pick on the big bully on the street. But it's, yes, there's a large amount of bytes on a webpage that are dedicated to images and video and animations. That's because we're trying to grip that user, engage that user because people don't read. Let's be honest. You wanna see the content. But I think I mentioned this earlier, that images are a low priority resource in the browser. So in the browser's parsing HTML, it's going to go and look for all of the CSS and JavaScript in the preloader. And if it finds images, it will make those requests in the network but it'll put those as a low priority request so that we get the CSS back and we can do the layout. So if you're gonna save bytes on images, most times it's really the JavaScript that's still holding up the train. Unless you've got really tight CSS and JavaScript already, saving bytes on images will not give you the same kind of bang, like chopping 50% of your bytes is not gonna speed up your page 50% because you usually have a lot of this JavaScript. There are, there's a balance, counterbalance though. The counterbalance is that if you are, a lot of times you're handcuffed to the system and solutions you have, saving bytes can actually help in some situations where you have a lot of JavaScript that's being injected and tag managers and things like that where the browser is actually getting confused and getting things out of order. I see lots of cases where the browser is, the preloader finds all of the images first before it finds the CSS and JavaScript, ships those requests, and then realizes that there's more JavaScript to be load and has to wait for those images to come across before it can do the other work.  But there's another challenge, another metric that I think that we often need to look at, which is not just what the raw time to first byte or the time to interactive or the first paint and things like that, but it's the, how long does the user stick around? So if the best practices are around making sure that images are small or byte-wise and that we are making sure our CSS and web fonts and things like that are discoverable and loaded first, but we want to make sure that those images and video are loaded so when the user starts scrolling, that content is there. So the balance we wanna find is deferring image loading and there's a lot of strategies to do that, right? From I mentioned earlier, low quality placeholders, we put a little SDG or inlined text to lazy loading, where we just use pure JavaScript and don't load that image until the user actually scrolls. Those are good strategies to make sure the bytes aren't plugging up the pipe, but you also then wanna counterbalance that with once the layout has happened, once you've got all the JavaScript, you wanna make sure that as the user goes to start scrolling that they're there, you wanna keep them there. You wanna keep the user going through their journey. So you wanna then open up the gate and make sure that those images start to flow in after you've got all the essentials. So this is give and take that you need to adjust, but don't expect the world to move on those core metrics for images and bytes savings. It will depends on your market, but what you will see is by optimizing those bytes is your sessions, your user engagement as they interact, those metrics should start to improve as well. So what is the ideal solution of optimizing the bytes with the quality of images? Is it for people to spend more time understanding all of these different levers on their configuration? Or is it we need to rely more on computers and algorithms to do the right thing? What you said, yes. Yeah, this is the challenge, right? We've got a lot of different formats, a lot of different levers to use and a lot of different contexts. And the good news is that we have a lot of developed industry knowledge to run through that rubric, to evaluate the context of those images. So this is where we are really getting good as an industry, be able to classify, is this a picture or an illustration? Is this got lots of high action movement or is it low action movement? Has this got lots of colors or is this grayscale? And from there, what is the right experience that we're trying to drive? So should we use this knob and this knob? Should we change the quality low and quality high? So yeah, absolutely we should be depending on algorithms because if we're changing our markup, there's a lot of different ways we can be expressing that. And there's almost a hamster wheel effect where there's always another technique that we can apply that haven't thought about before. There's a lot of great tools out there that allow us to just do that automatically. And you can see that, I think Squoosh, there's a nice tool there. You can see how those apply the different check boxes and apply if I wanna change it to Moz JPEG or whatever. You can see the impact, but there's also great tools that will do that for you, that you can apply that in your build chain, apply that into your CMS, et cetera. Not to mention you're not always in control over the images that get loaded on your site. You're not always putting them on your CMS. Sometimes your own users are uploading them to your site. For example, my avatar or something, like you need those algorithms to process images outside of your control. Yeah, and there's a whole actually, other dimension here of like, from a content reputation perspective, from content you maybe don't want to have on your platforms for legal or moral issues. But there's also security concerns. Images and videos and things like that have had histories of other vulnerabilities. So where they become a vehicle for passing on a CVE where a certain decoder if using image magic or something like that could be exploited with this certain kind of payload. So this is yeah, absolutely where the algorithms come into play. Using, depending on algorithms, it would be my number one recommendation for dealing with qualities and format selections. And then as we move into the other domains like adjusting for colors and things like that, that's where you now need to focus your, you should be focusing your attention with your creative teams, yeah. Finally, would you recommend any resources for people to learn more about image optimization? There's a lot of great resources. Addy has a great site images.guide. There's a lot of great content there. I've published a number of different articles around the role of client hints and some of the stats that we see there and the use of animations and Safari. And there's a bunch of, in the web.dev site, there's a ton of great resources about different strategies, different tools that you can put in the build chain to kind of fill that flow. Well Colin, this has been great. Thanks for coming on the show. Thank you. If you'd like to find links to everything we talked about, check out the description below. Thanks for watching. We'll see you next time.


I'm pretty sure that a good amount of subscribers to Super Mario Logan know who Jeffy is, but in case you don't know, Jeffy is a very controversial and infamous character in the SNL fanbase. He's known for having autism, and he annoys Mario and other characters all the time. In my opinion, I think Jeffy is an extremely offensive character due to the fact that it makes fun of people with autism and other mental disorders and makes them look bad.  I mean, the harassment has gotten so bad that one YouTuber shut down his channel to stop the cyberbullying. I can tell that Logan is extremely tired of using Jeffy in almost every video. But due to fear of losing his subscribers and receiving death threats from his fanbase, he just keeps giving into his fanbase which is understandable because I did hear that Logan has anxiety issues. It was pretty obvious when he was filming SNL movie The Watch, where JJ pranked Logan into thinking that he pawned a watch that his grandfather gave him, and Logan responded by physically attacking JJ, who is a minor nevertheless. And there was one time where Logan yelled at Chilly for ripping up a Jeffy decoy. Honestly Logan has really gone off the deep end ever since Jeffy got introduced as a character. Especially because of the fact that his videos have started losing their advertisements and getting age restricted in 2017.  I did think that Logan was going to get rid of Jeffy in the movies that he was planning on making in 2017, but I doubt it's going to be released now. I did however see a new SNL video in my recommendations even though I wasn't subscribed to him personally. The video was called, SNL Movie, Jeffy's Demise. I don't know what tempted me to click on the video but I did anyway. The video started with Mario in the kitchen, making Jeffy's meal while he was mumbling and really which went on a bit too long and should have been cut shorter to avoid making it look like a filler. The scene went on until Mario ran into Chef Pee Pee. Chef Pee Pee asked Mario why he was angry. Mario explains that Jeffy has been a lot more demanding lately and he's tired of it. Chef Pee Pee then asked Mario, Well what do you want me to do about it? Which did get a chuckle out of me. Mario said nothing as he makes his way towards Jeffy and placed his plate of chocolate cake and chicken nuggets in front of him. Jeffy thanked Mario as the latter began to mumble something and really under his breath. Jeffy then started to eat his food messily until it cuts to the former finishing. He then made his way towards the couch, not saying a word as he lies down next to Mario. Jeffy looked horrible. He looked extremely overweight and he was breathing heavily. Mario then stares at Jeffy extremely shocked and gave the latter a lecture about eating healthier. Big lecture Jeffy then started to grunt again in pain and clutch his chest. Mario pretty much didn't do anything but tell Jeffy to stop it like what he usually does. That was when Jeffy then collapsed on the floor. Mario then started to panic and checked Jeffy's pulse. He then realized that Jeffy has stopped breathing. Mario then decided to call the doctor explaining the situation. I was kind of excited to see the Brooklyn guy in this episode. He's one of my favorite characters in SML. The Brooklyn guy then appeared and gived at Mario for calling 911 due to the fact that he knows that it's not an emergency as usual. But Mario explains to the Brooklyn guy about what happened to Jeffy. That was when the Brooklyn guy then checked Jeffy's pulse and tried to use CPR to revive him but there was no success. The Brooklyn guy decided to give Jeffy an x-ray because he thinks that there's something in his heart. The Brooklyn guy then rushed Jeffy to the hospital to give his chest an x-ray.  The Brooklyn guy then tells Mario that he needs to give Jeffy surgery immediately. Mario then started to worry more and is worried about Jeffy's surgery and how much the surgery itself costs. The Brooklyn guy then tells Mario not to worry about the cost just yet as he rushed Jeffy into the surgery room. That was when the Brooklyn guy operated on Jeffy and started cutting into him. What's strange is that the Jeffy puppet that the Brooklyn guy was cutting into was the original plush, which I find pretty jarring because the original maker had a hard time attempting to recreate the original puppet. The Brooklyn guy continued to cut into Jeffy, trying to fix his heart as he hooked Jeffy up to a heart monitor. 30 seconds into the operation, the heart monitor then flatlined. It then cut to Brooklyn guy telling Mario the bad news as the latter begins to sob softly. The scene then cut to Jeffy's funeral and everybody was crying at his funeral. All of them except for Junior, Joseph and Cody. It cut to them having a conversation and what I heard was the most best up and awful thing that I have ever heard in an SNL video. The things that Junior and Joseph said was extremely horrible about Jeffy, making fun of his mental problems, and imitating him and staging a temper tantrum like him. That was when Cody yelled at the two and called them both. Freaking morons. And what Junior said next really made both me and Cody's blood boil. Junior said, Wow.  It really offended me because I'm autistic myself and it really made me want to punch Junior. My desire to punch Junior has been granted when Cody landed a punch on Junior so hard that it knocked him out of his chair. Then Junior grabbed Cody, threw him to the ground and then started to punch him in the face repeatedly, effectively breaking Cody's glasses. Cody then rolled over and proceeded to strangle Junior. This was certainly something I have never seen Cody do ever.  Mario and Joseph proceeded to break up the fight and restrain the two boys from hurting each other more.  Joseph then starts to shake violently and say, You two are insane. I'm out of your views. It then cut to the rest of the SNL characters except for Junior, Joseph and Cody at Jeffy's burial site. Everyone was carrying Jeffy's coffin up a hill and once they got to the top, they proceeded to bury Jeffy's coffin inside the ground. The video then focuses on Jeffy's coffin one last time before burial as the SNL question is seen. What would you do if one of your friends has a heart attack? Best comment wins a free video game. I was pretty shocked at what I saw. I mean the episode was okay for modern SNL standards, but a few scenes should have been toned down for younger audiences to handle, and I think that Junior was wildly out of character since he and Jeffy were supposed to be close friends. But I am pretty glad that Junior got what he deserved and got the hell beaten out of him by Cody, but other than that, it was a very underwhelming video. Despite it being mediocre I tried to download the link but the video said that it was deleted. Despite myself being a little nift about Logan deleting the video because that means I can't show it. But I did realize that if I show it then it will damage Logan's reputation. Maybe that's why Logan hasn't uploaded a video on trying to get rid of Jeffy and I think that it's best if he never does.


It's fun, it's exciting, and more importantly, it's very social. The Cube doesn't act. Live from Midtown Manhattan, The Cube's live coverage of big data NYC, a Silicon Angle Wikibon production, made possible by Hortonworks. We do Hadoop, and when does this go? Hadoop made invincible. And now your co-hosts, John Furrier and Dave Vellante. Hi everybody, we're back. This is Dave Vellante of Wikibon with Jeff Kelly, and we're here live at Big Data NYC. We're at the Warwick Hotel, right across the street from the Hilton on Sixth Ave. We've been here going wall to wall all day. We'll be here tomorrow. We were here last night. So Jeff and I are really excited about this segment. We've been tracking Wendisco for quite some time now. Brett Rudenstein is the product guy for non-stop Hadoop, which is really when Disco's claimed to fame. So Brett, welcome to theCUBE. Good to see you. Thank you very much. All right, so let's get into it. We were talking to some of your colleagues earlier about what it is you guys do. So I think we got that down, but talk about the product specifically that you run. Yeah, so Wendisco's non-stop Hadoop essentially provides active, active replication for name node, which essentially eliminates name node bottlenecking. It eliminates the single point of failure for name node, and it allows clients to essentially round robin their requests to each one of the active name nodes such that you also get a name node load balancing effect as well. So over the wide area network, we allow Hadoop to span geographic locations in a single Hadoop. So what that effectively means is you have one Hadoop cluster, a single cluster ID that spans and number of geographic territories. Okay, so that's a pretty complicated situation for a lot of people in the audience. So talk about name node. What's a name node do? What's the purpose of a name node? Yeah, the name node essentially is, or the purpose of name node is to basically store the file system metadata. So if you think about a standard disk drive, there's a placeholder, if you will, to know where all the files live on that disk drive. Now Hadoop is a distributor of architecture. So the name node basically handles all of the metadata information for where all the files live in the Hadoop cluster. So the name node allows you to get to the data. It knows where the data is.   Is that the official way we put it? That's a technical term. Now so, and the other piece of it is, as you said, Hadoop's a distributed file system, so the data's everywhere, right? It's not. And the name node keeps track of those locations so that a client that makes a request knows it knows where to go get that data from. So would you say active, active? Talk about what that means. Yeah, so, you know, and we look at it from two angles. One is from the name node perspective. So active, active means that a client could hit, in fact a single client will send a request to one name node. That request would be handled by that name node. And then the follow one request might land on another name node. So in a standby situation, all client requests would only go to a single name node. And only in the event of a failure would you switch over or fail over to that standby. In an active, active environment, clients round robin their request to each name node.    With multiple active name nodes because you're not limited to the just one, you're able to continue service and operation in the cluster without any downtime, without any interruption. So in an active, passive situation or what you call the standby situation, if there's a failure, you fail over to that passive name node which then becomes active, which of course takes time. Correct.  So you've got, you know, you've essentially got a virtual single point of control with your solution. Is that right? Yeah, in fact, well, you know, if you really look at our architecture, we call it a share nothing architecture because no one name node acts as that central coordinator. There isn't one machine that creates a leadership role where as you look at some of the other protocols out there, they, even if it's a temporary election to that role, it is a leadership role. And in the event of a failure, you hope that some other machine self-elects in its place and a share nothing architecture with no single point of failure, you always ensure consistency and continuous availability. This would be what Abby Metta calls an AC product that's after cutting as opposed to a BC before cutting. So you guys designed this for Hadoop, obviously. Okay, so take us through the demo. What are we doing?  What we do across a wide area network is we do stand up that single HDFS and it allows you to ingest data in both sites, both the data site A and the data site B or and it could be, you know, a third site or fourth site and so forth. The demonstration that I'm going to do today is going to use some of the standard sample applications that ship with Hadoop. So I'm going to run Terogen and Terrasort in data center one. If we have time, we'll run Terra Validate in data center number two to prove that the data made its way across the WAN. Now, if you look at the screen here. Sorry, it's rubbing, you're saying you could do this through N data centers. Correct, yes. If you look at the screen here, you see a map is sort of a live view of the active data centers and we have two data centers. There's a data center in Northern Virginia and it's data center one. These are running up in Amazon EC2 so this is a live clustered environment. So data center one is in Northern Virginia. Data center number two is in EC2 West in Oregon. So about 3,000 miles difference between the two data centers. So I'm going to drill down on the WAN link here, if you will, and that'll give us an exposed view of what's happening operationally inside the data center. So again, I've got the three name nodes that I talked about, three active name nodes and there's actually three data nodes there as well. Six node cluster in Northern Virginia and then three more data nodes in Oregon and the graphing application that you're seeing is showing the RPC bytes in and the RPC bytes out for each of the active name nodes. So let's go ahead and start our pterogen application. So we're gonna run a little script here and the script, we're gonna put it in a directory called SA. When we run the script, one of the first things that you'll notice on screen is that the name nodes start responding to the request. So you've seen each one of these name nodes respond to the request for client activity. So we're already kind of sort of demonstrating that load balancing effect that you see. Each one of the name nodes is responding to the client requests. One of the other things that you'll start to see is over in data center number two in Oregon, you're seeing these little blue bars appear. What these blue bars are is they're showing you something called foreign blocks. So because we've made Hadoop data center aware, all of the blocks that are being ingested into data center one happen in a synchronous fashion but the foreign blocks move asynchronously so we don't block the client. Now the next thing I'm gonna do is I'm gonna start to... Sorry to interrupt you, I just talked to our director we're having a slight technical problem so why don't we, we're gonna hold on the demo just for a moment. So why don't we jump in just for a second, ask a little bit about, you know, kind of the, let's talk a little bit about the application that this allows you to really run. What are some of the things that, if we can really talk about the business values. So if you're an enterprise and you're looking at the solution, you're looking at to do what is this really gonna allow you to do that maybe you couldn't do before? Yeah, I mean if there's a number of things, let me start with, you know, sort of, you know, contrasting it to some of the, you know, other methodologies that people use today to kind of create or keep data and sync across multiple Hadoops. So the first is that it's two different clusters. You've got two different clusters that you're keeping in sync and usually the two most popular methodologies are one is you use disk CP or something controlling disk CP on the back end to periodically send out that synchronization if you will, but the problems that we see is that disk CP won't copy a file if it's the same size and the same name that it's a predecessor. So how often does that actually happen? Well interestingly enough, if you're doing job transformations, you've got the same data inputs and that data input might be growing but the data that you're creating tends to be, you know, file part 001, file part 002, split points are about the same. So after a period of a couple of months you wind up having your cluster sort of diverge. Now it's a manual operation to have those two clusters or to have these sysadmins basically figure out what diverged. And another thing to sort of mention is disk CP runs as MapReduce. So you're actually using up various cluster resources to use it and then of course comparing when those two data centers start to diverge comparing the two of those can use up all the cluster resources as well essentially leaving you out of service. Not to mention the human. The human element, right? How does a sysadmin actually, describe the process that a sysadmin has to go to? Well a sysadmin essentially has to ensure or they have to check and run sums against each cluster individually and make sure that the files are essentially lining up and when they don't they then need to also involve the user community and say I have two copies of your file. You know, which one do I keep and which one do I throw away? So now the users are totally ticked off, right? Because first of all you've advertised to them that you really don't have your IT act together. Secondly you're asking them to stop selling for, hey, stop selling. Need your help to figure out which one of your files we should keep. So that's a real backlash. And there's a similar problem with doing parallel data ingest. You know, a hiccup in either Hadoop, you know, if you're using Flume or even a load balancer using Flume, Scoop or some technology to kind of keep two different Hadoops in synchronization. Those are problematic as well. So those are just sort of the admin problems. Now when we talk about the use cases that we solve from a use case perspective, one is, you know, failing over to a disaster recovery site. We use that word before active, active. Both sites are active. You can ingest data into both sites. You can run jobs in both sites. So if one data center is down, what is the length of time that you have to go through in order to get that site back up online? And with Wendisco's nonstop Hadoop, it's essentially zero. You can just start running jobs in the data center that remains. Awesome. Okay, now we were talking to Jigain earlier about you just mentioning some other sort of competitive approaches. What else is out there in the marketplace like this? Like a nonstop Hadoop? Yeah. There really isn't anything because no other technology stands up a single HDFS that spans multiple data centers. Okay, so you stand alone in that regard. So how does this differ, for instance, for something like NameNote Federation? So we've heard a little bit about that from some of the other players. Yeah, NameNote Federation is interesting, but what it does is you're effectively separating the namespace. And what that means is you're saying, conceptually anyway, I'm going to serve A through L on this NameNote, all the A through L files, that's not quite how it works, but conceptually, A through L will be served here and M through Z will be served on this NameNote. So if a NameNote fails, you're kind of back in the original position, which is you can't service the part of the NameNote that failed. Again, with the active-active architecture, you're able to maintain continuous availability. You don't really need federation. All right, fantastic. So I think we've got our technical difficulties solved. So we're going to jump back into that demo. Why don't we just, if it's okay with you, could we start at the beginning? Take the demo from the top. Yeah, I think that would be great. So you've got two EC2 instances, right? Different zones, right? Correct. So two EC2 instances, one in Amazon EC2 East and one in EC2 West. So Northern Virginia and Oregon. So what we're going to do is we're going to run some of the standard sample applications. We're going to run TerraGen and TerraSort in data center one. And then to prove that all the data migrated its way between the two data centers, we're going to run TerraValidate over in data center number two. Along the way, what we're going to do is we're going to throw a bunch of various failure scenarios. We'll fail a local NameNote. We'll fail a remote NameNote. And at some point, we'll even crash the entire WAN link. Let's pull a plug or two, yeah. Pull a plug or two and crash the entire WAN link. So let's start that off. I'm going to do a Gen. So this is our little script that runs TerraGen and TerraValidate. We'll put it in directory SA2. So this is just the directory where the output files are going to go into. And again, zooming back in on the internals of this cluster, you see the three NameNodes start to respond to the events.  We see all the NameNodes responding. And of course, over here in data center 2 in Oregon, we see these foreign blocks. So a foreign block is a block that is not yet replicated from the site of origin. In other words, we're putting data in data center 1. Everything is fine there. We asynchronously transmit the foreign blocks so that they don't block the current client while they replicate to data center number 2. Now let's throw some failures at this. I'm going to SSH into data center 1's NameNode, let's pick, I don't know, C. And I'm going to issue a reboot command. So what I've just done is I've rebooted this bottom machine here, and just a moment, the graphing application will update, and you'll see it flatline. Make sure I understand this. So you've done that while you've got data in flight? Data is in flight, correct. Yeah. And so you're feeling a plane in midair. Correct. So now while we still have our job running, I'm going to also kill a NameNode in data center number 2's NameNode C. So effectively what's happened here, let me set over to data center number 2. We'll do data center number 2's NameNode C as well, so it's in the same place in the screen, is while we're running our local ingest, I've also got those blocks that are replicating from data center 1 to data center number 2. I've killed the second one, so this one is down as well, and yet we're still replicating those blocks. So there's been no interruption of service on the client side. So the next thing that we're going to take a look at is that whole self-healing aspect that we talked about. So let's jump back one screen while we're replicating here. And one of the things that you'll see about the graphing application is one of the dots on the screen is turned blue. It's basically indicating that we're still providing service. It would have turned red if there was no service, but we're still providing service, but there's at least one NameNode down. And you see it's the blue dot on each side in both Virginia and Oregon. Yet a MapReduce job is still running. In fact, we're in the sort phase at this particular point in time. And in just a few moments, what you'll see is the first NameNode that I rebooted will come back online, so it'll turn green. What's really happening under the covers, because we have a global sequence of operations, is the NameNode will come up in Safe Mode. Now while it's in Safe Mode, it won't take any service requests. So clients will only work with the other NameNodes that are available. It will learn from the other NameNodes that it's behind in the global sequence. And then once it's completed that global sequence or caught up, it will become an active participant in the cluster. And as you can see from the screen now, each one of these is now back online. If I drill back into it, you'll see that they're servicing client requests again, and each one of those NameNodes is coming online again. And our job has just finished as well. Now we'll throw one more failure at it. I'm gonna run the application one more time.  So I'm gonna break the WAN link between the two environments. So let's go over here. And I'm just gonna close the tunnel between the two by issuing an IPsec stop. So I'm gonna give that another moment. Maybe we give it a chance to replicate some of the blocks, but not all of them. And then we'll shut it down. Okay, great. Raising the bar. On the demo. And we're looking at a live demo here, right? This is a live demo running again in two clusters running up in Amazon, EC2 East region and West region. So about 3,000 miles difference between the two. So let's go ahead and stop the tunnel between the two of those. What you're gonna see now is you'll see each of the NameNodes. It looks like they're down. They're not really down. They're still running in their respective data center, but the client no longer can see it. This application, this graphing application can no longer see it. And for a visual perspective, if I go back to the on-screen map, what you'll see in just a moment, once it comes back in the screen updates, you will see that it turns red, indicating that there's no service between the two. But if you look at our map-reduced job up in the upper right-hand corner, it's still running. It is still completing transactions. And it's queuing up those foreign blocks that need to cross the wide area network. So the last part of the demonstration that I'll do for you here today is I'll bring that tunnel back up. We'll do an IPsec start. In just a moment, we'll see the line turn green, and then we'll continue block replication and have complete synchronicity across the two clusters. Because remember, while it's two data centers, it is a single HDFS across one, or across the land, or when. But Brett, yeah. That's awesome. It's not trivial what you guys just showed, all the tech behind there. But I wonder if you could compare this to other sort of techniques to provide resiliency to data. For instance, I'm thinking about things like bit slicing and people using Reed-Solomon code to essentially, if you lose a slice, or two slices, or three slices, you can recreate the data. How does this compare? Well, this is really a way of doing coordinated synchronization. So it's not like even like a block device synchronization technology like an EMCS, RDF. Those technologies are technically limited. Typically, latency is a big factor in those technologies, and so you only have a land distance of about 200 miles or so, plus or minus. With this technology, it's time-independent, it's when-independent, and it's completely software-based. There's no hardware that's involved in it. As I mentioned earlier in the discussion, it's a share-nothing architecture. There's no central coordination that if that machine is down, you have to wait to elect a new leader. Okay, and you do this through the Paxos algorithm, right? It is a patented implementation of Paxos that is able to cross those geographic boundaries, those geographic distances. Juiced up Paxos, all right, great. Juiced up Paxos, I like that. It strikes me as it must be funded, developed as product, because every day you're saying, well, let's try a new way we can try to break this, throw a new problem at it and stay one step ahead of it. Yeah, I mean, some of the first early-on tests is we would basically reboot name nodes and we'd bring them back, and then someone through, let's try a network partition where A can talk to B and A can talk to C, but B and C can't talk to each other. And that Paxos algorithm, that patented Paxos, handles all those conditions, whereby any network condition that you can throw at it, just short of no availability will keep consistency and availability across the network. Excellent. All right, Brett, we're getting the hook. Thanks very much for the demo, and appreciate you coming on theCUBE and going deep with us. All right, keep it right there, everybody. We'll be back with our next guest. This is theCUBE, we're live from Big Data NYC. Right back.