Welcome back to Computer Science E1. My name is David Maylan. This is lecture seven, our lecture in multimedia. And what better way to start such a lecture than with a snippet of multimedia itself. I turn our attention to the above. Hello. My name is David Maylan. And I'm the instructor for Computer Science E1. Understanding computers and the internet at Harvard University's Extension School. You're watching one of our videos of the week. For more such videos or information about this course, visit us on the web at computerscience1.org. Enjoy the show. Well, as you probably realize, this was an excerpt from the teaching fellows volumes of videos of the week. So if you haven't been following along from home already, do remember that per the syllabus, in addition to these lectures of videos and in the future, these videos of lecture and in the future of some sections and workshops, each week the teaching fellows have been doing a remarkable job along with Chris Thayer, a former E1 student, in producing, scripting, shooting, editing, and delivering these videos to you, the students, via both the podcast that we've been distributing as well as via the course's website itself. In fact, if you haven't drawn your attention to such already, surf on over sometime this weekend to the course's website. You'll notice that the front of the website has recently changed, which is something we'll come back to shortly. But you now have two means of accessing the course's content, essentially the old-fashioned way of the worldwide web on the left here and then the new-fangled way on iTunes U, iTunes University, which again is a topic we'll come back to. If we do forge ahead to the course's website here, immediately at left, you will see that not only in CVS style, similarly as E1's website now sporting a Thanksgiving theme, how many of you notice not only the Halloween motif, but some things fluttering around past few days? All right, so good, a little implicit test if who's been checking out the course's website. Next time you do, go on down to Videos of the Week, and what you will see is an increasingly large set of these videos. In fact, the Max versus PC's one that you saw was way back from Volume 2, so if that's the first you've heard or seen that, do realize that you're a bit behind on some of this great content. And by no means are these videos required. They haven't been integrated per se to the problem sets, but what they are meant to be are supplements to these lectures and particularly more focused than these lectures are. Realize, of course, tonight is about multimedia, a fairly broad topic. Our first two lectures were about hardware. Well, in contrast to those broad topics, these videos of the week focus in on very narrow topics in maybe three to ten minutes maximum, and we sort of think of them as bite-sized pieces of knowledge or instruction, where I can't imagine when any of you feels up to sitting down to two hours of this on your computer or your iPod, but certainly these delivered by the teaching fellows is something you can sneak a peek at during commercials, perhaps right after dinner or the like. So do check those out. You can watch them in two different ways. Not only can you download the QuickTime version, and we will define later tonight what QuickTime is, but you can also watch them right in your browser. And I think you've seen me do this once or twice, or you may have done this yourself once or twice, but if you simply follow the flash version of the link, what you will actually get is these videos embedded right in your web browser, and for the most part you won't need it. Hello, my name is David Maylan, and I'm the instructor for Computer Science E1. In fact, I do love the Max vs. PC video. It takes sort of a strange turn I think toward the end there, but even I was smiling by the end, so that is the work of your teaching fellows and former E1 student. Recall a couple of weeks ago, we passed around these surveys, which really just asked you for some candid feedback. It was useful and we did process and absorb that and act on some of the suggestions, particularly one related to the course's website, a theme that came up two or three times in these surveys, was that even though there's a lot of information on the course's website, there's apparently a real lot of information on the course's website, such that a few of you at least have felt a little overwhelmed perhaps on first visit to the website, and even though we tried to categorize things nicely and alphabetically via the menu at left, and by design, nothing on the website is meant to be more than two clicks away. Choose a menu, choose the content. That's been the design. We admit that the menu, if and nothing else, has been somewhat long, increasingly long, so we actually trimmed that down by a few options, merged some things, and generally just eliminated stuff that probably wasn't very much in demand. If you didn't notice a thing, great. If you did, haven't visited the website recently, do check it out, especially if you are among those who felt a bit overwhelmed. It should be a little more simple, and there'll be more content ripped out in the future as we focus in on certain aspects of the course's website, those strange Google ads that you seem to see at the bottom of the website some time, more on that in a week or two. So two comments that I did want to relay verbatim since they were quite memorable, if nothing else. So one of the questions we're calling these surveys was what do you think of lectures specifically? And this answer was, I have yet to nod off. So we appreciate that. That is the bar that we set for ourselves here. One other comment was fun too. What do you think of lectures specifically? I enjoy free food and candy. Both of them, very much about lectures apparently. There is no candy today, but we can't have everything all of the time, but all that stuff is on sale now, I'm sure, at CBS. So with that said, let's forge ahead with our lecture on multimedia, and let's start perhaps with the easiest of questions, or the most obvious of questions, which is multimedia. What is it? Without looking at the yellow cheat sheets. Multimedia, what is it? Stuff that's not text. Okay, I'll take that. Let's see if we can elaborate a bit more. That's fair, though. Stuff that's not text. What else is multimedia? Oh, an excellent definition, better than ours, perhaps, on the yellow sheet there. So it's a composition of audio and video and maybe text, but certainly text in a more animated way, a more dynamic way. You saw an example of a moment of multimedia in the form of video, which obviously also contained audio. What else specifically might you describe as multimedia? What kinds of files? What kinds of content? Would you say, hey, that's a piece of multimedia? Yeah. Like a jpeg. Or sound itself. We'll hone in on each of those topics more narrowly in a bit. When else in this course have we seen examples of multimedia? What comes to mind? Yeah. Videos, be more specific. What have we seen? Yeah, good. So PowerPoint slides. We can chalk those up as a form of multimedia, especially if you've seen among the more stylistically designed PowerPoints that perhaps have less content and more swishes and zooms and fades and page tears. You can do a lot with PowerPoint, including audio and video. We tend to use them just for the static content up here. Anything else? Skype. And Google Earth could be said to be a form of multimedia in that it's sort of interactive, it's graphical, perhaps in the future you'll be able to do even more than that. But for the most part, there's no steadfast definition that we're looking for here. Media is the name implies multiple media, audio, video, animations, text, but in a more interesting way than a simple web page and so forth. So let's make this more real, especially since you probably see some form of Maldives high media every day. In fact, at work this morning or this afternoon, give me an example of a form of multimedia that you had on display in front of you. Anything at all? Yeah. Okay, it's different shockwave files. SWF files, shockwave files will come to an example of one of those in a bit. What else? What did you have on your screen? What did you do today at work using your computer that you could say, yeah, you know what? That was an instance of multimedia. You got an evite? Then okay, multimedia, sure, because those things sometimes maybe have a little animation to them, perhaps. Well, what about a simple web page? How many of you, by a quick and easy show of hands, pulled up a web page today? All right, so odds are you were not looking at the most boring of web pages, but there was at least some graphic in that web page. So why don't we start here perhaps with the most obvious, the most common form of multimedia, the graphics that you interact with, say, on the worldwide web. What are these graphic, what is the format in which these graphics come? Pull up a web page, you see text, you see images, maybe on CNN site, maybe on Google site. In what format, so to speak, are those images? Digital photos, and let's be even more technical. I heard JPEGs, GIFs. All right, so let's start with those and see if by the end of tonight you can't tell a bit to your family or friends about each of these formats, which in and of itself isn't a particularly useful skill to be able to say the GIF format supports 256 colors including interlaced frames as well as animation, but rather as you begin to progress and do things more technical yourself, whether it's in this course, playing around with Photoshop as you will in due time, or developing graphics, or just generally talking about something that you're seeing on the screen or describing a problem to someone in a sort of text support form, well let's just see how specific you can get in detail. So a GIF is perhaps one of the most common file formats on the web. What do I mean by file format? Well it's just the sort of definition of what it means to have an extension of .gif. So if you see a file like david.gif, well that appears to be then a file in the GIF file format. Just to contrast this with something more familiar, if you have something like resume.doc doc, well the file format in which your resume is in is what? Document or more specifically Microsoft Word's file format. So when we talk about file formats, we're pretty much talking about some kind of standard arrangement of zeros and ones inside that file who define this standard? Well usually it's a person or a group of people or a company. For instance Microsoft decided long ago that the .doc file format will be laid out in a certain way. And it takes Microsoft software or compatible software to understand that what at the end of the day are just zeros and ones arranged on your disk in the form of a .doc file, well they have some special meaning and you have to read them in a certain order. Similarly in the world of multimedia is a GIF just another file. A file on your hard drive composing lots of zeros and ones, bits, but they're arranged in patterns that sort of define what it means to be a file format. So let's make this a little more, a little less abstract.  Just tell me something about a GIF. Ah, okay, close. So there were some key components that you just said that are correct but not quite. So a GIF is lossless. Let's come back to that, let's come back to that in a moment. But a GIF is just a file format made up of zeros and ones but it mod and it's a graphical file format to be clear. This is a file format for storing graphics on a disk. Well pretty much any image, any graphic these days on a computer is laid out as a rectangle. A rectangle of pixels. So a typical image might be for instance 100 pixels tall and maybe let's say 200 pixels wide. Well what's a pixel? Who's seen a pixel today? It's a measurement, specifically it's one of those tiny dots on your screen. Ideally you are not aware that your screen is composed of pixels but if you put your eyes really close tonight, if you haven't noticed this before, put your eyes really close to your laptop or even your desktop's monitor and you should be able to see that your screen is actually made up of hundreds or thousands of tiny little squares, dots, a.k.a. pixels. In fact let's just relate this to our hardware lecture when we have our old fashioned computer here with a monitor on top of it. What were some of the common resolutions did we say for a monitor? What's a common resolution for a monitor? Anything come back to you? Or as of exam one is that part of the semester over? Give me a resolution. Anyone at all? A common resolution. What is resolution? Well that's just the number of pixels across and the number of pixels down. So what's a common resolution? Well something like 1024 across by 768 down. Something like 800 by 600. Something even like 1600 by 1200. For those of you who have tackled the current problem set and have gone to a computer store for instance and have looked at the specs of your LCD display or your CRT monitor well you probably saw some mention of the default or perhaps the maximum resolution possible on that screen. So all this is saying is that if your resolution of your monitor is 1024 by 768 when you look at your screen you are literally seeing 1024 pixels across times 768 downward. In fact if you want to put this into perspective or even play around on your own computer at home on Windows if you go to your display control panel go to settings and I may have pulled this up long ago under screen resolution. You see exactly this. I can pull the slider and the next tick down changes it to 800 by 600. I wonder who that could be. Hello? Yes just one moment please. It's the pizza guy. So no candy tonight though. So if I pull this slider to the left or to the right that would effectively change my resolution. Unfortunately this doesn't make for a great demo in class because these projectors are really designed to only work well with one resolution but if any of you have ever connected a computer to a meeting at work or some presentation at school and things aren't quite working right with your monitor and maybe when you plug it in the thing goes blank even though you can see things here and you've hit the requisite keys to make things go there well sometimes though not always that's simply because your computer is trying to display a resolution that the projector doesn't support. That's one possible scenario. I'm going to leave things alone because it will mess up our projection if I change it now but again the takeaway is that you have a finite amount of screen real estate when you're looking at your monitor. And the relevance then to the world of multimedia is that multimedia specifically graphics and videos also have resolutions. And if the resolution of your GIF or your movie is bigger than that of your screen well obviously that's going to not be a pleasant viewing experience since you're trying to look at more pixels than you actually have room for on your display. Moreover, graphics are typically laid out in rectangles. So this graphic might be 200 pixels dots across, 100 pixels up and down but each of those pixels meanwhile can be of some different color. In other words we have sort of a mapping of bits if you will. What ultimately comprises a graphic these days is just a whole bunch of dots and each of those dots has a color and the number of colors possible for each of these file formats differs and that's what ultimately defines GIFs and other file formats. How many possible colors are there for the GIF file format? In other words, each dot in a GIF how many different colors can it be? All right, good. So that was one of the things I rattled off earlier. GIFs are capable of let's say 256 colors they are also capable of animation and they are also capable of transparency. Well what does this mean? Well have you ever visited on the web the most hideous of websites where little things are dancing around and things are twirling and it's clearly made by someone who was a bit overzealous when it came to the development of their site? Well you might have seen things like and I'm just googling right now for animated GIFs and I'm pulling up a site that if all goes well let's say food and drinks category barbecue and picnics and if for some reason you would like a running puppy on your website the only file format that you really have it's your choice is the GIF file format for the reason that it allows you to have animation. Now take a guess how do animated GIFs work? What do you think is going on that gives it that animation? Good, so actually it's not a group of JPEGs going together each frame has to meet the same requirements but do you remember as a kid perhaps those little flip books, the faux comic books that if you flip through them really quickly you get the illusion of animation because each page represented like a frame of a video and if you flash them through your eyes quickly enough you get the illusion of animation well that's precisely how that dog is working someone has drawn maybe four or ten separate images in each one the puppy's legs are in a slightly different position so when you play them again and again in a loop you get the illusion of animation. What do I mean by transparency? Well it would be nice if you wanted to put that puppy onto your web page for instance and your web page has a blue or a green background representing grass or representing say the sky well it's things would start to look pretty ugly if the requirement for having animation on your page is that your animation has to be in the form of a rectangle in other words if you had a blue sky and green grass and to put this puppy on the grass you had to put a white rectangle with the puppy running inside that rectangle sort of defeats the purpose of having some nice multimedia in the first place so what's nice about GIFs and in your section on photoshop this coming week you'll get to experiment with this if you so choose GIFs also support transparency which is to say you can specify that a certain part of the rectangle that your image ultimately is is actually supposed to be transparent and whatever else is on say the web page should be allowed to shine through and this is in contrast with other file formats what is perhaps the second or the just as popular file format on the world wide web jpeg so a number of you rattled that off earlier what is jpeg good for photograph so how many of you have a digital camera for instance so for the most part when you have exported those photos to your hard drive and then maybe uploaded them to flicker or to kodak gallery dot com or other such sites odds are you are uploading or at some point you're converting your photos to the jpeg file format the reason being jpeg support millions of colors contrast that with 256 it's hard to capture the spirit and the color of an arbitrary photo of people and landscape and so forth if you only have a palette of 256 colors at your disposal jpegs by contrast then allow you to have millions of colors but they don't give you animation they don't give you transparency so it's a sort of trade off yeah a good question so does making the background of this dog image transparent does it perhaps affect the background that's behind the dog if perhaps the dog itself has some white in him you don't want blue sky or green grass shining through the dog is that the idea so the way long story short that animated gifs essentially work is you specify you essentially could fill the background of that dog with a random color like pink that simply does not appear in the rest of the image and then essentially you tell the file format you know what everything that's actually pink don't show pink make that transparent so in that way anything that's not pink is going to be opaque and nothing is going to be allowed to shine through from the web pages background so that's a good question and the nice thing is and you'll get to play again with something called Adobe Photoshop if you attend this coming weeks section is that software makes these processes fairly easy you will making images these days is not like drawing a dot for every pixel you know 200 times in one direction 100 times in the other direction well back to jpegs for a moment and the relevance to today's cameras those of you with digital cameras how good is your camera interpret that question as you will what are the specs of your camera ah so 7 point something megapixels alright how about the rest of you are your cameras also rated in megapixels okay so you've all got megapixels maybe slightly less than 7 of them what does that mean why did you buy all those megapixels okay helps with the resolution it or rather it is the resolution in effect but that's true why in more technical terms do you want more resolution to your photos why do you want more megapixels sharper images more depth you can enlarge it that's exactly right so think of it this way if you took a photograph of something like the grand canyon and your image your camera only supported photos of 200 pixels by 100 pixels you could take the photo and from a distance you could probably get a sense that this is a picture albeit not very compelling of the grand canyon but suppose then you wanted to print that photo and you've only got 200 dots this way 100 dots that way it's hard to imagine even intuitively stretching those 200 dots and those 100 dots to fill like a 5 by 7 photo 4 by 6 photo what would happen in fact if you took an image that in real terms is only this big on film so to speak and you try to blow it up to this size what does it look like it's very blurry or maybe splotchy where all those little square dots are now really big square dots and that might look fine if you're looking at the photo like this but you start to look up close and the thing just looks ridiculous and think about the analog by the way to these graphical formats in your daily newspaper if you pull up the newspaper on Sunday from the Boston Globe for instance and look at here's your excuse to regress for a bit pull up the Sunday comics those comic strips often come in color on Sunday but what do you notice when you look really close at colorful comic strips for the most part comic strips to this day are colored in the form of dots from a distance looks fine because your eye doesn't really distinguish that there's dozens or hundreds of dots there you look at it up close it doesn't really look so good and that's sort of an analog to zooming in on the image and trying to print it at a larger resolution than it was meant to so if you have a camera that only supports 200 pixels by 100 pixels I apologize you can't buy cameras of this size when the first cameras became popular digital cameras became popular a few years ago you'd get maybe a 1 megapixel camera I in fact still have one that's five years old it's 2.1 megapixels but what does that mean if my camera is 2.1 megapixels what does that mean it means what I just said so yes it means you can print them bigger you can zoom in you have better quality but now related to this like 2.1 megapixels well that seems to be unidimensional one dimensional it's the area so when the marketing when the canon companies of the world and sony says that this is a 2.1 megapixel camera that essentially means if you multiply the resolutions with by the resolutions height what you get is roughly 2.1 million if instead we use Eric's camera which is much newer clearly than mine he has seven some odd megapixels that means that he's presumably got way more horizontal pixels way more vertical pixels and the beautiful thing about that is that you just have sort of a bigger canvas on which to draw that photograph of the Grand Canyon you can crop out portions and if you have enough of these dots well even if you just take a snippet of it maybe that's enough resolution enough dots to go to press with them from like KodakGallery.com Peek and perhaps zoom in on just part of it too and crop away parts he's not interested in interested in so in short if you're in the market for a digital camera or perhaps finally realize oh that's why I paid for the more expensive digital camera what you get is one more quality but also sort of more flexibility more discretion for me if I want to go to press with my photographs I pretty much have to frame it the way I want it because I don't have the luxury of opening that photo later on my computer opening a program called Photoshop tweaking things zooming in cropping out because I just don't have enough dots enough paint to play with and to manipulate yeah question them back the image when you have more megapixels or the image is literally larger there are more pixels going this way and more pixels going that way no you're simply getting rather than so for instance suppose I frame Ray here with my camera and I'm using the real cheap camera with just the Ray the only guy eating the pizza minds you right now we'll do that in a minute or so so when if I just frame Ray with my really cheap camera and essentially I'm shooting Ray and let's say Dan and Eugenia here well 200 pixels across do I get 100 pixels up and down do I get well Ray let's say just fills the bottom corner of the photo the way I framed it so think of it as I only now have one Ray's trying to get out of the photo think of this as meaning that I only have because of the way I've framed my lens one pixel to devote to Ray and the other to 199 this way and the other 99 this way are being used to capture Dan and Eugenia in part of the wall well that's if I have really low resolution so if I then look at that photo after taking it on my computer Ray is in effect going to be represented as just one reddish dot and that's it but now suppose that I go with a more expensive camera same lens I stand in the same place I hold the camera in the same way now because I have higher resolution I get to now squeeze they're not shrunk but conceptually I get to squeeze more megapixels into between my left hand and my right hand both horizontally and vertically which means now in this bottom corner rather than just having one dot you know I have maybe a few thousand they're smaller in this context because I've taken my image and said you spend them this way but now I have a few hundred or a few thousand dots so that I can capture more of what's going on here with Ray I can see the dots on his shirt and the remaining slice of pizza on his plate and so forth but even though I'm framing it this way and shrinking them conceptually with my hands when you go to press it's like printing a photo that's this big you're just spending them and you're zooming in effectively and spending those pixels in a more effective way yeah good question is it possible then to compress the image in certain places more than others in theory yes right if I'm not that interested in getting Ray in this photo but I'm trying to be nice keeping him in frame but you know what spend most of the pixels on Dan and Eugenia let's just spend that one red dot on Ray in theory yes but this is what a file format does for you the JPEG file format beyond just being a sort of standard for how the zeros and ones are laid out it also specifies how you compress an image what you do to compress an image we'll come back to this in a bit but the short answer is that yes that's possible you can compress different parts of the image in different ways but you the photographer tend to have discretion over that's a function of your camera and its implementation of say the JPEG standard I don't think I can hold you off any longer why don't we take a five minute break here there is six cheese pizzas and four pepperoni pizzas less to slices in back along with some plates feel free to bring them to your seats and we'll resume in a few minutes alright we are back so lest you be wondering why we just served pizza this is a thank you of sorts from Harvard Institute for Learning and Retirement recall that our past this past Sunday one of our E1 sponsored workshops invited a number of you and as well as the staff and I to work with some of the members of Harvard's Institute for Learning and Retirement and we had a great time helping some of those folks to learn how to master the internet two students among you Don and Ray were kind enough to join us as well as Chris from last year so this pizza is a thanks of sort not only for their participation but also just from ILR for their appreciation of of how the day went so thank ILR maybe a few years from now if you some of you choose to join them after E1 so just to put this into perspective here's an example of a shot not of the Grand Canyon but let's say of a nice mountain and some trees well suppose that this image is in the GIF file format and it's a few pixels wide by a few pixels tall maybe and maybe like 100 pixels wide by 300 pixels tall give or take well if this isn't the GIF file format that means that it is a bit mapped file format which as we said before means that it this image is made up of tiny little dot called again what called pixels the problem though if you represent an image like this of photograph here using a GIF file format or even the JPEG file format because the JPEG file format to is a bit mapped file format it's good for photos because each dot each pixel in a JPEG can be any number of millions of colors which means you can get much nicer gradients whereas GIF only has 256 to work with but both file formats are bit mapped which means they're comprised of composed of pixels the problem though to put this into perspective then is if you try to zoom in on a bit mapped file format be it GIF or JPEG what you get is the effect there what you see on the slide above you is a zooming in of the top of one of the peaks of that mountain because this image is composed of dot or pixels and you only have a finite number of them you can't zoom in forever right the you know how they in law and order and CSI when the security camera has taken photographs of like the bad guy fleeing the scene or of the license plate and you know the cops will say can you enhance that can you zoom in on that no like you cannot just zoom in forever just to improve the quality of the license plate that was captured across the street from a camera dozens of meters away right if a camera is by definition just storing its photos in a bit mapped file format which pretty much they do whether it's GIF or JPEG or something proprietary for CCTV cameras at the end of the day you only have so much information there and if it's not there to begin with you can't zoom in and capture what information you would like to see so honestly the next time you watch law and order CSI and the computer tech and forensics is enhance that for me like no that is not possible to do ad infinitum which you eventually get as a license plate that looks like that or there for instance is your suspect that tends to be what happens in reality yeah they claim that yes so yes it is certainly true that with software and with intelligent software you can interpolate information and you can kind of smooth out an image and maybe get a better sense of what the license plate is saying by sort of smoothing things out so it's not as blotchy as this but at the end of the day if you only have a fixed amount of information to start with you can't come up with new information and in effect this is what these programs are doing and it's important for court because if you have computer enhanced images computer enhanced is another way of saying the computer added information or change the information that you were given and while that might help in a practical sense in giving you the first two digits of the license place and that might be admissible for instance as an ID of the suspect probably couldn't or shouldn't hold up if things are sufficiently blurry in the first place or a good defense attorney could pick holes in something like that but the first one order one of my favorite shows and CSI aren't so realistic but rather to at least relate it to again those images you've been seeing on your web page well if most of these file formats that you see every day gifs and jpegs are bit mapped well what's the alternative are there file formats in which you can zoom in ad infinitum and get just a sharper and sharper image even of just a small piece of that image well yeah there exist graphical file formats that are based not on just you know horizontal rows and columns of dots but rather vectors now this is just a way of saying that you can implement graphics using mathematical formulae in other words rather than representing a circle of the sun as a circle of dots literally drawn in a bitmap file format like this well wouldn't it be better to take advantage of some of the algebra or geometry you might remember from high school and say well actually I know that the formula for a circle is x squared plus y squared equals r squared the radius well wouldn't it be nice if in your graphic you don't represent circles as dots you literally represent them with formulas as circles well what is this imply if you desire to zoom in now on that sun well if you're representing a circle with a formula all you have to do is plug in bigger numbers to that formula and you'll get bigger but still perfect circle by contrast if you try to zoom in on a sun comprised of dots what you start to get is what we saw a moment ago you get blotchiness so what you have here is an example on the second slide is a wire frame of sorts of a block of some blocks of cheese well there's no mathematical formula to my knowledge for a block of cheese but you could certainly break it down into things like circles triangles or fragments thereof so what this wire frame is suggesting that rather than representing this block of cheese if it's in a vector based file format with just dots rather let's break each of the components of this picture each of the shapes within the picture down into some polygons circles squares rectangles straight lines and so forth and embed that information in our file format so if you want to zoom in all you do is scale things and you scale them in a way that the quality of the object is preserved let's take a look at that in this example a gentleman earlier mentioned the shock wave file format a file format ending in dot s w f this is a file format for animations essentially which means it's not just graphical necessarily it can also have audio embedded in it but it's a file format in which an example like this is stored so here is it's already a little strange here is a bunch of Swedish horses so it is so maybe you've even received this as an internet forward yes no so right now it's very small but arguably it's of good quality it's not blotchy and it looks like it's maybe what 200 pixels by 100 pixels and where am I getting these numbers from well what's my screen resolution now if you recall from before 1024 by 768 which daresay is the most common resolution today so that's debatable as hardware gets better and screens get larger so if I have 1024 across well it looks like it takes up maybe a third of the screen so maybe it's 300 400 pixels wide that's where I'm coming up with these numbers well it looks pretty sharp suppose then I want to zoom in it would be unfortunate if these cute little horses all of a sudden become blotchy and more square like like our mountaintop does but it turns out that they don't in fact this probably looks even sharper now because we're really spreading out how the dots are being spent on the screen or how they're being computed if I zoom in even further making this full screen it's a pretty sharp image in short even if I had a bigger monitor I could keep scaling this again and again and again and it would still the lines the curves the colors would still look just as sharp because these horses and everything around them is being represented somehow or other automatically not with just dots now this in and of itself might be cute wait till you see this how many of you have seen this one two turns out this is an acapella quartet so to put this into technical context shockwave flash is shockwave is a animation file format again supports audio and it also supports interaction so if you've ever been to a particularly sexy website which doesn't just have graphics and text but rather if you click something something swooshes away and maybe something zooms in if you visit a lot of fashion sites like banana republic or gap or if you visit a lot of furniture stores tend to have websites based in shockwave flash or similar technologies they're interactive so I clicked on that horse to get him singing and I'm going to try to time this correctly I got it wrong boom boom boom boom that's not bad okay that's wrong try again not bad so far we have to add the uh prano perhaps it's so simple and yet it's funny that's it that's all this thing does so anyhow we'll make a link this will perhaps be one of the most popular links visited after this lecture but that is an example of a file format that doesn't just use bitmaps it actually uses vectors and by vectors we mean mathematical formula mathematics in some sense such that the visual upsides of that usage are quite clear or hopefully more clear from an example like that well where else do mathematics come into play well this isn't a graphics class and so we won't go into detail as to how the latest and greatest video games are made but certainly if you've looked over your kid's shoulder at the latest play station game or game cube game or even pc game these days these graphics are far beyond the pole position and asteroids and donkey Kong that at least I grew up with which were very much more bitmap types of games well these are three figurines here that perhaps just capture the spirit of how some of today's games are implemented whereby characters are not just implemented like Mario and Luigi were in the first Nintendo is just a bitmap of colors that collectively represented Mario's face and mustache and hat and clothes but rather the represented more clearly as a collection of polygons of mathematical shapes the beauty of that is that if you want to implement a game where the player can go anywhere on the screen and can do different things with his arms and legs and you want to have the actual illusion of movement well it would not be ideal if you had to just to get your player to do this store a graphic that in precisely using pixels that precisely depict the player in this way if instead you wanted your player to be able to lift his hand slightly you don't want to have to recreate an entire image a jiff really or a jpeg just to capture that slight bit of movement in short you don't want to create a sort of flip book of all possible permutations of characters on the screen and physical locations because if nothing else you would need a dvd or more multiple dvds to store all of those darn images certainly at the resolution that people expect on today's games a much more efficient and faster way of storing information then tends to be using mathematics of some sort linear algebra and other things that you've probably happily forgotten from years ago but that at the end of the day help you model things in a way that's much more scalable and so if you want your character to move into some direction that essentially means that your video card or your cpu simply has to perform some mathematics and create on the fly the representation of that character in his new position in short a lot of today's craziest video games are developed in this much more dynamic rather than static way it's fun perhaps rather than playing one of today's latest and greatest games which you yourselves might own what I thought I would do is draw our attention to the courses website there's another under publicized link there perhaps you have stumbled across it late at night when looking for something better to do but if you click on the courses website the link entitled games what you have here is a little archive of free games that are actually implemented in the I think the shockwave flash file format which was developed by macromedias now owned by adobe and you have some examples of games here from yesteryear in fact we seem to have asteroids up top donkey Kong if you're familiar frogger moon patrol pac-man pong it doesn't get any simpler than pong really space invaders and tetris which is a little newer from the audience which was your favorite game yesteryear tetris space and I heard frogger is the favorite game who said frogger can I entice either of you to come down for a moment and show us just how good you are at frogger that's fine would you like to come remember up down left right there aren't 20 buttons in this game like today can I entice anyone to come down here and show us how good you are at frogger yes no come on alright come on down what's your name heather so heather do we get to type oh sorry yesterday's games only supported six character names okay so you're this frog here and you can use the arrow keys I think to move up down left right have you played this before alright ok try once more redeem yourself make us proud once more extra life once more alright once more let's end it with a positive note that's fine you're just getting worse it seems very well done congrats alright so there's plenty of other games there that you can play with later but what are some of the takeaways alright can we spin this as an academic exercise so what was interesting about that game in the context of all this stuff alright it's definitely bit mapped right I made the thing so big on my screen that you could see that everything was the result of putting little squares of color together green for the case of our frog what else did you notice so very flat two-dimensional right there really there's animation but when Heather was hitting the up key that whole gif or whatever it was back in the day was just moving entirely up you know a few pixels it wasn't gradually sliding it's certainly much easier and much more efficient to just have these things jump across the screen and fortunately the frog's moving relatively short distances so it looks like he's just taking one fluid step when reality he's hopping from one step to the other and there's nothing in the middle anything else come to mind well this is good that we had a note of video there let me ask Dan to come up and as we proceed to connect his Macintosh as the so we have the duality as we promised in our introductory video earlier for a moment though there was a question from before about compression and I said we'll come back to that later but the nice thing about these graphical file formats both gif and jpeg is that they are quote-unquote compressed well what does that mean well what does it mean to compress a file make yeah to make it smaller so what are some of the considerations there right like I could make a file really small I could make your resume really small by highly control a and then delete is that compression well I mean it is but it's what we would call compression lossy and this is a technical term lossy in the sense that yeah you're making the file smaller but by throwing away information so it's sort of a corner case in that yeah you can make the file smaller if you get rid of the files contents all together but that's sort of an extreme case of what is in fact done today well what rather do you want to keep in mind when you compress a file what are your concerns then ideally you want to keep all of the information unfortunately that seems like a catch 22 you want to use less information fewer bits to represent the same amount of information that is to say if you don't want to lose information you've got to keep it around well if you want to keep all of your information what could you possibly throw away becomes the question well and actually if you want to sit for a moment rather than stand there awkwardly we'll finish this it's alright the Mac will be back in a moment so how can you do it we'll consider this here is a flag of Germany and a flag of France suppose that I store both of these images as gifs which means again they can only have 256 colors maximally that's fine we only need three for each of them so that's not a constraint they can only be bit mapped therefore so essentially these flags so you can't quite see it are made up of a bunch of little dots left right top down and suppose that the file on top is suppose that it's let's say 300 pixels by 200 pixels well if it's 300 pixels by 200 pixels then what do we have so 300 by 200 what's the total resolution or how many pixels are in this flag all together alright so 6 1 2 3 4 so 60,000 right it's kind of impressive that you go from 300 across 200 down and you get 60,000 pixels but that's how many little dots are making up that say German flag and France flag so the question on the table is can you represent these flags in the same with the same aesthetic effect but using fewer than 60,000 bits and just to put the 60,000 pixels so to be clear each of these flags is made up of 300 dots horizontally 300 dots horizontally 200 dots vertically each of those dots in turn has a color for instance in Germany's flag most of the pixels of all the pixels up top the top third have the color black all the colors in the middle red and then yellow so you need to remember the size of this thing the shape of this thing the colors being used but so what can you throw away can you use fewer than 60,000 pixels to represent Germany's flag true yes is the answer you get the harder one how okay so a 3 by 3 image that would distort 3 by 6 would distort the dimensions too so can you make it smaller you could certainly in this case what do you mean by smaller sure so you could certainly just make the image smaller so that therefore you need fewer bits but I would say you're essentially telling me to delete parts of me right make this file smaller by making make the file size smaller by making your flag appear smaller but my goal is to put this flag on Germany's home page right www.de is where I want to put this thing so I want it that size I want it really big I can't cheat and save space by just making the flag smaller question back suggestion okay good so a slightly more mathematical approach in short and all we can tweak this just to lead us in the factual direction so keep around information for some of the pixels for instance the left most row or left most column of pixels so you've got you know 200 pixels on the on the vertical there so that means you've got like 66 black pixels beneath which are 66 red pixels beneath which are 66 yellow pixels and then plus something else right that gives us 200 so what if we just stored this column and then said you know what multiply this column by 300 what would that do the trick well if you think about it intuitively there's a lot of redundant information in that flag what is redundant about the flag it's just solid bands of color but in effect with jiff or jpeg as we've defined it thus far you're essentially saying if you're storing it as a bitmap make this pixel black make this pixel black and you're doing this 300 times then you're going back here stepping down this way once and doing it 300 more times eventually saying this one is red this one is red then eventually you get down to the yellow this one is yellow good that's essentially the answer you retain as much information as you need and then you multiply it by some factor and this in fact is how jiff works jiff in contrast with our microsoft word proposal earlier is a lossless compressed file format lossless as that word implies when you compress it you do make the file size smaller but the upside is you do not take away from the quality of that image at all well how is that even possible well in effect all the jiff file format needs to remember for germany's flag is that the leftmost column is 66 black pixels 66 red and 66 yellow then you know one extra pixel left over then multiply those by 300 so now in effect i only have to say this pixel is black 66 times i only have to say that this pixel is red 66 times and then the same thing for yellow and then i in fact just shout out repeat 300 times and you can say that just as i could verbally so much more efficiently then you could by just repeating yourself 300 times by 200 times contrast that now with the flag of france at right taking on faith that the jiff file format pretty much works as we just described defines as many pixels as are necessary vertically and then says repeat does france's flag compress using jiff better or worse than germany's flag you say worse why takes 300 instead of 200 what right because the colors are not contiguous throughout you essentially have to say this pixels blue repeat 100 times this pixels white repeat 100 times this pixels red repeat 100 times contrast that now with the story over here this top left pixels black repeat 300 times you're done with that row saying a third as many sentences and we can make this we can confirm as much in tonight's jiff directory which is linked on the course's website here are precisely images if I go to details mode notice how large is the france flag fr large five kilobytes a little small on screen how big is germany's flag three kilobytes so we have a bit of empirical evidence here too that the the mathematics do in fact corroborate the conceptual explanation we gave a second ago yeah correct correct the gift jiff file format is designed to read this left right manner let me pause for a moment let's sit still unless you'd like to grab a piece of pizza and we'll swap out tapes but let me take a question or two offline now so what about pings is an excellent question so we looked at gifs jpegs also on your yellow cheat sheet and also on the worldwide web is this third file format that's not nearly as popular as these two but in a sense is better and it is called the portable network graphic format png is its file extension this too like jpeg supports millions of colors but it is loss less so think of this sort of as an amalgam between what jiff and jpeg offer more over png is not encumbered by various patent issues that the company that invented the jiff file format for a while stomped their feet about people using it without paying royalties to them to output these gifs so png was partly a response to that that particular issue is not so hot anymore so the short of it is that gifs and jpegs remain the most popular file formats today and if you would like even more technical detail on any of these three and actually a really good resource would be just the computer dictionary link on the course's website type in png type in gif type in jpeg and you'll get back some more technical detail from the webopedia site as well how else might you compress information or what are some of the other considerations how about this apple this apple is on a blue background at left how much of that information do you need to keep around well just to reiterate what we considered verbally with the flags you pretty much only have to keep around as much information as is currently in the right hand version of this image right if you're saving this is jiff you've got to remember a column of blue and then in effect the details of this are not so important for our purposes tonight but in effect we'll say you know what anything that's just white fill it in with blue and so with a graphical file format can you use tricks like that as well are we losing any information if the right hand version doesn't represent the visual result of compression but this is depicting how much information we're physically storing in the file can you reconstruct using that much information at right the original image is the question and the short answer using the heuristics we've discussed yes because we've remembered just enough especially in this context if we advance to the level where we can say you know what don't just repeat horizontally go fill in anything that's white with blue that would be sort of a neat trick to employ as well well graphics are one thing we've seen a tad of audio we'll come back to that but video was certainly in vogue these days and as bandwidth speeds increase and people have faster connections to the internet and as disk storage gets even deeper you see sites like google video and youtube the last of which we've pulled up an example a couple weeks ago what video did we watch on youtube yeah it was that hard drive where some guy had opened up his hard drive but still left it in operable condition deleted all those files well that was an example of youtube youtube uses a file format called flash video to store all of their videos what this means for you the user is that the wonderful thing about youtube and say google video today and even e1's website if like before i did you click the flash version of our videos of the week or the lectures you don't need special software because where did that video of the week start playing right on the courses web page right there was no prompt to say do you want to open this or save this there was no new window that came up it was just the courses website with the video embedded therein now that was a bit of a white lie you do need special software to play flash videos you do need special software to play frogger as we did before but according to a company called adobe 97.2% of computers on the internet today are thought according to their marketing folks to have what's called the flash player installed already what you've seen happening these days is when you buy a dell or you buy or you download software from yahoo or google a lot of these companies tend to embed other people's software into your computer for you the result of which is usually kickbacks of some sort or some financial gain for them but the nice thing about this because the browsers have done this as well most of today's browsers if you install IE or you install firefox they too come with support for flash that is to say they come with the plugin quote-unquote that allows you to play those videos inside of them so back in the day when this was not as popular you would decide upon visiting the course's website if you don't have the flash plugin installed you'd get like a broken icon or you'd get a prompt saying you need this plugin do you want to install it these days we've reached a point where so much stuff do you get for free automatically when you buy your computer or download some other program that you have the illusion of everything just working seamlessly but contrast that with for instance quicktime quicktime is the file format in which we played Dan soapbox at the start of this lecture and if you recall we did need a special player for that a new window came up and that was the quicktime player made by apple quicktime player by contrast to flash does not come on every windows pc these days rather if you have wanted to watch download the course's podcast you may have in fact had to yourselves go to apple.com and download itunes or quicktime funny thing is there even apple does it you download itunes get what you guess what you get for free quicktime right so good and bad things about that but for practical purposes it tends to be a useful thing that this stuff just seems to work so why don't we motivate this foray into video by pulling up a video I know what's about to happen so I don't want to be the guy on video or ironically but I need another volunteer who wouldn't mind being not only on video but also on video an ergo video come on anyone at all we go through this every week and then there's always someone that volunteers come on it's a non-speaking role alright come on down dawn alright dawn is going to stand in front of dan's macbook pro here nice fancy macintosh laptop you will see the effects of what we're doing but we will see them even bigger up here I'm gonna have dan use his mac savvy to touch the right buttons this is obviously a web camera of sorts built into dan's laptop you might use this for video conferencing like we did with albeit in one direction with victor of the typical pc user podcast a while back dan is going to use that as an example not only of just multimedia and video but what kind of software exists today and how fast processors are today such that they can render the images of dawn you're about to see in real time there was a time where all of these morphing examples does anyone remember the michael jackson video from five ten years ago where each of the actors in the video morphed and by that I mean sort of blended into one another that as I understood it took weeks if not months to develop on computers because it took so damn long for the computers to generate all of the intermediate pictures what you're about to see is what was now possible in 2006 I give you dawn this is good for anyone who didn't happen to go trick-or-treating I guess let's see here's the fun one so like I said this is good for anyone who didn't get to go trick-or-treating so just here just move around you can make some great face okay wait right there hold it move to the left just a tad okay let's see alright wait wait for it wait wait oh there it is okay it takes a photo okay let's see what else we've got here we have oh fisheye you get really close you can make your nose quite large yeah let's do that oh here's a good one here's a good one bulge so one more photo I guess okay we'll put these on the course's website after for us and we do actually have one of David here that he so kindly took for us I think he was going for the Jay Leno like I'm not quite sure we'll get to privacy and security in a couple weeks I had no idea this photo was being taken when we were experimenting earlier I guess in fairness we'll put that on the course's website so the question was if you can change an artwork in the same way that you're changing these photos and basically all there is is just the camera right here so anything that you can hold up in front of the camera you will change with these different effects so there were color effects in this first where was it there was color effects in this first section here so if we held up for example David's notes that we have we could just hold it up and change the coloring or we could go into the twisting effects and the various other things and we can actually see the different in real time the different effects that we have so yeah it's not really the highest quality basically what you see for the resolution this is about 640 by 480 so it's just probably a shade over 1 megapixel the efficiency of the camera you know this is really disturbing the woman on the overhead okay here we'll make it a little bit better normal okay this program comes with new Macs that have the webcam built in this includes the laptops and some of the desktops as well it's just it's in the applications no you cannot layer effects just one at a time are there questions a big round of applause for our guinea pig here thank you so on this same note you'll notice that distributed tonight is problem set 5 multimedia this is a particularly fun problem set in that it's very hands on and graphically ory where is this you didn't switch it and um graphically oriented among the tasks ahead of you are the following on top of the course's website as of yesterday is a big turkey and you may have noticed for quite a while if you click on the banner at top left on the course's webpage it actually changes automatically for you now we have gobble gobble two turkeys if we click it again we have a a dog dressed as a American and a cat dressed as a pilgrim so I wish I could say this was our handiwork but these are actually images developed by last year's student body in e1 because if you skim the problem set being distributed tonight the first challenge of this problem set is to task you with learning a bit or applying a bit of knowledge about adobe photoshop which though we try to be software and platform independent in this course hands down the de facto standard for graphics design these days is adobe photoshop which is a very expensive program normally but for which there exists a free trial that you can download off of adobe's website we have a link on our own course's website under software and under the multimedia category and it works perfectly for 30 days after which they try to coax you to buying it but with your Harvard ID numbers you can actually use Harvard's version of the software too so long as you're registered in or affiliated with the university so long as you have a network connection to the campus this week's section if you are the sort who does not like dabbling on your own or feels a little overwhelmed by pulling up a new piece of software a la google earth and learning yourself how to manipulate images this week's coming sections will focus in particular on designing JPEGs and pings using adobe photoshop and these will really be hands on tutorials, hand holding tutorials on using software and adobe photoshop and this problem set will invite you to apply that new found knowledge to the design of your own very banner and in fact what we will do after the images are all submitted as part of the problem set is your own work will be exhibited for a number of weeks on the course's website and if you get tired of looking at someone else's image you can always click it and change it to your own and if you don't click the image up top it actually changes automatically using something called cookies which will come to a development lecture to actually remember which one you saw yesterday so that you don't show the same one again moreover as extra credit in this problem set you'll notice specifications toward the back of the shape of a mouse pad so each semester we have a contest of sorts whereby for extra credit any number of students are welcome to submit candidate designs for this semester's fall 2006 mouse pad so long as mouse pad somehow embodies this theme of survival of computer science e1 per the specification we will then enter you in a voting process that will have in a few weeks after this problem set is submitted you the audience will vote on your favorite design at the very last lecture you and as well as our distance students who will take good care of via postal mail will walk home with a souvenir of sorts from e1 as if the candy and pizza weren't enough will give you something unedible to remember the course by and that will be the mouse pad that wins the popular votes and that will be coming up in the form of this problem set 5 let me also draw your attention to the blue handout tonight which is the final project which per the syllabus challenges you to develop your very own presence on the worldwide web funny we should hand this out now given that you have yet to see your website development lecture or content thereof that's okay the first task in the final project the so-called part one tasks you with thinking about what kind of website you want to design and fortunately the only prerequisite for have it going answering this part of the problem is what sites have you visited before what do you like what would you personally like to do literally at the end of this course you will have your own presence on the web something of the form www.davidmalen.com but with your name inserted or you can have it be something completely random like www.myhomepage.com I would wager that both of those actually are already taken and as we mentioned in this part of the project perhaps they're say the hardest part of this project frankly because the rest is all fun this parts the frustrating part finding a domain name that someone or some squatter hasn't already bought but we don't constrain you to only choosing a dot com you can choose a dot net dot org heck you can have you know Dan Armandara's dot tv if you wish I would wager that that one is still available as of now with that said though there will be the cost of buying a domain name for the period of a year is like ten or so dollars and that it's all spelled out there and we would ask that you take care of that part of it of their all alternatives if you'd rather not incur the expense but we E1 the course will host your website for you up through spring semester so we will provide you with an account we will provide you with an email address of the form username at davidmailin.com but substituting your domain name of course so literally at courses end and through may you will have your own presence on the web will provide you with the disk space you'll get a gigabyte of storage especially which is very useful if you want to have say a website of photographs of your recent trip right gone are the days where people come over and watch slide shows of their latest trip you can just put that on the web and email all of your friends those photos this would be a wonderful application for the final project to do something like that really the sky is the limit but more on that in technical detail and our next lecture the first part of this project which is due at the end of November invites you to start thinking about your website and also to choose your domain name because we'll need a week or so to set up on our end all of your website domain so that you have some place to put all of your content when you start writing html yourself so any questions on that just let us know in the meantime any questions on this or another thing yes no yes no alright so let's go back to where we began if we pull up the very first clip from tonight this was this so-called quick time video quick time is a video file format that was developed by apple if you have a macintosh these days you already have support for quick time installed which is to say if you visit some website and they say hey download this video it's in quick time format all you have to do is download that video double click it and it will start playing if by contrast you're on a pc odds are you first have to go to apple.com download the software install it and then you can double click on the dot m o v file which is the file extension for quick time movies and then it too will play this software here that I just pulled up is the so-called quick time player it's the software that apple developed and if you want to play quick time movies you need a software or compatible software when I click play what it starts playing is that video of the week that we began tonight doing but rather than play this again I'm going to go up to it's window menu and choose show movie info and there's a bit of interesting information here it's a little small for you to see but is it big enough to tell me what the resolution of this video is what's the resolution of this video 5 current size that's the current size so that's not so much it's that's not it's native resolution it's built in resolution it's the 320 by 240 so 320 by 240 is the resolution of this thing ignore for tonight's purposes the 356 by 240 but 320 by 240 that should sound somewhat familiar at least if you multiply 2 what do you get if you multiply those dimensions by 2 640 by 480 so we're back to a sort of 4 by 3 roughly 4 by 3 aspect ratio if you do the math there which is to say that these videos that we're distributing are actually pretty small right when we blew up those Swedish horses they filled the screen quite nicely and quite clearly by contrast if I blow up this video the projector doesn't really give you a sense of just how blotchy things get but can you already tell that at this resolution the text becomes a little blurry and if we say fast forward to this well this stuff's not bad it's pretty clear but it's also a bit it's a little we'll pick this one randomly so tell me about this photo critique this photo ignoring the subject but talk to me in technical terms what's interesting about this anything so the outlines of Ray like of his profile yeah so again these overhead projectors don't really do it justice things look much nicer here Ray looks nicer at any resolution but things are a little it's not terribly sharp certainly right the Swedish horses there say look better at least in terms of their sharpness on the screen Ray by contrast you know no sleight on his profile it's just a little blurry it's a little washed out because what file format is probably being used here it is a video but for this individual frame of the movie what's probably being used jpeg or something like it so within videos you effectively have frames literally like you have in a TV show or in a movie right when you had the old real projector a movie is in fact just like a flip book but the book is just so long and has so many frames that it really looks fluid how many frames does a movie or does TV typically show you every second 29.7 29.7 30 give or take usually around there depending on the encoding scheme but that is to say literally when you're watching TV or a movie it's as though you're watching a really long flip book but one that shows you 29 or 30 images within a second and once you start to see that many frames per second your eyes can't even tell that it's 30 different frames because your eyes just can't keep up your brain can't keep up with that so you have the illusion of moving pictures but think about moving pictures what does that mean well they're just pictures that create the appearance of motion or of movement so quick time is one file format developed by apple for video we mentioned another file format for video a little bit ago what was that so flash is another one so quick time and this ends in dot MOV flash video which you don't usually see this file extension because you yourselves don't tend to download these files rather as you saw they play in the browser so these details are usually hidden from you but they're usually dot FLV yeah sorry shockwave we could put it in this category but for now we'll keep it somewhat distinct because dot SWF files the Swedish horses was a file called dot SWF for now let's just distinguish that ever so slightly by saying that's more of an animation format and not put it on this list for now but you could make an argument there what's another video file format you've seen yeah so if you've watched this own courses lectures on the via the courses website you've watched real video whose file extensions there's several tend to end in dot RM for real media real videos were developed by a company called real how many of you have actually installed the real software on your computer so many of you most of you so and presumably all of you who are tuning in from afar at least with the real video versions what do you get in this course with the real video versions sorry so they're bigger at least if you choose that bigger option what else is interesting about the real video version of this course what's going on over on the right side of your screen so you have that split screen effect if you've tuned in that way you have like a list of shortcuts to the 10 minute mark the 20 minute mark little things like that but the slides are synchronized right we've started to move away from using slides and have been moving more toward demonstrations but for the PowerPoint slides they are roughly time synchronized in fact perhaps unbeknownst to you every time we use a PowerPoint presentation here there's a piece of software running on my laptop that writes to a file the exact time at which we change slides each time so after each lecture I actually email that small file to the video production staff and they use it to synchronize those slides with the video which is kind of a cool trick but to play these videos you need special software in fact how many of you now that you have the real player installed seem to get little pop ups in the bottom right of your screen from real telling you what's late in great and news worthy anyone yes no so yes no alright so those of you who do you can get rid of all that stuff real has long taken it upon itself this is just a company a multimedia company taking it upon themselves to put a whole lot of stuff in software that at the end of the day you probably only want to play movies in real video format so for those of you who have PC's I would actually recommend going to our courses website going to the software link and download something called real alternative which needless to say is an alternative and it's sort of like a program that it's the bare bones all it does is play the videos and the audio files none of these pop ups and news and weather and so forth that come with it but the purpose for tonight is that real video is certainly an alternative to these other video formats and clearly using real video or the technology made by the real media company you can do other things synchronization and time stamps and so forth but you can do something else with real video and actually you can do it with all three of these formats when you start playing a real video those videos tend to be about 200 to 400 megabytes each week how quickly do the video start playing for you when you click on them for the most part right away maybe a five second delay maybe a 20 second delay right there's that little icon if you've seen it that sort of says buffering or loading or something well what is going on there well if these videos are 200 to 400 megabytes is your computer simply that fast is it downloading a 200 or 400 megabyte file within five or 10 seconds storing it on your hard drive and then playing it for you to watch probably not right sort of a leading question so what's going on it's I hear that's streaming so these three formats can be quote-unquote streamed so these are all streaming file formats which simply means that you can start playing them before the files are downloaded in their entirety which is to say when a program like real is buffering or when a video is loading even when you go to like Google video or you go to YouTube what those websites do is they start to download just a bit of the video and then once they have a buffer of like five seconds or ten seconds then you get to start seeing the get to start watching the video and as you're watching the video the program is ahead of you by five or ten seconds downloading the next five or ten seconds sometimes though as beautiful as this setup is what happens to you the user what do you see the video it starts jumping or just stops and then you watch some little annoying icon or whatever and then eventually it starts to proceed again well why in the world if you buffer for ten seconds and then start watching and you continue to buffer ahead how could my two hands ever meet so that's true so eventually you will download the entire file but my question for now is why in the world given this seemingly elegant design would we ever experience what many of you have probably experienced which is like the movie pauses for a second or freezes for a moment for several seconds why can sometimes real time catch up to where the file is still being downloaded memory be more specific be more detailed okay okay not bad it's not that type of memory that's so much that issue suggestion back here so yeah I mean intuitively that's got to be what's going on right the movie is playing faster than it's being downloaded which is to say ten seconds of buffering just wasn't enough well at first you only needed to wait ten seconds to download ten seconds of video but you know as well as I do that sometimes the internet just slows down right or there's congestion at routers we already know that because of IP routers can drop packets all together well if the packets get dropped and you're using TCP for instance to transport this information what is TCP guaranteed did we say a couple lectures ago it'll do what well it'll retransmit that data but if you're going to have if you're if that's it is going to be retransmitted well someone's got to wait for it and that someone is you and that's why sometimes these programs would stop there's a difference here though notice that all the videos that we've been discussing for the most part are pre-recorded right we record them either right now and we'll put it online by tomorrow or Friday the videos of the week that the teaching fellows have been shooting well those are produced each week and then they're put online a week later in the entire volume so what if by contrast you're trying to watch a baseball game live or presidential address live or listen to it live well similarly can you tune in these days via the internet to live music or live video broadcast in fact the groovy music that we tend to play at the start of lectures is the result of my pulling up iTunes which is a multifaceted multi-media program today if you've never used it before it's fairly straightforward on the left so link these days to radio and then you see all these different categories and notice what's the column described as here a stream so what is a stream a stream is sort of an available feed of bits that you can tune into and they're streaming in the literal sense they'll start coming to your computer but you're going to tune in wherever that stream currently is and so in effect what these are is online radio stations and if I go to ambiant which one do we usually use oh you usually listen to groove salad here in E1 last night you listen to groove salads it's starting to sound like this but this is a radio station that is streaming itself over the internet those of you who like Boston radio there are a number of radio stations even in Boston where if you don't have access to your car radio your stereo but you are sitting at work perhaps with headphones on on your laptop with an internet connection some of these local radio stations actually allow you to tune in via the internet and they just stream out the information that way an interesting thing to note is that there seems to be three different versions of groove salad here what's the distinction among these three lines if you can see the small print yeah so kilobits per second so good here too is sort of a building block from a few weeks ago what does this mean if the one we're tuned into per the little speaker icon there apparently has a bit rate of 128 kilobits per second finer sound so the more bits per second just intuitively the more information you're getting which means the better it sounds that's true and what does it literally mean if this stream is 128 kilobits per second what is my computer doing each second it's downloading 128 thousand bits every second and with a decent internet connection that's fine if instead I were on a slow internet connection which is hard to simulate since we do have a pretty fast connection here well sometimes bits might get dropped or maybe the server would get busy and it just can't keep up with the demand right now so what might happen well we could buffer again and again but when you're talking about live feeds like a presidential address or a baseball game or really a radio station which by definition streams in real time right it's not like Tivo where you download it and play it back later radio is sort of by nature real time podcasts by contrast are like offline radio to make a simple distinction well what would you rather happen during a baseball game if your whole purpose is in trying to watch this game live albeit over the internet or your whole purpose is to listen to the game live over the internet would you like your software to wait 10 seconds let me buffer and then I'll show you what happened 10 seconds ago right it's a great way for you to lose bets right if this is how you're tuning into the game betting someone else who's watching live TV so by contrast to a lot of the stuff we do in the course which really you do not need to know about computers in the internet right now right it's okay to wait until tomorrow it's okay to wait 10 more seconds 20 more seconds but if you're watching something that by nature is meant to be live like a baseball game or like a radio station or presidential address well you'd probably rather the software do what in those instances stream it yes but streaming just means deliver those bits in real time but what's the problem server gets busy router starts dropping packets so that's a clever idea if there are just if there's just not possible to get all of those bits to your computer what could you sacrifice well you could just turn the thing off right that's a wonderful way to compress the information just stop playing bush's speech right man no comments but what else could you do you could just degrade the quality and in fact we see this here even though it's not very dynamic right if I tune into 128 kilobits per second that's the stream I'm going to get and if you play this radio station long enough you'll see eventually iTunes sometimes pops up a message that literally says I think buffering stream or something to that effect and essentially you just have to wait iTunes does buffer for a few seconds just to give you that sort of buffer literally so what iTunes does is those of you with disc men if you even carry around those ridiculously large CD players anymore well remember how they used to have or still have shock guard or something like that where theoretically you can shake the heck out of the thing for 5 seconds 10 seconds whatever it tells you on the box well what are those things doing right those devices too are buffering the song maybe 5 or 10 seconds worth of the song so that if you're jogging or shaking the heck out of the thing it can withstand that but you can try this at home if you have one of these things or maybe your roommate has one of these things after 11 seconds you start no longer to hear the audio so CD players don't just degrade the quality but radio stations by contrast might tune you down to a lower bitrate let's see if we can see the difference hear the difference so this is 128k good you can really hear the difference this is 24 kilobits per second so in order of magnitude smaller and even though if I had turned this on this way when lecture first began you might still think the song is bad but you might not realize that the stream is of low quality but contrast this again now with ok this song sounds bad at any bit stream really but the point hopefully is clear so to recap streaming technology just lets you watch a video in near real time but if you're watching something that by definition is live you tend not to just get this buffering effect rather the song will degrade in quality or what's one other option for a baseball game annoying as it might be if your goal is to continue watching this game live what's another option the software could do besides just degrading the images or the quality of the announcer at the game drop bits and so what a lot of these streaming protocols and programs do is if they just can't keep up with you you'll just miss that sentence that bush just spoke or you'll miss that save in the game because it's a trade off the designers of the software figure well this guy would probably rather we just forge ahead rather than get out of step with real time other video file formats that you might be familiar with AVI is another one and I'll drop this on the board we won't spend as much time on here you have a whole cheat sheet if you would like on your yellow sheet tonight AVI is another older file format another popular one DivX is popular those of you who deal in online movies or downloads there of might be more familiar with that mpeg in fact how many of you have ever watched an mpeg two movie before two of you three of you anyone else this is always going to be a trick question so how many of you have watched a DVD before BAM all of you have watched an mpeg two video so we've been talking all tonight about file formats well all a DVD is is a piece of plastic that has a bunch of bits pits and lands if you recall stored on it what format are those bits well DVDs use this format and it's just a standard for video and it's pretty high quality and when you start talking about things like Blu-ray and HD DVD and all these things for the most part they're not necessarily just talking about differences in hardware they're also talking about different schemes for encoding the bits what does that mean just means different options just as jiff and jpeg gave us different features so do different formats for video give you different features that at the end of the day aren't as important to you the user because for the most part when you the user pull up a website you don't usually have choice over the format of the video it's either watch this or not watch this though some websites do let you choose the formats including our own but let's pose this one question about video before ending on a quick chat about audio you've got a video clearly these things are big these lectures themselves are how large did we say in terms of videos in terms of bits large good even answer so this five seconds of awkward silence how many bits did we just waste on those tuning in at home so each of these videos which clocks in around two hours long should less break time or about 200 250 megabytes that is compressed that is watching these lectures in 320 by 240 resolution at 32 kilobits per second audio and so funny thing that this music sounds so bad at 24 kilobits well and maybe I do myself sound bad at 32 kilobits but voice does not need as high of a bit rate because I'm just not I mean I'm just not delivering that much information per second so to speak but how do you get away with using it's up to you well how do you get away with using fewer bits to store video well what's the analog to our graphical world will consider these movie frames so suppose that at the top you have sort of a real film and that film has captured the act of some RV driving past a house and a tree well how in the world if you want to show users that same reality that same event can you do it in fewer bits that is to say rather than keeping around a bitmap for every frame can you throw away any of the bits well what does the top reel of film suggest one technique is for compressing video good so just like jiff sort of uses within one frame with when within one image this sort of cheat scene says hey that was blue make the rest of this blue similarly can you do that trick in video whereby you have what's called interframe compression inter like an interstate highway goes within multiple states interframe compression goes within multiple frames well if you already know from the first frame what the house looks like what the tree looks like why do you need to keep that information around again and so what file formats like these tend to do to compress themselves losslessly is just throw away the stuff that we can just repeat from before but there's a lossy approach to to interframe compression what could you also do if you want to save space but still give a pretty good approximation of reality skip frames right the answers right there you can skip frames or drop them together now this doesn't mean drop half the frames and then show the video twice as fast because that wouldn't fly too well but think about the frogger game that we saw before that in effect had skipped frames right at one point in time the frog is here the next moment when Heather clicked the mouse it then went there but we didn't see anything in between so in spirit that's sort of like interframe compression that's just skipping frames you don't know what happened to the frog between here and the eyes and your brain sort of decide he clearly went from here to here besides interframe compression you can also do in trough frame compression inferring simply from these two reels here what is this very much like well it's sort of like the previous example whereby what's changing on the bottom reel for instance from frame one to two to three to four right in this case there's no background yes but what that suggests is that we don't need it we don't need the several thousands or tens of thousands of bits that might otherwise tell us what the leaves colors are and what the petals colors are why because the flower is not moving the bee is moving so in this way could we similarly compress video but in this case are we losing information or is this lossless lossless lossy lossless good question when you play it can you tell there's no frame there it depends partly on how sharp your eye is and if you slow the video down you'll actually see and you can do this with the courses quick time movies just pause it and then drag the little slider and you won't see fluid motion usually you'll see this so yes you can see it depending on the precision with which you can navigate the video so I hesitated to say that this is lossless only because as of this frame do we know what that flower looks like in its entirety right well I mean as a frame one if this is the start of the video just looking at this bottom real technically we don't know what is behind that bee right it probably just if we let our brains fill in the blanks what is it it's another pink leaf and maybe some more yellow to the center there but that's interpolation and we said this earlier what can a computer do to sort of get the CSI effect and zoom in that infinitum to the license plate we'll can sort of interpolate and guess you know this looks like this and this looks like this I'm going to guess that this thing in the middle is like the average of those two things it's a little purple and it's a little yellow and so a video could use a bit of lossy compression and just fill in those holes for us it's ultimately a tradeoff but it's the file formats that decide how best to do this a final note we've looked at audio tonight we've heard a bit of audio consider though the following there are multiple types of file formats for audio what is the one that most of you or many of you have probably downloaded before so mp3s right this is sort of what's on your iPod often though not always Apple has its own format for that mp3s this is what was popularized by Napster if you weren't quite tuned into what was going on or didn't really know much about computers in the internet back in the day of Napster which was like 98 99ish give or take what was all the rage was the sharing illegal perhaps of mp3s mp3s actually as trivia is mpeg 1 layer 3 though that's technical detail that is not terribly important but this is only to say that it's related even to the stuff we've been talking about and mp3 is just a file format for audio but the neat thing is is that it's compressed but it's lossy compressed but turns out human ears not so good which is to say if you play a song from a cd that tends to be for the most part uncompressed though that's a bit of a white lie but if you play an mp3 that's very much compressed consider this and you've answered this question before how big is a typical mp3 file 2 megabytes 6 megabytes 10 maximally at least for reasonably long or short songs well a cd we talked about cds in lectures one in two how big is a typical cd so it's like 700 megabytes give or take well how many songs are on a typical cd that you buy at the store 16 I'm going to do easy math let's say 10 is typical and that's kind of reasonable maybe a little low so if you have 700 megabytes available to you on a plastic disc called the cd and you want to fit 10 songs on it how big does that suggest each song is on a cd you know roughly 70 70 megabytes that's pretty big downloading a 70 megabyte file is not so fast on a lot of internet connections but an mp3 is how big did you say roughly what so let's keep the math simple let's say 7 megabytes just as a rough estimate right so it's 10 the size of the original song so what does that imply how do you achieve a compression ratio of 70 megabytes originally to 7 megabytes so you have so it doesn't sound as good because you presumably have to drop information intuitively why is that the case well if you're not dropping information why was the original song 70 megabytes in the first place what was it just a bunch of zeros probably not but again the human ear not so finely tuned such that mp3 is for the most part are difficult for at least non-crazy audio files to distinguish from say the original cd version I for one can never really tell the difference between a store bought cd and say an mp3 a set of mp3s of the exact same cd even though they're a 10th the size consider this example so again we don't necessarily have the latest and greatest audio hardware here but this is an mp3 of one of Enya's song it still sounds pretty good even in this audio system but it's a 10th of the size that it originally was when I ripped it from its original cd well how about this for contrast suppose that back in the day we lived in a world where if you wanted to play music on your computer what you heard was what's called the midi file format those of you who are musicians and might have digital keyboards at home and compose music yourself typically you're storing the songs you compose in midi format midi essentially midi stores the musical notes a b cd and so forth sharps and flats in some kind of file and then your computer just plays them back literally well that's one thing but if instead you want to actually enjoy the music you're playing you can't just store all of the notes as notes themselves rather you need say the mp3 version or a non midi version that doesn't just represent each sound with an individual note but sort of stores the information in the aggregate and you have the same song this time in a much higher fidelity version now we actually have a recording of actual strings and so forth so your homework for this week realizes one to familiarize yourself more with a bit of multimedia particularly in your first banner and perhaps a mouse pad project and also do take a moment if you haven't already to pull up the course's website and tune in to the videos of the week since those two are not meant to just repeat things that I may have said in lecture but rather are meant to introduce you to topics that we did not cover in lecture that's it we will see you next week