This is a general overview of the different and audio video standards for computers, movies, Blu-Ray players, and so on. It will not delve into how each standard works.

The first thing you must know is that the real world is not digital, but analog. A synonym for analog is infinite and for digital is finite. What this means is that we must convert the analog real world to a format our electronics can process. This is where a codec comes into play. Codec is short for coder/decoder. A codec is an algorithm that converts the analog real world into groups of 1’s and 0’s. Newer codecs do a better job of this conversion and thus have a smaller file size or are closer to the real world equivalent.

The second thing you must know is the difference between lossless and lossy. Like the names suggest, lossless does not discard any data in the algorithm. The quality is therefore higher but the file size is higher too. Lossy algorithms discard data to achieve a smaller file size. Ideally a lossy algorithm would appear the same as a lossless one.

Video codecs

There is a group called the Motion Pictures Expert Group, of MPEG for short. They created one of the first codecs which was called MPEG, but now called MPEG-1. It converted both audio and video. The video part is called H.261.

Being the first means there is room for improvement. MPEG-2 is the successor to MPEG-1. The entire standard was subdivided into 3 parts. The first part dealt with necessary overhead in using the standard. The second part dealt with the video and it is called H.262. The third part dealt with audio, the MP3 standard. The video standard part is usually just called MPEG-2. This is the standard used with DVD’s and HD broadcasts.

There was a MPEG-3 proposal, but it wasn’t very good and so abandoned.

The current version is MPEG-4. The video codec standard used in this is called H.264 or AVC — Advanced Video Coding. The MPEG group decided to start using more friendly names. So all video codec standards now have both an official name an acronym name. Any video encoded in H.264 is about 40% smaller than H.262. This is one of the two standards used on Blu-Ray discs. MPEG-4 also has an audio part developed with Dolby Labs, AAC — Advanced Audio Coding.

The MPEG-4 standard allows for different video codec standards. The H.264/AVC standard is fine for regular HD material, but when it comes to 4K UHD material, the files are too large. The successor standard is H.265 or HEVC — High Efficiency Video Coding. Any video encoded in H.265 is about 40% smaller than H.264. This is the standard used for 4K ultra-HD Blu-Ray and likely for the upcoming 4K broadcasts. Because of Blu-Ray support, if you do any 4K video editing, you will want to use this standard.

The successor to H.265 is H.266 or VCC — Versatile Video Coding. The standard is finalized but it will be a long time before anything comes of it.

Microsoft also had a video and audio standard. Windows Media Video is about as good as H.262/MPEG-2 above. It didn’t get much traction.

The successor to Windows Media Video was a standard called VC-1 — Video Coding 1. This is the second standard used on regular Blu-Ray. It is as good as H.264/AVC. Microsoft grew bored of competing with others, and so stopped updating this standard.

All the above standards have or had patent royalties to pay. That could be a big problem for a company like Google with their millions of videos on YouTube. So Google funds and supports a group called On2, they created audio and video standards that are royalty free. On2 created several video codecs, VP1 to VP7. These have patents, but that is done to prevent others from patenting their ideas and charging royalties. VP8 is as good as H.262/MPEG-2.

Google bought On2 to ensure that nobody would charge royalties. With Google’s money, this standard was improved. VP8 came out and it is as good as H.264/AVC. Later VP9 was released. This is as a little better than H.264/AVC but not quite as good as H.265/HEVC. VP9 is the video standard YouTube uses.

Later on, Google, Amazon, Apple, Microsoft, and many other companies formed a consortium called Alliance for Open Media, or AOMedia. This whole purpose is to make royalty-free video codec standards. Their first standard is AV1 — AOMedia Video 1. AV1 is as good as H.265/HEVC.

Audio codecs

There are many audio codecs, some are meant for movies, some for music, and some are for both. Some are meant for voice recordings and digital cellular phones.

Uncompressed audio is simply called WAVE. This is what CD’s use.

The audio codecs meant for music are MP3, as described above; Advanced Audio Coding (AAC), popularized by the iPod; Apple Lossless; Windows Media Audio (WMA), which has a lossy and lossless version; Free Lossless Audio Coding (FLAC), a royalty free codec; and Ogg Vorbis, another royalty free codec. AAC and Ogg Vorbis are both better than MP3. Ogg Vorbis is sometimes used with Google’s VP9 standard. FLAC is supported on higher end stereos but not by Apple.

The ones meant for movies are more numerous. AAC and FLAC can be used for movies, but they are not as good as the others.

Dolby Digital or Audio Coding 3 (AC-3) is the standard used in DVD’s and HDTV broadcasts. DTS – Digital Theater Sound — is also used on DVD’s and it is a higher quality than Dolby Digital.

For Blu-Ray, the audio quality is improved. There is Dolby Digital+ and DTS-HD. These are both lossy formats. Many movies now support Dolby TrueHD and DTS-MA, both lossless formats.

The newest standard is Dolby ATMOS. This standard attempts to better simulate 3D sound than the other standards.

Picture codecs

And finally, there are a few picture codecs to consider.

The most popular is the JPEG, or JPG. This standard is maintained by the Joint Pictures Expert Group. The JPEG is ideal for pictures but not good for drawings or illustrations. JPEG is a lossy format and it uses many tricks to reduce file size. The result is that edges are not crisp and sharp, irrelevant for pictures but very relevant for illustrations. There have been improvements to the JPEG standard, but these never caught on.

A new picture codec that is being promoted by Apple is HEIF – High Efficiency Image File. This is based on the H.265/HEVC video codec. There isn’t much support for it right now, but Apple devices save pictures in this format. If you have a photo on your iPhone, you might have to email it to someone else before they can view it. The iPhone converts HEIF pictures to regular JPEG pictures in the email app. It is probably only a matter of time before this replaced JPEG because the files are smaller and it is royalty free. What is probably preventing this standard from large adoption is Photoshop does not support it, and that doesn’t look like it will change anytime soon. Another roadblock is neither Google nor Mozilla support it in their browsers.

Google also has a picture codec called WebP. This is based on the VP9 video codec. This standard isn’t very popular.

The most popular standard for illustrations is PNG — Portable Network Graphics. This is a lossless codec. When using Photoshop or similar programs, people might use PNG until they are done, and then they will convert to JPEG to share with others.

Container files

The tie all this together, you need a file that contains the video and synchronized audio data. These are container files. You need to know this if you ever want to play your videos with a USB device and a smart TV or Blu-Ray player. The most popular are Matroska (MKV), MPEG-4 (MP4), and MPEG-2 TS (M2TS or TS ). M2TS is the container used for Blu-Ray discs. MP4 is limited to either H.264/AVC or H.265/HEVC video and AAC audio. But MKV is most popular among content creators because it is the most versatile. As of mid-2020, MKV now officially supports Dolby Vision HDR and Samsung’s alternative, HDR10+.

Above are two pages from a Samsung Blu-Ray player that detail what videos files it will play. It is grouped by the two left-most column, the file extension of the container file. The sub-rows to the right must be within in container on the right. For instance, suppose you have a movie in the MKV container. The video data is in H.265/HEVC at 1920×1080 and the audio data is Dolby Digital (AC-3). This Samsung Blu-Ray player will play the movie. However, suppose you have a different movie. It also in the MKV container but the video is Google’s VP9 instead of H.265/HEVC. This player will not play that movie.

This information is vital if you ever want to play your own recordings with your devices without making a disc. Or you could buy a program that makes Blu-Ray movies.