Overview of the Scalable Video Coding Extension of the H.264/AVC Standard
---
paper_title: Multipoint videoconferencing with scalable video coding

paper_content:

We describe a system for multipoint videoconferencing that offers extremely low end-to-end delay, low cost and complexity, and high scalability, alongside standard features associated with high-end solutions such as rate matching and personal video layout. The system accommodates heterogeneous receivers and networks based on the Internet Protocol and relies on scalable video coding to provide a coded representation of a source video signal at multiple temporal and spatial resolutions as well as quality levels. These are represented by distinct bitstream components which are created at each end-user encoder. Depending on the specific conferencing environment, some or all of these components are transmitted to a Scalable Video Conferencing Server (SVCS). The SVCS redirects these components to one or more recipients depending on, e.g., the available network conditions and user preferences. The scalable aspect of the video coding technique allows the system to adapt to different network conditions, and also accommodates different end-user requirements (e.g., a user may elect to view another user at a high or low spatial resolution). Performance results concerning flexibility, video quality and delay of the system are presented using the Joint Scalable Video Model (JSVM) of the forthcoming SVC (H.264 Annex G) standard, demonstrating that scalable coding outperforms existing state-of-the-art systems and offers the right platform for building next-generation multipoint videoconferencing systems.
---
paper_title: Rate-constrained coder control and comparison of video coding standards

paper_content:

A unified approach to the coder control of video coding standards such as MPEG-2, H.263, MPEG-4, and the draft video coding standard H.264/AVC (advanced video coding) is presented. The performance of the various standards is compared by means of PSNR and subjective testing results. The results indicate that H.264/AVC compliant encoders typically achieve essentially the same reproduction quality as encoders that are compliant with the previous standards while typically requiring 60% or less of the bit rate.
---
paper_title: Motion-compensated 3-D subband coding of video

paper_content:

This paper describes a video coding system based on motion-compensated three-dimensional (3-D) subband/wavelet coding (MC-3DSBC), which can overcome the limits of both 3-D SBC and MC prediction-based coding. In this new system, spatio-temporal subbands are generated by MC temporal analysis and a spatial wavelet transform, and then encoded by 3-D subband-finite state scalar quantization (3DSB-FSSQ). The rate allocation from the GOP level to each class of subbands is optimized by utilizing the structural property of MC-3DSBC that additive superposition approximately holds for both rate and distortion. The proposed video coding system is applied to several test video clips. Its performance exceeds that of both a known MPEG-1 implementation and a similar subband MC predictive coder while maintaining modest computational complexity and memory size.
---
paper_title: MOTION-COMPENSATED HIGHLY SCALABLE VIDEO COMPRESSION USING AN ADAPTIVE 3 D WAVELET TRANSFORM BASED ON LIFTING

paper_content:

This paper proposes a new framework for the construction of motion compensated wavelet transforms, with application to efficient highly scalable video compression. Motion compensated transform techniques, as distinct from motion compensated predictive coding, represent a key tool in the development of highly scalable video compression algorithms. The proposed framework overcomes a variety of limitations exhibited by existing approaches. This new method overcomes the failure of frame warping techniques to preserve perfect reconstruction when tracking complex scene motion. It also overcomes some of the limitations of block displacement methods. Specifically, the lifting framework allows the transform to exploit inter-frame redundancy without any dependence on the model selected for estimating and representing motion. A preliminary implementation of the proposed approach was tested in the context of a scalable video compression system, yielding PSNR performance competitive with other results reported in the literature.
---
paper_title: New Standardized Extensions of MPEG4-AVC/H.264 for Professional-Quality Video Applications

paper_content:

To support high quality video applications, the Joint Video Team (JVT) has recently added five new profiles, two new supplemental enhancement information (SEI) messages, and two new extended gamut color space indicators to the MPEG4-AVC/H.264 video coding standard. The new profiles include substantial feature enhancements for high-quality video applications, including improved-efficiency 4:4:4 video format coding, improved-efficiency lossless macroblock coding, coding 4:4:4 video pictures using three separately-coded color planes, and support of bit depths up to 14 bits per sample. The new features were developed to support a wide range of applications where high quality video compression is demanded, including professional and semi-professional scenarios in particular. They also anticipate the introduction of higher fidelity displays. In this paper, the new extensions are presented along with quantitaive estimates of the benefits of the new features and a discussion o the target aplication environments.
---
paper_title: Motion-compensated 3-D subband coding of video

paper_content:

This paper describes a video coding system based on motion-compensated three-dimensional (3-D) subband/wavelet coding (MC-3DSBC), which can overcome the limits of both 3-D SBC and MC prediction-based coding. In this new system, spatio-temporal subbands are generated by MC temporal analysis and a spatial wavelet transform, and then encoded by 3-D subband-finite state scalar quantization (3DSB-FSSQ). The rate allocation from the GOP level to each class of subbands is optimized by utilizing the structural property of MC-3DSBC that additive superposition approximately holds for both rate and distortion. The proposed video coding system is applied to several test video clips. Its performance exceeds that of both a known MPEG-1 implementation and a similar subband MC predictive coder while maintaining modest computational complexity and memory size.
---
paper_title: Rate-constrained coder control and comparison of video coding standards

paper_content:

A unified approach to the coder control of video coding standards such as MPEG-2, H.263, MPEG-4, and the draft video coding standard H.264/AVC (advanced video coding) is presented. The performance of the various standards is compared by means of PSNR and subjective testing results. The results indicate that H.264/AVC compliant encoders typically achieve essentially the same reproduction quality as encoders that are compliant with the previous standards while typically requiring 60% or less of the bit rate.
---
paper_title: A locally optimal design algorithm for block-based multi-hypothesis motion-compensated prediction

paper_content:

Multi-hypothesis motion-compensated prediction extends traditional motion-compensated prediction used in video coding schemes. Known algorithms for block-based multi-hypothesis motion-compensated prediction are, for example, overlapped block motion compensation (OBMC) and bidirectionally predicted frames (B-frames). This paper presents a generalization of these algorithms in a rate-distortion framework. All blocks which are available for prediction are called hypotheses. Further, we explicitly distinguish between the search space and the superposition of hypotheses. Hypotheses are selected from a search space and their spatio-temporal positions are transmitted by means of spatio-temporal displacement codewords. Constant predictor coefficients are used to combine linearly hypotheses of a multi-hypothesis. The presented design algorithm provides an estimation criterion for optimal multi-hypotheses, a rule for optimal displacement codes, and a condition for optimal predictor coefficients. Statistically dependent hypotheses of a multi-hypothesis are determined by an iterative algorithm. Experimental results show that Increasing the number of hypotheses from 1 to 8 provides prediction gains up to 3 dB in prediction error.
---
paper_title: Bit allocation for dependent quantization with applications to multiresolution and MPEG video coders.

paper_content:

We address the problem of efficient bit allocation in a dependent coding environment. While optimal bit allocation for independently coded signal blocks has been studied in the literature, we extend these techniques to the more general temporally and spatially dependent coding scenarios. Of particular interest are the topical MPEG video coder and multiresolution coders. Our approach uses an operational rate-distortion (R-D) framework for arbitrary quantizer sets. We show how a certain monotonicity property of the dependent R-D curves can be exploited in formulating fast ways to obtain optimal and near-optimal solutions. We illustrate the application of this property in specifying intelligent pruning conditions to eliminate suboptimal operating points for the MPEG allocation problem, for which we also point out fast nearly-optimal heuristics. Additionally, we formulate an efficient allocation strategy for multiresolution coders, using the spatial pyramid coder as an example. We then extend this analysis to a spatio-temporal 3-D pyramidal coding scheme. We tackle the compatibility problem of optimizing full-resolution quality while simultaneously catering to subresolution bit rate or quality constraints. We show how to obtain fast solutions that provide nearly optimal (typically within 0.3 dB) full resolution quality while providing much better performance for the subresolution layer (typically 2-3 dB better than the full-resolution optimal solution).
---
paper_title: Analysis of Hierarchical B Pictures and MCTF

paper_content:

In this paper, an investigation of H.264/MPEG4-AVC conforming coding with hierarchical B pictures is presented. We analyze the coding delay and memory requirements, describe details of an improved encoder control, and compare the coding efficiency for different coding delays. Additionally, the coding efficiency of hierarchical B picture coding is compared to that of MCTF-based coding by using identical coding structures and a similar degree of encoder optimization. Our simulation results turned out that in comparison to the widely used IBBP...structure coding gains of more than 1 dB can be achieved at the expense of an increased coding delay. Further experiments have shown that the coding efficiency gains obtained by using the additional update steps in MCTF coding are generally smaller than the losses resulting from the required open-loop encoder control.
---
paper_title: Interlaced Coding in SVC

paper_content:

The scalable extension of AVC scalable video coding (SVC) is a current standardization project of the joint video team (JVT) of the ITU-T video coding experts group (VCEG) and the ISO/IEC moving picture experts group (MPEG). SVC has been initially designed for progressive video. However, even if progressive material is becoming the favorite format for production, broadcasting and consumers equipments, interlaced material is still widely used in the video world and will not disappear in the next few years. The specification of SVC should be finalized in 2007. Thanks to its AVC-based design and its AVC base-layer compatibility, migration from AVC to SVC equipments can be considered in a near future. In this context, interlaced support becomes an important requirement for SVC. This paper presents the main concepts for supporting interlaced coding in SVC. After a brief description of the basics of SVC, the generalizations of AVC interlaced tools are first described. Then main issues related to interlaced video scalable encoding are identified and the new mechanisms introduced in the SVC specification for raising these issues are presented. The paper also discusses related applications side and identifies several use cases illustrating the interest of interlaced support in SVC.
---
paper_title: Constrained inter-layer prediction for single-loop decoding in spatial scalability

paper_content:

The scalability extension of H.264/AVC uses an oversampled pyramid representation for spatial scalability, where for each spatial resolution a separate motion compensation or MCTF loop is deployed. When the reconstructed signal at a lower resolution is used to predict the next higher resolution, the motion compensation or MCTF loops including the deblocking filter operations of both resolutions have to be executed. This imposes a large complexity burden on the decoding of the higher resolution signals, especially when multiple spatial layers are utilized. In this paper, we investigate the approach to only allow prediction between spatial layers for parts of the lower resolution pictures that are intra-coded in order to avoid decoding that requires multiple motion compensation or MCTF loops. Experimental results evaluate the effectiveness of the proposed approach.
---
paper_title: Interlaced Coding in SVC

paper_content:

The scalable extension of AVC scalable video coding (SVC) is a current standardization project of the joint video team (JVT) of the ITU-T video coding experts group (VCEG) and the ISO/IEC moving picture experts group (MPEG). SVC has been initially designed for progressive video. However, even if progressive material is becoming the favorite format for production, broadcasting and consumers equipments, interlaced material is still widely used in the video world and will not disappear in the next few years. The specification of SVC should be finalized in 2007. Thanks to its AVC-based design and its AVC base-layer compatibility, migration from AVC to SVC equipments can be considered in a near future. In this context, interlaced support becomes an important requirement for SVC. This paper presents the main concepts for supporting interlaced coding in SVC. After a brief description of the basics of SVC, the generalizations of AVC interlaced tools are first described. Then main issues related to interlaced video scalable encoding are identified and the new mechanisms introduced in the SVC specification for raising these issues are presented. The paper also discusses related applications side and identifies several use cases illustrating the interest of interlaced support in SVC.
---
paper_title: Constrained inter-layer prediction for single-loop decoding in spatial scalability

paper_content:

The scalability extension of H.264/AVC uses an oversampled pyramid representation for spatial scalability, where for each spatial resolution a separate motion compensation or MCTF loop is deployed. When the reconstructed signal at a lower resolution is used to predict the next higher resolution, the motion compensation or MCTF loops including the deblocking filter operations of both resolutions have to be executed. This imposes a large complexity burden on the decoding of the higher resolution signals, especially when multiple spatial layers are utilized. In this paper, we investigate the approach to only allow prediction between spatial layers for parts of the lower resolution pictures that are intra-coded in order to avoid decoding that requires multiple motion compensation or MCTF loops. Experimental results evaluate the effectiveness of the proposed approach.
---
paper_title: Rate-constrained coder control and comparison of video coding standards

paper_content:

A unified approach to the coder control of video coding standards such as MPEG-2, H.263, MPEG-4, and the draft video coding standard H.264/AVC (advanced video coding) is presented. The performance of the various standards is compared by means of PSNR and subjective testing results. The results indicate that H.264/AVC compliant encoders typically achieve essentially the same reproduction quality as encoders that are compliant with the previous standards while typically requiring 60% or less of the bit rate.
---
paper_title: R-D Optimized Multi-Layer Encoder Control for SVC

paper_content:

The scalable video coding (SVC) extension of H.264/AVC was recently standardized. The encoder control that is described in the joint scalable video model for SVC specifies a bottom-up process in which first the base layer and then the enhancement layer is encoded. The base layer is encoded conforming to H.264/AVC without considering its impact on enhancement layers, which limits the achievable enhancement layer coding efficiency. The losses relative to single-layer H.264/AVC coding are unevenly distributed between base and enhancement layers. In this paper, we present a multi-layer encoder control for SVC, by which we can trade off coding efficiency for base and enhancement layers and generally improve the efficiency of spatial and fidelity scalable coding.
---
paper_title: Advances in Scalable Video Coding

paper_content:

Scalable video coding is attractive due to the capability of reconstructing lower resolution or lower quality signals from partial bit streams. This allows for simple solutions in adaptation to network and terminal capabilities. Different modalities of scalability are specified by video coding standards like MPEG-2 and MPEG-4. This paper gives a short overview over these techniques and analyzes in more detail the encoder/decoder drift problem, which is the major reason why scalable coding has been significantly less efficient than single-layer coding in most of these implementations. Only recently, new scalable video coding technology has evolved, which seems to close the gap of compression performance compared to state of the art single-layer video coding. New methods of efficient enhancement layer prediction were developed to improve traditional (motion-compensated hybrid) scalable coders, providing more flexible compromises on the drift problem. As a new technology trend, motion-compensated spatiotemporal wavelet coding has matured which entirely discards the drift and allows most flexible combinations of spatial, temporal, and signal-to-noise ratio (SNR) scalability with fine granularity over a broad range of data rates.
---
paper_title: Performance Analysis of SVC

paper_content:

This paper provides a performance analysis of the scalable video coding (SVC) extension of H.264/AVC. A short overview presenting the main functionalities of SVC is given and main issues in encoder control and bit stream extraction are outlined. Some aspects of rate-distortion optimization in the context of SVC are discussed and strategies for derivation of optimized configurations relative to the investigated scalability scenarios are presented. Based on these methods, rate-distortion results for several SVC configurations are presented and compared to rate-distortion optimized H.264/AVC single layer coding. For reference, a comparison to rate-distortion optimized MPEG-4 visual (advanced simple profile) coding results is provided. The results show that the performance gap between single layer coding and scalable video coding can be very small and that SVC clearly outperforms previous video coding technology such as MPEG-4 ASP.
---

Title: Overview of the Scalable Video Coding Extension of the H.264/AVC Standard

Section 1: INTRODUCTION
Description 1: Introduce advancements in video coding technology, discuss traditional and modern video transmission and storage systems, and introduce Scalable Video Coding (SVC).

Section 2: TYPES OF SCALABILITY, APPLICATIONS, AND REQUIREMENTS
Description 2: Explain the different types of scalability (temporal, spatial, quality), discuss representative applications of SVC, and outline essential requirements for efficient SVC.

Section 3: HISTORY OF SVC
Description 3: Provide a historical overview of SVC, the development process, and related research and standardization activities.

Section 4: H.264/AVC BASICS
Description 4: Review the basic design concepts of H.264/AVC, covering the Video Coding Layer (VCL) and Network Abstraction Layer (NAL).

Section 5: BASIC CONCEPTS FOR EXTENDING H.264/AVC TOWARDS AN SVC STANDARD
Description 5: Detail the concepts and necessary adjustments for extending H.264/AVC to support scalable video coding, including temporal, spatial, and quality scalability.

Section 6: SVC HIGH-LEVEL DESIGN
Description 6: Summarize the high-level design of SVC, including combined scalability types, system interface, bit stream switching, and profile specifications.

Section 7: CONCLUSION
Description 7: Conclude the study by summarizing the key extensions introduced in SVC, their impact on coding efficiency, and the overall improvements in rate-distortion performance.

Section 8: APPENDIX TEST SEQUENCES
Description 8: Provide detailed information on the test sequences used for simulations in the paper, including their specifications and classification.