An Overview of Emerging Technologies for High Efficiency 3D Video Coding
---
paper_title: Unsupervised Temporal Segmentation of Repetitive Human Actions Based on Kinematic Modeling and Frequency Analysis

paper_content:

In this paper, we propose a method for temporal segmentation of human repetitive actions based on frequency analysis of kinematic parameters, zero-velocity crossing detection, and adaptive k-means clustering. Since the human motion data may be captured with different modalities which have different temporal sampling rate and accuracy (e.g., Optical motion capture systems vs. Microsoft Kinect), we first apply a generic full-body kinematic model with an unscented Kalman filter to convert the motion data into a unified representation that is robust to noise. Furthermore, we extract the most representative kinematic parameters via the primary frequency analysis. The sequences are segmented based on zero-velocity crossing of the selected parameters followed by an adaptive k-means clustering to identify the repetition segments. Experimental results demonstrate that for the motion data captured by both the motion capture system and the Microsoft Kinect, our proposed algorithm obtains robust segmentation of repetitive action sequences.
---
paper_title: Overview of the Stereo and Multiview Video Coding Extensions of the H.264/MPEG-4 AVC Standard

paper_content:

Significant improvements in video compression capability have been demonstrated with the introduction of the H.264/MPEG-4 advanced video coding (AVC) standard. Since developing this standard, the Joint Video Team of the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG) has also standardized an ex- tension of that technology that is referred to as multiview video coding (MVC). MVC provides a compact representation for multiple views of a video scene, such as multiple synchronized video cameras. Stereo-paired video for 3-D viewing is an im- portant special case of MVC. The standard enables inter-view prediction to improve compression capability, as well as sup- porting ordinary temporal and spatial prediction. It also sup- ports backward compatibility with existing legacy systems by structuring the MVC bitstream to include a compatible Bbase view.( Each other view is encoded at the same picture reso- lution as the base view. In recognition of its high-quality encod- ing capability and support for backward compatibility, the stereo high profile of the MVC extension was selected by the Blu-RayDiscAssociationas thecoding formatfor3-Dvideowith high-definition resolution. This paper provides an overview of the algorithmic design used for extending H.264/MPEG-4 AVC towards MVC. The basic approach of MVC for enabling inter- view prediction and view scalability in the context of H.264/ MPEG-4 AVC is reviewed. Related supplemental enhancement information (SEI) metadata is also described. Various Bframe compatible( approaches for support of stereo-view video as an alternative to MVC are also discussed. A summary of the coding performance achieved by MVC for both stereo- and multiview video is also provided. Future directions and challenges related to 3-D video are also briefly discussed.
---
paper_title: Efficient Prediction Structures for Multiview Video Coding

paper_content:

An experimental analysis of multiview video coding (MVC) for various temporal and inter-view prediction structures is presented. The compression method is based on the multiple reference picture technique in the H.264/AVC video coding standard. The idea is to exploit the statistical dependencies from both temporal and inter-view reference pictures for motion-compensated prediction. The effectiveness of this approach is demonstrated by an experimental analysis of temporal versus inter-view prediction in terms of the Lagrange cost function. The results show that prediction with temporal reference pictures is highly efficient, but for 20% of a picture's blocks on average prediction with reference pictures from adjacent views is more efficient. Hierarchical B pictures are used as basic structure for temporal prediction. Their advantages are combined with inter-view prediction for different temporal hierarchy levels, starting from simulcast coding with no inter-view prediction up to full level inter-view prediction. When using inter-view prediction at key picture temporal levels, average gains of 1.4-dB peak signal-to-noise ratio (PSNR) are reported, while additionally using inter-view prediction at nonkey picture temporal levels, average gains of 1.6-dB PSNR are reported. For some cases, gains of more than 3 dB, corresponding to bit-rate savings of up to 50%, are obtained.
---
paper_title: Technologies for 3D mesh compression: A survey

paper_content:

Three-dimensional (3D) meshes have been widely used in graphic applications for the representation of 3D objects. They often require a huge amount of data for storage and/or transmission in the raw data format. Since most applications demand compact storage, fast transmission, and efficient processing of 3D meshes, many algorithms have been proposed to compress 3D meshes efficiently since early 1990s. In this survey paper, we examine 3D mesh compression technologies developed over the last decade, with the main focus on triangular mesh compression technologies. In this effort, we classify various algorithms into classes, describe main ideas behind each class, and compare the advantages and shortcomings of the algorithms in each class. Finally, we address some trends in the 3D mesh compression technology development.
---
paper_title: 3D spatial reconstruction and communication from vision field

paper_content:

Vision field describes the real world visual information by summarizing the seven-dimensional plenoptic function into three domains: view, light, time, from which a better understanding of previous 3D capture and reconstruction systems can be provided. In this paper, we first show how to reconstruct 3D spatial information from all the three attributes of the vision field, namely full-space vision field reconstruction. Then, based on Laplacian iterative geometry prediction, a 3D mesh coding algorithm with cascaded quantization is presented to facilitate the communication of the reconstructed 3D models from vision field. At last, experimental results of both the 3D spatial reconstruction and the 3D mesh coding are demonstrated.
---
paper_title: 3D video and free viewpoint video-From capture to display

paper_content:

This paper gives an end-to-end overview of 3D video and free viewpoint video, which can be regarded as advanced functionalities that expand the capabilities of a 2D video. Free viewpoint video can be understood as the functionality to freely navigate within real world visual scenes, as it is known for instance from virtual worlds in computer graphics. 3D video shall be understood as the functionality that provides the user with a 3D depth impression of the observed scene, which is also known as stereo video. In that sense as functionalities, 3D video and free viewpoint video are not mutually exclusive but can very well be combined in a single system. Research in this area combines computer graphics, computer vision and visual communications. It spans the whole media processing chain from capture to display and the design of systems has to take all parts into account, which is outlined in different sections of this paper giving an end-to-end view and mapping of this broad area. The conclusion is that the necessary technology including standard media formats for 3D video and free viewpoint video is available or will be available in the future, and that there is a clear demand from industry and user for such advanced types of visual media. As a consequence we are witnessing these days how such technology enters our everyday life
---
paper_title: 3-D Video Representation Using Depth Maps

paper_content:

Current 3-D video (3DV) technology is based on stereo systems. These systems use stereo video coding for pictures delivered by two input cameras. Typically, such stereo systems only reproduce these two camera views at the receiver and stereoscopic displays for multiple viewers require wearing special 3-D glasses. On the other hand, emerging autostereoscopic multiview displays emit a large numbers of views to enable 3-D viewing for multiple users without requiring 3-D glasses. For representing a large number of views, a multiview extension of stereo video coding is used, typically requiring a bit rate that is proportional to the number of views. However, since the quality improvement of multiview displays will be governed by an increase of emitted views, a format is needed that allows the generation of arbitrary numbers of views with the transmission bit rate being constant. Such a format is the combination of video signals and associated depth maps. The depth maps provide disparities associated with every sample of the video signal that can be used to render arbitrary numbers of additional views via view synthesis. This paper describes efficient coding methods for video and depth data. For the generation of views, synthesis methods are presented, which mitigate errors from depth estimation and coding.
---
paper_title: State of the Art in Stereoscopic and Autostereoscopic Displays

paper_content:

Underlying principles of stereoscopic direct-view displays, binocular head-mounted displays, and autostereoscopic direct-view displays are explained and some early work as well as the state of the art in those technologies are reviewed. Stereoscopic displays require eyewear and can be categorized based on the multiplexing scheme as: 1) color multiplexed (old technology but there are some recent developments; low-quality due to color reproduction and crosstalk issues; simple and does not require additional electronics hardware); 2) polarization multiplexed (requires polarized light output and polarization-based passive eyewear; high-resolution and high-quality displays available); and 3) time multiplexed (requires faster display hardware and active glasses synchronized with the display; high-resolution commercial products available). Binocular head-mounted displays can readily provide 3-D, virtual images, immersive experience, and more possibilities for interactive displays. However, the bulk of the optics, matching of the left and right ocular images and obtaining a large field of view make the designs quite challenging. Some of the recent developments using unconventional optical relays allow for thin form factors and open up new possibilities. Autostereoscopic displays are very attractive as they do not require any eyewear. There are many possibilities in this category including: two-view (the simplest implementations are with a parallax barrier or a lenticular screen), multiview, head tracked (requires active optics to redirect the rays to a moving viewer), and super multiview (potentially can solve the accommodation-convergence mismatch problem). Earlier 3-D booms did not last long mainly due to the unavailability of enabling technologies and the content. Current developments in the hardware technologies provide a renewed interest in 3-D displays both from the consumers and the display manufacturers, which is evidenced by the recent commercial products and new research results in this field.
---
paper_title: Learning Depth from Single Monocular Images

paper_content:

We consider the task of depth estimation from a single monocular image. We take a supervised learning approach to this problem, in which we begin by collecting a training set of monocular images (of unstructured outdoor environments which include forests, trees, buildings, etc.) and their corresponding ground-truth depthmaps. Then, we apply supervised learning to predict the depthmap as a function of the image. Depth estimation is a challenging problem, since local features alone are insufficient to estimate depth at a point, and one needs to consider the global context of the image. Our model uses a discriminatively-trained Markov Random Field (MRF) that incorporates multiscale local- and global-image features, and models both depths at individual points as well as the relation between depths at different points. We show that, even on unstructured scenes, our algorithm is frequently able to recover fairly accurate depthmaps.
---
paper_title: Symmetric stereo matching for occlusion handling

paper_content:

In this paper, we propose a symmetric stereo model to handle occlusion in dense two-frame stereo. Our occlusion reasoning is directly based on the visibility constraint that is more general than both ordering and uniqueness constraints used in previous work. The visibility constraint requires occlusion in one image and disparity in the other to be consistent. We embed the visibility constraint within an energy minimization framework, resulting in a symmetric stereo model that treats left and right images equally. An iterative optimization algorithm is used to approximate the minimum of the energy using belief propagation. Our stereo model can also incorporate segmentation as a soft constraint. Experimental results on the Middlebury stereo images show that our algorithm is state-of-the-art.
---
paper_title: Evaluation of Pose Tracking Accuracy in the First and Second Generations of Microsoft Kinect

paper_content:

Microsoft Kinect camera and its skeletal tracking capabilities have been embraced by many researchers and commercial developers in various applications of real-time human movement analysis. In this paper, we evaluate the accuracy of the human kinematic motion data in the first and second generation of the Kinect system, and compare the results with an optical motion capture system. We collected motion data in 12 exercises for 10 different subjects and from three different viewpoints. We report on the accuracy of the joint localization and bone length estimation of Kinect skeletons in comparison to the motion capture. We also analyze the distribution of the joint localization offsets by fitting a mixture of Gaussian and uniform distribution models to determine the outliers in the Kinect motion data. Our analysis shows that overall Kinect 2 has more robust and more accurate tracking of human pose as compared to Kinect 1.
---
paper_title: Microsoft Kinect Sensor and Its Effect

paper_content:

Recent advances in 3D depth cameras such as Microsoft Kinect sensors (www.xbox.com/en-US/kinect) have created many opportunities for multimedia computing. The Kinect sensor lets the computer directly sense the third dimension (depth) of the players and the environment. It also understands when users talk, knows who they are when they walk up to it, and can interpret their movements and translate them into a format that developers can use to build new experiences. While the Kinect sensor incorporates several advanced sensing hardware, this article focuses on the vision aspect of the Kinect sensor and its impact beyond the gaming industry.
---
paper_title: MC3D: Motion Contrast 3D Scanning

paper_content:

Structured light 3D scanning systems are fundamentally constrained by limited sensor bandwidth and light source power, hindering their performance in real-world applications where depth information is essential, such as industrial automation, autonomous transportation, robotic surgery, and entertainment. We present a novel structured light technique called Motion Contrast 3D scanning (MC3D) that maximizes bandwidth and light source power to avoid performance trade-offs. The technique utilizes motion contrast cameras that sense temporal gradients asynchronously, i.e., independently for each pixel, a property that minimizes redundant sampling. This allows laser scanning resolution with single-shot speed, even in the presence of strong ambient illumination, significant inter-reflections, and highly reflective surfaces. The proposed approach will allow 3D vision systems to be deployed in challenging and hitherto inaccessible real-world scenarios requiring high performance using limited power and bandwidth.
---
paper_title: Online distribution and interaction of video data in social multimedia network

paper_content:

As the emerging multimedia on the latest social multimedia network, high dimensional video (HDV) data distribution is one of the most challenging problems in social multimedia applications. In the paper, we provide a state-of-the-art analysis on the streaming and interactive technologies for the two most popular representations of HDV, video plus depth map and multi-view video since both of them can enable the view-point interaction and behavior analysis in social media applications. Additionally, view-point switching techniques in HDV streaming with emphasis on interactive functionality are discussed, including SP-frame, Wyner-ziv frame and view switchable stream generation. Moreover, the overall performance of key tools focusing on HDV streaming is evaluated and potential improvements are discussed.
---
paper_title: Depth Reconstruction Filter and Down/Up Sampling for Depth Coding in 3-D Video

paper_content:

A depth image represents three-dimensional (3-D) scene information and is commonly used for depth image-based rendering (DIBR) to support 3-D video and free-viewpoint video applications. The virtual view is generally rendered by the DIBR technique and its quality depends highly on the quality of depth image. Thus, efficient depth coding is crucial to realize the 3-D video system. In this letter, we propose a depth reconstruction filter and depth down/up sampling techniques to improve depth coding performance. Experimental results demonstrate that the proposed methods reduce the bit-rate for depth coding and achieve better rendering quality.
---
paper_title: Free Viewpoint Video Coding With Rate-Distortion Analysis

paper_content:

To improve free viewpoint video (FVV) coding efficiency and optimize the quality of the synthesized virtual view video, this paper proposes a depth-assisted FVV coding framework and analyzes the rate-distortion (R-D) property of the synthesized virtual view video in FVV coding. In the depth-assisted FVV coding framework, the depth assigned disparity compensated prediction is introduced to exploit the correlation between multiview video (MVV) and depth. To model the R-D property of the synthesized virtual view video, a region-based view synthesis distortion estimation approach is investigated with respect to the distortion of MVV and depth. Subsequently, the general R-D property estimation models of MVV and depth are analyzed. Finally, a rate-allocation scheme is designed to optimize the quantization parameter pair of MVV and depth in FVV coding. The simulation results demonstrate that the proposed depth-assisted FVV coding framework can improve the FVV coding efficiency. The region-based view synthesis distortion estimation approach and the general R-D model are able to precisely approximate the R-D property of synthesized virtual view video in the multiview video plus depth based FVV coding frameworks. The proposed rate-allocation scheme can optimize the overall FVV coding efficiency to achieve a high-quality reconstructed video at the desired viewpoint with a given rate constraint.
---
paper_title: Reduced-complexity search for video coding geometry partitions using texture and depth data

paper_content:

In this paper, a texture-space geometry partitioning approach is proposed to reduce the computational complexity of searching for geometry partitions for video coding. Additionally, for systems that capture both video and depth data, an enhanced geometry partitioning approach using both texture and depth information is proposed to further improve the partitioning accuracy and reduce the search complexity. Compared with a full-search approach, the proposed geometry partition search approaches achieve about 94% reduction of the encoding time while retaining similar rate-distortion performance.
---
paper_title: Frame Compatible Formats for 3D Video Distribution

paper_content:

Stereoscopic video will soon be delivered to the home through various channels. To make this feasible for some channels, the representation of the stereo video is modified to accommodate certain constraints on legacy systems. Among the various constraints that must be considered include the capabilities of production equipment and transmission infrastructure, as well as existing receivers and uncompressed digital interfaces between devices within the home. This paper outlines the typical constraints that are encountered in these domains and provides an overview of the various frame-compatible formats that are being considered for distribution of 3D video through such legacy systems. The benefits and drawbacks of these formats are discussed and the current status in various industry forums is reviewed.
---
paper_title: Inter-View Direct Mode for Multiview Video Coding

paper_content:

Global disparity between views is usually caused by the displacement between cameras, which can be accurately represented by a global geometric transformation. In this paper, we first propose an inter-view motion model in terms of the global geometric transformation to represent the motion correlation between two adjacent views. Specifically, the motion vector of a pixel in one view may be directly derived from that in another view according to the inter-view motion model. Further, we propose an inter-view direct mode to signal the decoder that the motion of a macroblock (MB) can be achieved from the coded view without any coding bits. The proposed inter-view direct mode is further incorporated in the existing multiview video coding (MVC) schemes (i.e., AVC-based MVC and 4-D wavelet-based MVC), working together with the other classical coding modes. The mode selection at each MB is accomplished with the rate-distortion optimization technique. The proposed inter-view direct mode can significantly reduce bits to code motion vectors especially at low bit rates, thus improving the coding efficiency
---
paper_title: Complexity-reduced geometry partition search and high efficiency prediction for video coding

paper_content:

To reduce the complexity of searching for wedge-based geometry partitions in video coding, we propose a texture-difference based partition line selection approach with a skipping strategy. Applying this approach can reduce the encoding time by 90% while retaining the similar rate-distortion performance to that of exhaustive searching. We also propose a background-compensated prediction scheme to improve the rate-distortion performance of the geometry partitioning prediction by reducing its motion vector overhead. Incorporating our proposed approach into object-boundary-based geometry partitioning can achieve about 10% bit-rate savings relative to the full-search approach while keeping the complexity at about the same level as our proposed complexity-reduced wedge-based geometry partitioning.
---
paper_title: Complexity Reduction and Performance Improvement for Geometry Partitioning in Video Coding

paper_content:

Geometry partitioning for video coding involves establishing a partition line boundary within each block-shaped region and applying motion-compensated prediction to the two sub-regions created by the partition line. This paper presents techniques for enhancing the effectiveness and reducing the complexity of geometry partitioning schemes. A texture-difference-based approach is described to simplify the process of selecting the partition lines. Applying this approach together with a described skipping strategy for blocks with uniform texture can achieve a 94% reduction of encoding time while retaining a similar rate-distortion (R-D) performance to the full-search partitioning approach, when implemented for wedge-based geometry partitioning (WGP) in the context of H.264/MPEG-4 AVC JM 16.2. A bit rate improvement of approximately 6% is shown relative to not using geometry partitioning. For further R-D improvement, we describe a background-compensated prediction scheme to reduce the number of overhead bits used for motion vectors. Additionally, for systems in which high-quality depth maps are available, we incorporate depth map usage into the described approaches to generate a more accurate partitioning. Using these approaches with object-boundary-based geometry partitioning can achieve about 9% bit rate savings relative to using WGP, while keeping a similar computational complexity to the described complexity-reduced WGP.
---
paper_title: Depth Map Coding Based on Synthesized View Distortion Function

paper_content:

This paper presents an efficient depth map coding method based on a newly defined rendering view distortion function. As compared to the conventional depth map coding in which distortion is measured by only investigating the coding error in depth map, the proposed scheme focuses on virtually synthesized view quality by involving co-located color information. In detail, the proposed distortion function estimates rendered view quality, where area-based scheme is provided in order to mimic the warping/view-rendering process accurately. Moreover, the coding performance of the proposed distortion metric is even improved by involving the additional SKIP mode derived by co-located color coding information. The simulation results show the proposed scheme could achieve approximately 30% bit-rate saving for depth data, and about 10% bit-rate saving for overall multi-view data.
---
paper_title: Joint multiview video plus depth coding

paper_content:

Multiview video plus depth (MVD), where per-pixel depth map sequences are associated with multiview texture video, is a promising approach for three-dimensional (3D) video solutions requiring view synthesis. The Multiview Video Coding (MVC) standard has been typically used to compress the MVD representation, resulting into the multiview texture video and the multiview depth video being in two separate MVC bitstreams. Such a coding scheme does not, however, utilize the similarities of the motion information of the multiview texture video and the multiview depth video. The joint multiview video plus depth coding (JMVDC) scheme presented in this paper uses inter-view prediction similarly to MVC as well as the inter-layer motion prediction tool of the Scalable Video Coding (SVC) standard to exploit the correlation of the motion in the texture and depth video sequences. The simulation results show that the JMVDC method achieves about 10 to 20% saving in depth bitrate compared with conventional MVC-based coding of MVD representations.
---
paper_title: View Scalable Multiview Video Coding Using 3-D Warping With Depth Map

paper_content:

Multiview video coding demands high compression rates as well as view scalability, which enables the video to be displayed on a multitude of different terminals. In order to achieve view scalability, it is necessary to limit the inter-view prediction structure. In this paper, we propose a new multiview video coding scheme that can improve the compression efficiency under such a limited inter-view prediction structure. All views are divided into two groups in the proposed scheme: base view and enhancement views. The proposed scheme first estimates a view-dependent geometry of the base view. It then uses a video encoder to encode the video of base view. The view-dependent geometry is also encoded by the video encoder. The scheme then generates prediction images of enhancement views from the decoded video and the view-dependent geometry by using image-based rendering techniques, and it makes residual signals for each enhancement view. Finally, it encodes residual signals by the conventional video encoder as if they were regular video signals. We implement one encoder that employs this scheme by using a depth map as the view-dependent geometry and 3-D warping as the view generation method. In order to increase the coding efficiency, we adopt the following three modifications: (1) object-based interpolation on 3-D warping; (2) depth estimation with consideration of rate-distortion costs; and (3) quarter-pel accuracy depth representation. Experiments show that the proposed scheme offers about 30% higher compression efficiency than the conventional scheme, even though one depth map video is added to the original multiview video.
---
paper_title: Client-Driven Selective Streaming of Multiview Video for Interactive 3DTV

paper_content:

We present a novel client-driven multiview video streaming system that allows a user to watch 3D video interactively with significantly reduced bandwidth requirements by transmitting a small number of views selected according to his/her head position. The user's head position is tracked and predicted into the future to select the views that best match the user's current viewing angle dynamically. Prediction of future head positions is needed so that views matching the predicted head positions can be prefetched in order to account for delays due to network transport and stream switching. The system allocates more bandwidth to the selected views in order to render the current viewing angle. Highly compressed, lower quality versions of some other views are also prefetched for concealment if the current user viewpoint differs from the predicted viewpoint. An objective measure based on the abruptness of the head movements and delays in the system is introduced to determine the number of additional lower quality views to be prefetched. The proposed system makes use of multiview coding (MVC) and scalable video coding (SVC) concepts together to obtain improved compression efficiency while providing flexibility in bandwidth allocation to the selected views. Rate-distortion performance of the proposed system is demonstrated under different experimental conditions.
---
paper_title: Online distribution and interaction of video data in social multimedia network

paper_content:

As the emerging multimedia on the latest social multimedia network, high dimensional video (HDV) data distribution is one of the most challenging problems in social multimedia applications. In the paper, we provide a state-of-the-art analysis on the streaming and interactive technologies for the two most popular representations of HDV, video plus depth map and multi-view video since both of them can enable the view-point interaction and behavior analysis in social media applications. Additionally, view-point switching techniques in HDV streaming with emphasis on interactive functionality are discussed, including SP-frame, Wyner-ziv frame and view switchable stream generation. Moreover, the overall performance of key tools focusing on HDV streaming is evaluated and potential improvements are discussed.
---
paper_title: Free Viewpoint Video Coding With Rate-Distortion Analysis

paper_content:

To improve free viewpoint video (FVV) coding efficiency and optimize the quality of the synthesized virtual view video, this paper proposes a depth-assisted FVV coding framework and analyzes the rate-distortion (R-D) property of the synthesized virtual view video in FVV coding. In the depth-assisted FVV coding framework, the depth assigned disparity compensated prediction is introduced to exploit the correlation between multiview video (MVV) and depth. To model the R-D property of the synthesized virtual view video, a region-based view synthesis distortion estimation approach is investigated with respect to the distortion of MVV and depth. Subsequently, the general R-D property estimation models of MVV and depth are analyzed. Finally, a rate-allocation scheme is designed to optimize the quantization parameter pair of MVV and depth in FVV coding. The simulation results demonstrate that the proposed depth-assisted FVV coding framework can improve the FVV coding efficiency. The region-based view synthesis distortion estimation approach and the general R-D model are able to precisely approximate the R-D property of synthesized virtual view video in the multiview video plus depth based FVV coding frameworks. The proposed rate-allocation scheme can optimize the overall FVV coding efficiency to achieve a high-quality reconstructed video at the desired viewpoint with a given rate constraint.
---
paper_title: Asymmetric Coding of Multi-View Video Plus Depth Based 3-D Video for View Rendering

paper_content:

The recent years have witnessed three-dimensional (3-D) video technology to become increasingly popular, as it can provide high-quality and immersive experience to end users, where view rendering with depth-image-based rendering (DIBR) technique is employed to generate the virtual views. Distortions in depth map may induce geometry changes in the virtual views, and distortions in texture video may be propagated to the virtual views. Thus, effective compression of both texture videos and depth maps is important for 3-D video system. From the perspective of bit allocation, asymmetric coding of the texture videos and depth maps is an effective way to get the optimal solution of 3-D video compression and view rendering problems. In this paper, a novel asymmetric coding method of multi-view video plus depth (MVD) based 3-D video is proposed on purpose of providing high-quality view rendering. In the proposed method, two models are proposed to characterize view rendering distortion and binocular suppression in 3-D video. Then, an asymmetric coding method of MVD-based 3-D video is proposed by combining two models in encoding framework. Finally, a chrominance reconstruction algorithm is presented to achieve accurate reconstruction. Experimental results show that compared with other methods, the proposed method can obtain higher performance of view rendering under the total bitrate constraint. Moreover, the perceptual visual quality of 3-D video is almost unaffected with the proposed method.
---
paper_title: Rate Distortion Optimized Inter-View Frame Level Bit Allocation Method for MV-HEVC

paper_content:

In multi-view video coding, since inter-view prediction has been adopted as an important coding tool which could improve coding efficiency greatly, inter-view dependency is inevitable, i.e., the distortion of the reference view (RV) picture could be propagated to the non-reference view (NRV) pictures . Therefore, in order to achieve higher coding efficiency , the inter-view dependency must be taken into account for inter-view bit allocation. In this paper, the inter-view dependency is analyzed in detail, and a rate-distortion (RD) model for NRVs is derived by taking the distortion of RV into account. Based on the derived RD model, the inter-view bit allocation is represented as a mathematical problem with an analytic form, and is solved by a convex optimization (Lagrangian Multiplier) method. Experimental results demonstrate that the RD performance and the inter-view quality consistency of the proposed method is better than existing methods, while the complexity of the proposed method is comparable with the existing methods.
---
paper_title: Region Based Rate-Distortion Analysis for 3D Video Coding

paper_content:

In 3D video coding, to provide the high-quality interactive viewpoint video to audience, it is necessary to jointly optimize coding efficiency of color and depth images at a given bit-rates. In this paper, a region based distortion model is proposed to precisely estimate the error of synthesized virtual view. Furthermore, combined with the rate-distortion (R-D) model of color and depth images coding, an overall R-D model is built up for 3D video coding. Experimental results exhibit that the proposed approach can efficiently measure the R-D property of 3D video coding.
---
paper_title: Model-Based Joint Bit Allocation Between Texture Videos and Depth Maps for 3-D Video Coding

paper_content:

In 3-D video coding, texture videos and depth maps need to be jointly coded. The distortion of texture videos and depth maps can be propagated to the synthesized virtual views. Besides coding efficiency of texture videos and depth maps, joint bit allocation between texture videos and depth maps is also an important research issue in 3-D video coding. First, we present comprehensive analyses on the impacts of the compression distortion of texture videos and depth maps on the quality of the virtual views, and then derive a concise distortion model for the synthesized virtual views. Based on this model, the joint bit allocation problem is formulated as a constrained optimization problem, and is solved by using the Lagrangian multiplier method. Experimental results demonstrate the high accuracy of the derived distortion model. Meanwhile, the rate-distortion (R-D) performance of the proposed algorithm is close to those of search-based algorithms which can give the best R-D performance, while the complexity of the proposed algorithm is lower than that of search-based algorithms. Moreover, compared with the bit allocation method using fixed texture and depth bits ratio (5:1), a maximum 1.2 dB gain can be achieved by the proposed algorithm.
---
paper_title: Joint video/depth rate allocation for 3D video coding based on view synthesis distortion model

paper_content:

Joint video/depth rate allocation is an important optimization problem in 3D video coding. To address this problem, this paper proposes a distortion model to evaluate the synthesized view without access to the captured original view. The proposed distortion model is an additive model that accounts for the video-coding-induced distortion and the depth-quantization-induced distortion, as well as the inherent geometry distortion. Depth-quantization-induced distortion not only considers the warping error distortion, which is described by a piecewise linear model with the video power spectral property, but also takes into account the warping error correlation distortion between two sources reference views. Geometry distortion is approximated from that of the adjacent view synthesis. Based on the proposed distortion model, a joint rate allocation method is proposed to seek the optimal trade-off between video bit-rate and depth bit-rate for maximizing the view synthesis quality. Experimental results show that the proposed distortion model is capable of approximately estimating the actual distortion for the synthesized view, and that the proposed rate allocation method can almost achieve the identical rate allocation performance as the full-search method at less computational cost. Moreover, the proposed rate allocation method consumes less computational cost than the hierarchical-search method at high bit-rates while providing almost the equivalent rate allocation performance.
---

Title: An Overview of Emerging Technologies for High Efficiency 3D Video Coding

Section 1: INTRODUCTION
Description 1: This section introduces the importance of 3D video coding and the advancements made in the field, discussing different types of 3D video representations and the necessity for efficient data compression.

Section 2: 3D VIDEO REPRESENTATIONS
Description 2: This section briefly introduces the background information about the 3D video representations, depth map generation, and virtual view synthesis.

Section 3: SIMULCAST AND MULTIVIEW VIDEO CODING
Description 3: This section describes the frameworks for encoding multiview video, including both simulcast and multiview video coding (MVC) techniques, and discusses the trade-offs between coding efficiency and view switching latency.

Section 4: MULTIVIEW VIDEO AND DEPTH MAPS CODING
Description 4: This section analyzes the joint coding of multiview video and depth maps, exploring different coding schemes and their effects on R-D (rate-distortion) performance.

Section 5: SCALABLE 3D VIDEO CODING
Description 5: This section reviews scalable 3D video coding frameworks, including resolution and quality scalable MVC, and discusses their implications on coding efficiency and view scalability.

Section 6: BIT ALLOCATION OF 3D VIDEO CODING
Description 6: This section examines bit allocation strategies in 3D video coding, including the impact of texture video and depth map quality on the synthesized video, and bit allocation among different views.

Section 7: CONCLUSIONS
Description 7: This section summarizes the recent progress in high-efficiency 3D video coding technologies and outlines future research directions including view switching optimization, scalable coding, and bit allocation algorithms.