Perceptual-based Quality Metrics for Image and Video Services: A Survey
---
paper_title: Digital Video Quality: Vision Models and Metrics

paper_content:

About the Author. Acknowledgements. Acronyms. 1 Introduction. 1.1 Motivation. 1.2 Outline. 2 Vision. 2.1 Eye. 2.2 Retina. 2.3 Visual Pathways. 2.4 Sensitivity to Light. 2.5 Color Perception. 2.6 Masking and Adaptation. 2.7 Multi-channel Organization. 2.8 Summary. 3 Video Quality. 3.1 Video Coding and Compression. 3.2 Artifacts. 3.3 Visual Quality. 3.4 Quality Metrics. 3.5 Metric Evaluation. 3.6 Summary. 4 Models and Metrics. 4.1 Isotropic Contrast. 4.2 Perceptual Distortion Metric. 4.3 Summary. 5 Metric Evaluation. 5.1 Still Images. 5.2 Video. 5.3 Component Analysis. 5.4 Summary. 6 Metric Extensions. 6.1 Blocking Artifacts. 6.2 Object Segmentation. 6.3 Image Appeal. 6.4 Summary. 7 Closing Remarks. 7.1 Summary. 7.2 Perspectives. Appendix: Color Space Conversions. References. Index.
---
paper_title: Digital Video Quality: Vision Models and Metrics

paper_content:

About the Author. Acknowledgements. Acronyms. 1 Introduction. 1.1 Motivation. 1.2 Outline. 2 Vision. 2.1 Eye. 2.2 Retina. 2.3 Visual Pathways. 2.4 Sensitivity to Light. 2.5 Color Perception. 2.6 Masking and Adaptation. 2.7 Multi-channel Organization. 2.8 Summary. 3 Video Quality. 3.1 Video Coding and Compression. 3.2 Artifacts. 3.3 Visual Quality. 3.4 Quality Metrics. 3.5 Metric Evaluation. 3.6 Summary. 4 Models and Metrics. 4.1 Isotropic Contrast. 4.2 Perceptual Distortion Metric. 4.3 Summary. 5 Metric Evaluation. 5.1 Still Images. 5.2 Video. 5.3 Component Analysis. 5.4 Summary. 6 Metric Extensions. 6.1 Blocking Artifacts. 6.2 Object Segmentation. 6.3 Image Appeal. 6.4 Summary. 7 Closing Remarks. 7.1 Summary. 7.2 Perspectives. Appendix: Color Space Conversions. References. Index.
---
paper_title: Foundations of vision

paper_content:

Originally published in Contemporary Psychology: APA Review of Books, 1997, Vol 42(7), 649-649. In Foundations of Vision (see record 1995-98050-000), Brian Wandell divides the study of vision into three parts: encoding, representation, and interpretation. Each part is designed to inform students on
---
paper_title: Digital Video Image Quality and Perceptual Coding

paper_content:

PICTURE CODING AND HUMAN VISUAL SYSTEM FUNDAMENTALS Digital Picture Compression and Coding Structure . Introduction to Digital Picture Coding . Characteristics of Picture Data . Compression and Coding Techniques . Picture Quantization . Rate-Distortion Theory . Human Visual Systems . Digital Picture Coding Standards and Systems . Summary Fundamentals of Human Vision and Vision Modeling . Introduction . A Brief Overview of the Visual System . Color Vision . Luminance and the Perception of Light Intensity . Spatial Vision and Contrast Sensitivity . Temporal Vision and Motion . Visual Modeling . Conclusions Coding Artifacts and Visual Distortions . Introduction . Blocking Effect . Basis Image Effect . Blurring . Color Bleeding . Staircase Effect . Ringing . Mosaic Patterns . False Contouring . False Edges . MC Mismatch . Mosquito Effect . Stationary Area Fluctuations . Chrominance Mismatch . Video Scaling and Field Rate Conversion . Deinterlacing . Summary PICTURE QUALITY ASSESSMENT AND METRICS Video Quality Testing . Introduction . Subjective Assessment Methodologies . Selection of Test Materials . Selection of Participants-Subjects . Experimental Design . International Test Methods . Objective Assessment Methods . Summary Perceptual Video Quality Metrics-A Review . Introduction . Quality Factors . Metric Classification . Pixel-Based Metrics . The Psychophysical Approach . The Engineering Approach . Metric Comparisons . Conclusions and Perspectives Philosophy of Picture Quality Scale . Objective Picture Quality Scale for Image Coding . Application of PQS to a Variety of Electronic Images . Various Categories of Image Systems . Study at ITU . Conclusion Structural Similarity Based Image Quality Assessment . Structural Similarity Based Image Quality . The Structural SIMilarity (SSIM) Index . Image Quality Assessment Based on the SSIM Index . Discussions Vision Model Based Digital Video Impairment Metrics . Introduction . Vision Modeling for Impairment Measurement . Perceptual Blocking Distortion Metric . Perceptual Ringing Distortion Measure . Conclusion Computational Models for Just-Noticeable Difference . Introduction . JND with DCT Subbands . JND with Pixels . JND Model Evaluation . Conclusions No-Reference Quality Metric for Degraded and Enhanced Video . Introduction . State-of-the-Art for No-Reference Metrics . Quality Metric Components and Design . No-Reference Overall Quality Metric . Performance of the Quality Metric . Conclusions and Future Research Video Quality Experts Group . Formation . Goals . Phase I . Phase II . Continuing Work and Directions . Summary PERCEPTUAL CODING AND PROCESSING OF DIGITAL PICTURES HVS Based Perceptual Video Encoders . Introduction . Noise Visibility and Visual Masking . Architectures for Perceptual Based Coding . Standards-Specific Features . Salience/Maskability Pre-Processing . Application to Multi-Channel Encoding Perceptual Image Coding . Introduction . A Perceptual Distortion Metric Based Image Coder . Model Calibration . Performance Evaluation . Perceptual Lossless Coder . Summary Foveated Image and Video Coding . Foveated Human Vision and Foveated Image Processing . Foveation Methods . Scalable Foveated Image and Video Coding . Discussions Artifact Reduction by Post-Processing in Image Compression . Introduction . Image Compression and Coding Artifacts . Reduction of Blocking Artifacts . Reduction of Ringing Artifacts . Summary Reduction of Color Bleeding in DCT Block-Coded Video . Introduction . Detailed Analysis of the Color Bleeding Phenomenon . Description of the Post-Processor . Experimental Results-Concluding Remarks Error Resilience for Video Coding Service . Introduction to Error Resilient Coding Techniques . Error Resilient Coding Methods Compatible with MPEG-2 . Methods for Concealment of Cell Loss . Experimental Procedure . Experimental Results . Conclusions Critical Issues and Challenges . Picture Coding Structures . Vision Modeling Issues . Spatio-Temporal Masking in Video Coding . Picture Quality Assessment . Challenges in Perceptual Coder Design . Codec System Design Optimization . Summary Appendix: VQM Performance Metrics . Metrics Relating to Model Prediction Accuracy . Metrics Relating to Prediction Monotonicity of a Model . Metrics Relating to Prediction Consistency . MATLAB(R) Source Code . Supplementary Analyses INDEX
---
paper_title: Image quality measures and their performance

paper_content:

A number of quality measures are evaluated for gray scale image compression. They are all bivariate, exploiting the differences between corresponding pixels in the original and degraded images. It is shown that although some numerical measures correlate well with the observers' response for a given compression technique, they are not reliable for an evaluation across different techniques. A graphical measure called Hosaka plots, however, can be used to appropriately specify not only the amount, but also the type of degradation in reconstructed images.
---
paper_title: The effects of a visual fidelity criterion of the encoding of images

paper_content:

Shannon's rate-distortion function provides a potentially useful lower bound against which to compare the rate-versus-distortion performance of practical encoding-transmission systems. However, this bound is not applicable unless one can arrive at a numerically-valued measure of distortion which is in reasonable correspondence with the subjective evaluation of the observer or interpreter. We have attempted to investigate this choice of distortion measure for monochrome still images. This investigation has considered a class of distortion measures for which it is possible to simulate the optimum (in a rate-distortion sense) encoding. Such simulation was performed at a fixed rate for various measures in the class and the results compared subjectively by observers. For several choices of transmission rate and original images, one distortion measure was fairly consistently rated as yielding the most satisfactory appearing encoded images.
---
paper_title: Model of visual contrast gain control and pattern masking

paper_content:

We have implemented a model of contrast gain and control in human vision that incorporates a number of key features, including a contrast sensitivity function, multiple oriented bandpass channels, accelerating nonlinearities, and a devisive inhibitory gain control pool. The parameters of this model have been optimized through a fit to the recent data that describe masking of a Gabor function by cosine and Gabor masks [J. M. Foley, "Human luminance pattern mechanisms: masking experiments require a new model," J. Opt. Soc. Am. A 11, 1710 (1994)]. The model achieves a good fit to the data. We also demonstrate how the concept of recruitment may accommodate a variant of this model in which excitatory and inhibitory paths have a common accelerating nonlinearity, but which include multiple channels tuned to different levels of contrast.
---
paper_title: An information fidelity criterion for image quality assessment using natural scene statistics

paper_content:

Measurement of visual quality is of fundamental importance to numerous image and video processing applications. The goal of quality assessment (QA) research is to design algorithms that can automatically assess the quality of images or videos in a perceptually consistent manner. Traditionally, image QA algorithms interpret image quality as fidelity or similarity with a "reference" or "perfect" image in some perceptual space. Such "full-reference" QA methods attempt to achieve consistency in quality prediction by modeling salient physiological and psychovisual features of the human visual system (HVS), or by arbitrary signal fidelity criteria. In this paper, we approach the problem of image QA by proposing a novel information fidelity criterion that is based on natural scene statistics. QA systems are invariably involved with judging the visual quality of images and videos that are meant for "human consumption". Researchers have developed sophisticated models to capture the statistics of natural signals, that is, pictures and videos of the visual environment. Using these statistical models in an information-theoretic setting, we derive a novel QA algorithm that provides clear advantages over the traditional approaches. In particular, it is parameterless and outperforms current methods in our testing. We validate the performance of our algorithm with an extensive subjective study involving 779 images. We also show that, although our approach distinctly departs from traditional HVS-based methods, it is functionally similar to them under certain conditions, yet it outperforms them due to improved modeling. The code and the data from the subjective study are available at [1].
---
paper_title: Subjective quality evaluation of video sequences by using motion information

paper_content:

Many different quality metrics have been proposed an order to evaluate the reconstructed quality of compressed sequences. The most used measures are quantitative metrics based on a simple difference between frames. Their main drawback is that they do not correlate well with subjective quality evaluations. Up to now the only standardized method used to estimate the perceived quality of a coding system is the subjective assessment: a series of test scenes is shown to a number of selected observers, in a controlled environment. This method of quality evaluation shows limits in term of time consumption and results' interpretation. The purpose of many recent studies is to develop objective measures having good correlation with subjective quality scores in relation to the same video system and test scenes. A number of parameters influence the viewers' judgment of the sequence quality; in this paper we analyze the sequence motion effects on the viewers video quality perception. This paper proposes preliminary results in terms of the correlation between a linear model, that takes into account the “criticality” of the scene motion, and the results of formal subjective assessments.
---
paper_title: A no-reference perceptual blur metric

paper_content:

We present a no-reference blur metric for images and video. The blur metric is based on the analysis of the spread of the edges in an image. Its perceptual significance is validated through subjective experiments. The novel metric is near real-time, has low computational complexity and is shown to perform well over a range of image content. Potential applications include optimization of source coding, network resource management and autofocus of an image capturing device.
---
paper_title: Efficient DCT-domain Blind Measurement and Reduction of Blocking Artifacts

paper_content:

Blocking artifacts continue to be among the most serious defects that occur in images and video streams compressed to low bit rates using block discrete cosine transform (DCT)-based compression standards (e.g., JPEG, MPEG, and H.263). It is of interest to be able to numerically assess the degree of blocking artifact in a visual signal, for example, in order to objectively determine the efficacy of a compression method, or to discover the quality of video content being delivered by a web server. We propose new methods for efficiently assessing, and subsequently reducing, the severity of blocking artifacts in compressed image bitstreams. The method is blind, and operates only in the DCT domain. Hence, it can be applied to unknown visual signals, and it is efficient since the signal need not be compressed or decompressed. In the algorithm, blocking artifacts are modeled as 2-D step functions. A fast DCT-domain algorithm extracts all parameters needed to detect the presence of, and estimate the amplitude of blocking artifacts, by exploiting several properties of the human vision system. Using the estimate of blockiness, a novel DCT-domain method is then developed which adaptively reduces detected blocking artifacts. Our experimental results show that the proposed method of measuring blocking artifacts is effective and stable across a wide variety of images. Moreover, the proposed blocking-artifact reduction method exhibits satisfactory performance as compared to other post-processing techniques. The proposed technique has a low computational cost hence can be used for real-time image/video quality monitoring and control, especially in applications where it is desired that the image/video data be processed directly in the DCT-domain.
---
paper_title: Blur determination in the compressed domain using DCT information

paper_content:

The paper presents a simple yet robust measure of image quality in terms of global (camera) blur. It is based on histogram computation of non-zero DCT coefficients. The technique is directly applicable to images and video frames in compressed (MPEG or JPEG) domain and to all types of MPEG frames (I-, P- or B-frames). The resulting quality measure is proved to be in concordance with subjective testing and is therefore suitable for quick qualitative characterization of images and video frames.
---
paper_title: No-reference sharpness metric based on local edge kurtosis

paper_content:

Sharpness metrics that use the frequency spectrum of the image cannot separate the sharpness information from scene content. Sharpness metrics that use spatial gradients of the edges of objects work only for comparisons among versions of the same image. We have developed a content independent, no-reference metric. In this approach, we create an edge profile by detecting edge pixels and enclosing them with 8/spl times/8 pixel blocks. For each block we compute sharpness using kurtosis of the DCT. The final metric is the average sharpness of the blocks in the edge profile. The advantages of this method include robust combination of spatial and frequency domain information, flexibility in the edge detection component, and a normalized sharpness scale. The results show high correlation with subjective quality for sharpness-enhanced video.
---
paper_title: Detection of blocking artifacts in compressed video

paper_content:

A novel algorithm for the detection of artifacts in compressed video is described. The algorithm does not require access to the original video and therefore can be very useful in situations where quality assessment needs to be carried out at the receiver end. The proposed method is insensitive to other common video artifacts such as blurring for similar levels of objective distortion. The use of fast transformations in the frequency domain makes the method attractive for real-time applications.
---
paper_title: A generalized block-edge impairment metric for video coding

paper_content:

A mew generalized block-edge impairment metric (GBIM) is presented in this paper as a quantitative distortion measure for blocking artifacts in digital video and image coding. This distortion measure does not require the original image sequence as a comparative reference, and is found to be consistent with subjective evaluation.
---
paper_title: A single-ended blockiness measure for JPEG-coded images

paper_content:

In three subjective experiments, dissimilarity data and numerical category scaling data were obtained to determine the underlying attributes of image quality in sequential baseline-coded JPEG images. Although several distortions were perceived, namely blockiness, ringing and blur, the subjective data for all attributes were highly correlated, so that image quality could approximately be described in one dimension. We therefore proceeded by developing an instrumental measure for one of the distortions, namely, blockiness. In this paper a single-ended blockiness measure is proposed, i.e., one that uses only the coded image. Our approach is therefore fundamentally different from most (double-ended) image quality models that need both the original and the degraded image. The proposed measure is based on detecting the low-amplitude edges that result from blocking and estimating the edge amplitudes. Because of the approximate one-dimensionality of the underlying psychological space, the proposed blockiness measure also predicts the image quality of sequential baseline coded JPEG images.
---
paper_title: No-reference video quality metric based on artifact measurements

paper_content:

In this paper we present a no-reference video quality metric based on individual measurements of three artifacts: blockiness, blurriness, and noisiness. The set of artifact metrics (physical strength measurements) was designed to be simple enough to be used in real-time applications. The metrics are tested using a proposed procedure that uses synthetic artifacts and subjective data obtained from previous experiments. The technique has the advantage of allowing us to test each metric on videos which contain only the desired artifact signal or a combination of artifact signals. Models for the overall annoyance based on a combination of the artifact metrics using both a Minkowski metric and a linear model are developed. Both models present a very good correlation with the data and show no statistical difference in their performances.
---
paper_title: No-reference quality assessment using natural scene statistics: JPEG2000

paper_content:

Measurement of image or video quality is crucial for many image-processing algorithms, such as acquisition, compression, restoration, enhancement, and reproduction. Traditionally, image quality assessment (QA) algorithms interpret image quality as similarity with a "reference" or "perfect" image. The obvious limitation of this approach is that the reference image or video may not be available to the QA algorithm. The field of blind, or no-reference, QA, in which image quality is predicted without the reference image or video, has been largely unexplored, with algorithms focussing mostly on measuring the blocking artifacts. Emerging image and video compression technologies can avoid the dreaded blocking artifact by using various mechanisms, but they introduce other types of distortions, specifically blurring and ringing. In this paper, we propose to use natural scene statistics (NSS) to blindly measure the quality of images compressed by JPEG2000 (or any other wavelet based) image coder. We claim that natural scenes contain nonlinear dependencies that are disturbed by the compression process, and that this disturbance can be quantified and related to human perceptions of quality. We train and test our algorithm with data from human subjects, and show that reasonably comprehensive NSS models can help us in making blind, but accurate, predictions of quality. Our algorithm performs close to the limit imposed on useful prediction by the variability between human subjects.
---
paper_title: Image quality assessment: from error visibility to structural similarity

paper_content:

Objective methods for assessing perceptual image quality traditionally attempted to quantify the visibility of errors (differences) between a distorted image and a reference image using a variety of known properties of the human visual system. Under the assumption that human visual perception is highly adapted for extracting structural information from a scene, we introduce an alternative complementary framework for quality assessment based on the degradation of structural information. As a specific example of this concept, we develop a structural similarity index and demonstrate its promise through a set of intuitive examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000. A MATLAB implementation of the proposed algorithm is available online at http://www.cns.nyu.edu//spl sim/lcv/ssim/.
---
paper_title: No-reference perceptual quality assessment of JPEG compressed images

paper_content:

Human observers can easily assess the quality of a distorted image without examining the original image as a reference. By contrast, designing objective No-Reference (NR) quality measurement algorithms is a very difficult task. Currently, NR quality assessment is feasible only when prior knowledge about the types of image distortion is available. This research aims to develop NR quality measurement algorithms for JPEG compressed images. First, we established a JPEG image database and subjective experiments were conducted on the database. We show that Peak Signal-to-Noise Ratio (PSNR), which requires the reference images, is a poor indicator of subjective quality. Therefore, tuning an NR measurement model towards PSNR is not an appropriate approach in designing NR quality metrics. Furthermore, we propose a computational and memory efficient NR quality assessment model for JPEG images. Subjective test results are used to train the model, which achieves good quality prediction performance.
---
paper_title: Segmentation-driven perceptual quality metrics

paper_content:

We present a full-reference and a no-reference perceptual video quality metric that incorporate both low-level and high-level aspects of vision. Low-level aspects include color perception, contrast sensitivity, masking as well as artifact analysis. High-level aspects take into account the cognitive behavior of an observer when watching a video by means of semantic segmentation. Using the special case of semantic face segmentation, we evaluate the proposed segmentation-driven perceptual quality metrics using a range of test sequences and demonstrate an improvement of their prediction performance.
---
paper_title: Evaluation of video quality based on objectively estimated metric

paper_content:

Multimedia applications and especially encoded video services, are expected to play a major role in the 3rd generation (3G) and beyond mobile communication systems. Given that future service providers are expected to provide video applications at various price and quality levels, quick and economically affordable methods for preparing/encoding the offering media at various qualities are necessary to be developed. This paper presents a method for objective evaluation of the perceived quality of MPEG-4 video content, based on a quantification of subjective assessments. Showing that subjectively derived perceived quality of service (PQoS) vs. bit rate curves can be successfully approximated by a group of exponential functions, the proposed method exploits a simple objective metric, which is obtained from the mean frame rate vs. bit rate curves of an encoded clip. The validity of this metric is assessed by comparing subjectively derived PQoS results to the corresponding ones, which come from the proposed objective method, showing that the proposed technique provides satisfactory PQoS estimation.
---
paper_title: Pseudo no reference image quality metric using perceptual data hiding

paper_content:

Regarding the important constraints due to subjective quality assessment, objective image quality assessment has recently been extensively studied. Such metrics are usually of three kinds, they might be Full Reference (FR), Reduced Reference (RR) or No Reference (NR) metrics. We focus here on a new technique, which recently appeared in quality assessment context: data-hiding-based image quality metric. Regarding the amount of data to be transmitted for quality assessment purpose, watermarking based techniques are considered as pseudo noreference metric: A little overhead due to the embedded watermark is added to the image. Unlike most existing techniques, the proposed embedding method exploits an advanced perceptual model in order to optimize both the data embedding and extraction. A perceptually weighted watermark is embedded into the host image, and an evaluation of this watermark allows to assess the host image's quality. In such context, the watermark robustness is crucial; it must be suffciently robust to be detected after very strong distortions, but it must also be suffciently fragile to be degraded along with the host image. In other words, the watermark distortion must be proportional to the image's distortion. Our work is compared to existing standard RR and NR metrics in terms of both the correlation with subjective assessment and of data overhead induced by the mark.
---
paper_title: Video quality objective metric using data hiding

paper_content:

In this paper a non-reference objective video quality metric is proposed. The quality metric is obtained by means of a non-conventional use of data hiding technique. Test data are embedded in an MPEG-2 video; the basic assumption is that the data embedded undergo under the same degradation as the host video. To analyze the performance of the system, a comparison between the results obtained using this metric and the perceived mean annoyance values was performed. The annoyance values were obtained through a psychophysical experiment, which measured the threshold and mean annoyance values of compressed videos.
---
paper_title: Quality-aware images

paper_content:

We propose the concept of quality-aware image , in which certain extracted features of the original (high-quality) image are embedded into the image data as invisible hidden messages. When a distorted version of such an image is received, users can decode the hidden messages and use them to provide an objective measure of the quality of the distorted image. To demonstrate the idea, we build a practical quality-aware image encoding, decoding and quality analysis system, which employs: 1) a novel reduced-reference image quality assessment algorithm based on a statistical model of natural images and 2) a previously developed quantization watermarking-based data hiding technique in the wavelet transform domain.
---
paper_title: Reduced-reference image quality assessment using a wavelet-domain natural image statistic model

paper_content:

Reduced-reference (RR) image quality measures aim to predict the visual quality of distorted images with only partial information about the reference images. In this paper, we propose an RR quality assessment method based on a natural image statistic model in the wavelet transform domain. In particular, we observe that the marginal distribution of wavelet coefficients changes in different ways for different types of image distortions. To quantify such changes, we estimate the Kullback-Leibler distance between the marginal distributions of wavelet coefficients of the reference and distorted images. A generalized Gaussian model is employed to summarize the marginal distribution of wavelet coefficients of the reference image, so that only a relatively small number of RR features are needed for the evaluation of image quality. The proposed method is easy to implement and computationally efficient. In addition, we find that many well-known types of image distortion lead to significant changes in wavelet coefficient histograms, and thus are readily detectable by our measure. The algorithm is tested with subjective ratings of a large image database that contains images corrupted with a wide variety of distortion types.
---
paper_title: A framework for advanced video traces: evaluating visual quality for video transmission over lossy networks

paper_content:

Conventional video traces (which characterize the video encoding frame sizes in bits and frame quality in PSNR) are limited to evaluating loss-free video transmission. To evaluate robust video transmission schemes for lossy network transport, generally experiments with actual video are required. To circumvent the need for experiments with actual videos, we propose in this paper an advanced video trace framework. The two main components of this framework are (i) advanced video traces which combine the conventional video traces with a parsimonious set of visual content descriptors, and (ii) quality prediction schemes that based on the visual content descriptors provide an accurate prediction of the quality of the reconstructed video after lossy network transport. We conduct extensive evaluations using a perceptual video quality metric as well as the PSNR in which we compare the visual quality predicted based on the advanced video traces with the visual quality determined from experiments with actual video. We find that the advanced video trace methodology accurately predicts the quality of the reconstructed video after frame losses.
---
paper_title: Perceived Audiovisual Quality of Low-Bitrate Multimedia Content

paper_content:

This paper studies the quality of multimedia content at very low bitrates. We carried out subjective experiments for assessing audiovisual, audio-only, and video-only quality. We selected content and encoding parameters that are typical of mobile applications. Our focus were the MPEG-4 AVC (a.k.a. H.264) and AAC coding standards. Based on these data, we first analyze the influence of video and audio coding parameters on quality. We investigate the optimal trade-off between bits allocated to audio and to video under global bitrate constraints. Finally, we explore models for the interactions between audio and video in terms of perceived audiovisual quality
---
paper_title: DVQ: A digital video quality metric based on human vision

paper_content:

The growth of digital video has given rise to a need for computational methods for evaluating the visual quality of digital video. We have developed a new digital video quality metric, which we call DVQ (Digital Video Quality) 1 . Here we provide a brief description of the metric, and give a preliminary report on its performance. DVQ accepts a pair of digital video sequences, and computes a measure of the magnitude of the visible difference between them. The metric is based on the Discrete Cosine Transform. It incorporates aspects of early visual processing, including light adaptation, luminance and chromatic channels, spatial and temporal filtering, spatial frequency channels, contrast masking, and probability summation. It also includes primitive dynamics of light adaptation and contrast masking. We have applied the metric to digital video sequences corrupted by various typical compression artifacts, and compared the results to quality ratings made by human observers.
---
paper_title: Model of visual contrast gain control and pattern masking

paper_content:

We have implemented a model of contrast gain and control in human vision that incorporates a number of key features, including a contrast sensitivity function, multiple oriented bandpass channels, accelerating nonlinearities, and a devisive inhibitory gain control pool. The parameters of this model have been optimized through a fit to the recent data that describe masking of a Gabor function by cosine and Gabor masks [J. M. Foley, "Human luminance pattern mechanisms: masking experiments require a new model," J. Opt. Soc. Am. A 11, 1710 (1994)]. The model achieves a good fit to the data. We also demonstrate how the concept of recruitment may accommodate a variant of this model in which excitatory and inhibitory paths have a common accelerating nonlinearity, but which include multiple channels tuned to different levels of contrast.
---
paper_title: A new standardized method for objectively measuring video quality

paper_content:

The National Telecommunications and Information Administration (NTIA) General Model for estimating video quality and its associated calibration techniques were independently evaluated by the Video Quality Experts Group (VQEG) in their Phase II Full Reference Television (FR-TV) test. The NTIA General Model was the only video quality estimator that was in the top performing group for both the 525-line and 625-line video tests. As a result, the American National Standards Institute (ANSI) adopted the NTIA General Model and its associated calibration techniques as a North American Standard in 2003. The International Telecommunication Union (ITU) has also included the NTIA General Model as a normative method in two Draft Recommendations. This paper presents a description of the NTIA General Model and its associated calibration techniques. The independent test results from the VQEG FR-TV Phase II tests are summarized, as well as results from eleven other subjective data sets that were used to develop the method.
---
paper_title: Blind measurement of blocking artifacts in images

paper_content:

The objective measurement of blocking artifacts plays an important role in the design, optimization, and assessment of image and video coding systems. We propose a new approach that can blindly measure blocking artifacts in images without reference to the originals. The key idea is to model the blocky image as a non-blocky image interfered with a pure blocky signal. The task of the blocking effect measurement algorithm is then to detect and evaluate the power of the blocky signal. The proposed approach has the flexibility to integrate human visual system features such as the luminance and the texture masking effects.
---
paper_title: A reduced-reference perceptual quality metric for in-service image quality assessment

paper_content:

User-oriented image quality assessment has become a key factor in multimedia communications. However, existing image quality metrics such as peak signal-to-noise ratio (PSNR) are inappropriate for in-service quality monitoring, since they require the original image to be available at the receiver. Although PSNR and others are objective metrics, they are not based on human visual perception. In order to overcome these problems, we propose a novel reduced-reference objective hybrid image quality metric (HIQM) that accounts for human visual perception and does not require a reference image at the receiver. This approach is based on a combination of various image artifact measures. The result is a single number, which represents an overall image quality. Experimental results indicate that HIQM outperforms PSNR.
---
paper_title: Quality Evaluation in Wireless Imaging Using Feature-Based Objective Metrics

paper_content:

This paper addresses the evaluation of image quality in the context of wireless systems using feature-based objective metrics. The considered metrics comprise of a weighted combination of feature values that are used to quantify the extend by which the related artifacts are present in a processed image. In view of imaging applications in mobile radio and wireless communication systems, reduced-reference objective quality metrics are investigated for quantifying user-perceived quality. The examined feature-based objective metrics provide options for controlling the processing complexity and implementation resources required for performing the quality evaluation. Furthermore, the metrics support the scaling of overhead needed for communicating the image features over the wireless link and the accuracy for predicting the image quality
---

Title: Perceptual-based Quality Metrics for Image and Video Services: A Survey
Section 1: INTRODUCTION
Description 1: Introduce the importance of image and video quality assessment, the issues with traditional metrics, and the need for perceptual-based quality metrics. Provide an organization overview of the paper.

Section 2: Subjective and objective methods
Description 2: Describe subjective and objective methods for quality evaluation, emphasizing their differences and the challenges associated with each.

Section 3: Psychophysical and engineering approach
Description 3: Discuss the psychophysical and engineering approaches to designing objective quality metrics, including the incorporation of human visual system aspects and image analysis techniques.

Section 4: Reference-based classification
Description 4: Explain the classification of quality metrics based on the availability of reference information, including no-reference, reduced-reference, and full-reference methods.

Section 5: NO-REFERENCE QUALITY METRICS
Description 5: Provide a review of no-reference quality metrics, focusing on the challenges and methods for quality assessment without reference information.

Section 6: Single feature metrics
Description 6: Discuss metrics that measure specific artifacts such as blocking, blur, and ringing.

Section 7: Metrics of combined features and structural information
Description 7: Explore metrics that combine various features and structural information for a comprehensive quality assessment.

Section 8: Metrics incorporating codec parameter settings
Description 8: Describe metrics that utilize codec-specific parameters to facilitate quality prediction.

Section 9: Metrics using data hiding techniques
Description 9: Explain the use of data hiding and watermarking techniques for quality assessment.

Section 10: REDUCED-REFERENCE QUALITY ASSESSMENT
Description 10: Review reduced-reference quality assessment methods, detailing the features and the amount of overhead needed for transmitting the reference information.

Section 11: SUMMARY AND CONCLUSIONS
Description 11: Summarize the key findings of the survey, emphasize the need for scalable metrics for recent codecs, and suggest future research directions.