A SURVEY ON FULL REFERENCE IMAGE QUALITY ASSESSMENT ALGORITHMS
---
paper_title: Image information and visual quality

paper_content:

Measurement of visual quality is of fundamental importance to numerous image and video processing applications. The goal of quality assessment (QA) research is to design algorithms that can automatically assess the quality of images or videos in a perceptually consistent manner. Image QA algorithms generally interpret image quality as fidelity or similarity with a "reference" or "perfect" image in some perceptual space. Such "full-reference" QA methods attempt to achieve consistency in quality prediction by modeling salient physiological and psychovisual features of the human visual system (HVS), or by signal fidelity measures. In this paper, we approach the image QA problem as an information fidelity problem. Specifically, we propose to quantify the loss of image information to the distortion process and explore the relationship between image information and visual quality. QA systems are invariably involved with judging the visual quality of "natural" images and videos that are meant for "human consumption." Researchers have developed sophisticated models to capture the statistics of such natural signals. Using these models, we previously presented an information fidelity criterion for image QA that related image quality with the amount of information shared between a reference and a distorted image. In this paper, we propose an image information measure that quantifies the information that is present in the reference image and how much of this reference information can be extracted from the distorted image. Combining these two quantities, we propose a visual information fidelity measure for image QA. We validate the performance of our algorithm with an extensive subjective study involving 779 images and show that our method outperforms recent state-of-the-art image QA algorithms by a sizeable margin in our simulations. The code and the data from the subjective study are available at the LIVE website.
---
paper_title: Information Content Weighting for Perceptual Image Quality Assessment

paper_content:

Many state-of-the-art perceptual image quality assessment (IQA) algorithms share a common two-stage structure: local quality/distortion measurement followed by pooling. While significant progress has been made in measuring local image quality/distortion, the pooling stage is often done in ad-hoc ways, lacking theoretical principles and reliable computational models. This paper aims to test the hypothesis that when viewing natural images, the optimal perceptual weights for pooling should be proportional to local information content, which can be estimated in units of bit using advanced statistical models of natural images. Our extensive studies based upon six publicly-available subject-rated image databases concluded with three useful findings. First, information content weighting leads to consistent improvement in the performance of IQA algorithms. Second, surprisingly, with information content weighting, even the widely criticized peak signal-to-noise-ratio can be converted to a competitive perceptual quality measure when compared with state-of-the-art algorithms. Third, the best overall performance is achieved by combining information content weighting with multiscale structural similarity measures.
---
paper_title: Image quality assessment: from error visibility to structural similarity

paper_content:

Objective methods for assessing perceptual image quality traditionally attempted to quantify the visibility of errors (differences) between a distorted image and a reference image using a variety of known properties of the human visual system. Under the assumption that human visual perception is highly adapted for extracting structural information from a scene, we introduce an alternative complementary framework for quality assessment based on the degradation of structural information. As a specific example of this concept, we develop a structural similarity index and demonstrate its promise through a set of intuitive examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000. A MATLAB implementation of the proposed algorithm is available online at http://www.cns.nyu.edu//spl sim/lcv/ssim/.
---
paper_title: Image quality assessment: from error visibility to structural similarity

paper_content:

Objective methods for assessing perceptual image quality traditionally attempted to quantify the visibility of errors (differences) between a distorted image and a reference image using a variety of known properties of the human visual system. Under the assumption that human visual perception is highly adapted for extracting structural information from a scene, we introduce an alternative complementary framework for quality assessment based on the degradation of structural information. As a specific example of this concept, we develop a structural similarity index and demonstrate its promise through a set of intuitive examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000. A MATLAB implementation of the proposed algorithm is available online at http://www.cns.nyu.edu//spl sim/lcv/ssim/.
---
paper_title: Most apparent distortion: full-reference image quality assessment and the role of strategy

paper_content:

The mainstream approach to image quality assessment has centered around accurately modeling the single most relevant strategy employed by the human visual system (HVS) when judging image quality (e.g., detecting visible differences, and extracting image structure/information). In this work, we suggest that a single strategy may not be sufficient; rather, we advocate that the HVS uses multiple strategies to determine image quality. For images containing near-threshold distortions, the image is most apparent, and thus the HVS attempts to look past the image and look for the distortions (a detection-based strategy). For images containing clearly visible distortions, the distortions are most apparent, and thus the HVS attempts to look past the distortion and look for the image's subject matter (an appearance-based strategy). Here, we present a quality assessment method [most apparent distortion (MAD)], which attempts to explicitly model these two separate strategies. Local luminance and contrast masking are used to estimate detection-based perceived distortion in high-quality images, whereas changes in the local statistics of spatial-frequency components are used to estimate appearance-based perceived distortion in low-quality images. We show that a combination of these two measures can perform well in predicting subjective ratings of image quality.
---
paper_title: The similarity metric

paper_content:

A new class of distances appropriate for measuring similarity relations between sequences, say one type of similarity per distance, is studied. We propose a new "normalized information distance," based on the noncomputable notion of Kolmogorov complexity, and show that it is in this class and it minorizes every computable distance in the class (that is, it is universal in that it discovers all computable similarities). We demonstrate that it is a metric and call it the similarity metric . This theory forms the foundation for a new practical tool. To evidence generality and robustness, we give two distinctive applications in widely divergent areas using standard compression programs like gzip and GenCompress. First, we compare whole mitochondrial genomes and infer their evolutionary history. This results in a first completely automatic computed whole mitochondrial phylogeny tree. Secondly, we fully automatically compute the language tree of 52 different languages.
---
paper_title: Image distortion analysis based on normalized perceptual information distance

paper_content:

Image distortion analysis is a fundamental issue in many image processing problems, including compression, restoration, recognition, classification, and retrieval. Traditional image distortion evaluation approaches tend to be heuristic and are often limited to specific application environment. In this work, we investigate the problem of image distortion measurement based on the theory of Kolmogorov complexity, which has rarely been studied in the context of image processing. This work is motivated by the normalized information distance (NID) measure that has been shown to be a valid and universal distance metric applicable to similarity measurement of any two objects (Li et al. in IEEE Trans Inf Theory 50:3250–3264, 2004). Similar to Kolmogorov complexity, NID is non-computable. A useful practical solution is to approximate it using normalized compression distance (NCD) (Li et al. in IEEE Trans Inf Theory 50:3250–3264, 2004), which has led to impressive results in many applications such as construction of phylogeny trees using DNA sequences (Li et al. in IEEE Trans Inf Theory 50:3250–3264, 2004). In our earlier work, we showed that direct use of NCD on image processing problems is difficult and proposed a normalized conditional compression distance (NCCD) measure (Nikvand and Wang, 2010), which has significantly wider applicability than existing image similarity/distortion measures. To assess the distortions between two images, we first transform them into the wavelet transform domain. Assuming stationarity and good decorrelation of wavelet coefficients beyond local regions and across wavelet subbands, the Kolmogorov complexity may be approximated using Shannon entropy (Cover et al. in Elements of information theory. Wiley-Interscience, New York, 1991). Inspired by Sheikh and Bovik (IEEE Trans Image Process 15(2):430–444, 2006), we adopt a Gaussian scale mixture model for clusters of neighboring wavelet coefficients and a Gaussian channel model for the noise distortions in the human visual system. Combining these assumptions with the NID framework, we derive a novel normalized perceptual information distance measure, where maximal likelihood estimation and least square regression are employed for parameter fitting. We validate the proposed distortion measure using three large-scale, publicly available, and subject-rated image databases, which include a wide range of practical image distortion types and levels. Our results demonstrate the good prediction power of the proposed method for perceptual image distortions.
---
paper_title: MULTI-SCALE STRUCTURAL SIMILARITY FOR IMAGE QUALITY ASSESSMENT

paper_content:

The structural similarity image quality paradigm is based on the assumption that the human visual system is highly adapted for extracting structural information from the scene, and therefore a measure of structural similarity can provide a good approximation to perceived image quality. This paper proposes a multi-scale structural similarity method, which supplies more flexibility than previous single-scale methods in incorporating the variations of viewing conditions. We develop an image synthesis method to calibrate the parameters that define the relative importance of different scales. Experimental comparisons demonstrate the effectiveness of the proposed method.
---
paper_title: FSIM: A Feature Similarity Index for Image Quality Assessment

paper_content:

Image quality assessment (IQA) aims to use computational models to measure the image quality consistently with subjective evaluations. The well-known structural similarity index brings IQA from pixel- to structure-based stage. In this paper, a novel feature similarity (FSIM) index for full reference IQA is proposed based on the fact that human visual system (HVS) understands an image mainly according to its low-level features. Specifically, the phase congruency (PC), which is a dimensionless measure of the significance of a local structure, is used as the primary feature in FSIM. Considering that PC is contrast invariant while the contrast information does affect HVS' perception of image quality, the image gradient magnitude (GM) is employed as the secondary feature in FSIM. PC and GM play complementary roles in characterizing the image local quality. After obtaining the local quality map, we use PC again as a weighting function to derive a single quality score. Extensive experiments performed on six benchmark IQA databases demonstrate that FSIM can achieve much higher consistency with the subjective evaluations than state-of-the-art IQA metrics.
---
paper_title: Practical Image Quality Metric Applied to Image Coding

paper_content:

Perceptual image coding requires an effective image quality metric, yet most of the existing metrics are complex and can hardly guide the compression effectively. This paper proposes a practical full-reference metric with consideration of the texture masking effect and contrast sensitivity function. The metric is capable of evaluating typical image impairments in real-world applications and can achieve the comparable performance as the state-of-the-art metrics on the publicly available subjectively-rated image databases. Due to its simplicity, the metric is embedded into JPEG image coding to ensure a better perceptual rate-distortion performance.
---
paper_title: ON BETWEEN-COEFFICIENT CONTRAST MASKING OF DCT BASIS FUNCTIONS

paper_content:

In this paper we propose a simple and effective model of visual between-coefficient contrast masking of DCT basis functions based on a human visual system (HVS). The model operates with the values of DCT coefficients of 8x8 pixel block of an image. For each DCT coefficient of the block the model allows to calculate its maximal distortion that is not visible due to the between-coefficient masking. A modification of the PSNR is also described in this paper. The proposed metric, PSNR-HVS-M, takes into account the proposed model and the contrast sensitivity function (CSF). For efficiency analysis of the proposed model, a set of 18 test images with different effects of noise masking has been used. During experiments, 155 observers have sorted this set of test images in the order of their visual appearance comparing them to undistorted original. The new metric, PSNRHVS-M has outperformed other well-known reference based quality metrics and demonstrated high correlation with the results of subjective experiments (Spearman correlation is 0.984, Kendall correlation is 0.948).
---
paper_title: Information Content Weighting for Perceptual Image Quality Assessment

paper_content:

Many state-of-the-art perceptual image quality assessment (IQA) algorithms share a common two-stage structure: local quality/distortion measurement followed by pooling. While significant progress has been made in measuring local image quality/distortion, the pooling stage is often done in ad-hoc ways, lacking theoretical principles and reliable computational models. This paper aims to test the hypothesis that when viewing natural images, the optimal perceptual weights for pooling should be proportional to local information content, which can be estimated in units of bit using advanced statistical models of natural images. Our extensive studies based upon six publicly-available subject-rated image databases concluded with three useful findings. First, information content weighting leads to consistent improvement in the performance of IQA algorithms. Second, surprisingly, with information content weighting, even the widely criticized peak signal-to-noise-ratio can be converted to a competitive perceptual quality measure when compared with state-of-the-art algorithms. Third, the best overall performance is achieved by combining information content weighting with multiscale structural similarity measures.
---
paper_title: MULTI-SCALE STRUCTURAL SIMILARITY FOR IMAGE QUALITY ASSESSMENT

paper_content:

The structural similarity image quality paradigm is based on the assumption that the human visual system is highly adapted for extracting structural information from the scene, and therefore a measure of structural similarity can provide a good approximation to perceived image quality. This paper proposes a multi-scale structural similarity method, which supplies more flexibility than previous single-scale methods in incorporating the variations of viewing conditions. We develop an image synthesis method to calibrate the parameters that define the relative importance of different scales. Experimental comparisons demonstrate the effectiveness of the proposed method.
---

Title: A SURVEY ON FULL REFERENCE IMAGE QUALITY ASSESSMENT ALGORITHMS

Section 1: INTRODUCTION
Description 1: Introduce the need for accurate image quality assessment, the types of distortion in images, and the classification of image quality assessment methods into full reference, reduced reference, and no reference.

Section 2: FULL REFERENCE IMAGE QUALITY ASSESSMENT ALGORITHMS
Description 2: Provide an overview of various full reference image quality assessment algorithms proposed by researchers and the performance metrics used for comparison.

Section 3: SSIM (Structural Similarity Index Measure)
Description 3: Discuss the SSIM algorithm, its basis on human visual system adaptation for extracting structural information, and its implementation.

Section 4: MAD (Most Apparent Distortion)
Description 4: Explain the MAD algorithm, its dependence on human visual system properties for judging image quality, and the combination of detection-based and appearance-based strategies.

Section 5: MS-SSIM (Multi Scale Structural Similarity Index Measure)
Description 5: Describe the MS-SSIM algorithm, its flexibility over single scale methods, and its process involving low-pass filtering and down sampling.

Section 6: FSIM (Feature Similarity Index)
Description 6: Detail the FSIM algorithm, focusing on low-level features such as phase congruency and gradient magnitude, and its applicability to both grayscale and color images.

Section 7: Practical Image Quality Index
Description 7: Outline the practical image quality index metric based on contrast sensitivity function and texture masking effect, and its easy integration into the JPEG compression standard.

Section 8: Content Partitioned Structural Similarity Index
Description 8: Discuss the content partitioned similarity index, a modified version of SSIM and MS-SSIM, which divides images into four component regions for quality assessment.

Section 9: PERFORMANCE COMPARISON METHODS
Description 9: Explain the methods for comparing the performance of image quality assessment algorithms, the use of correlation coefficients and mean squared error, and the importance of mean opinion scores.

Section 10: CONCLUSIONS
Description 10: Summarize the survey, listing the various full reference image quality assessment algorithms, their merits and demerits, and suggestions on selecting algorithms based on accuracy, time, and other requirements.