An ensemble image quality assessment algorithm based on deep feature clustering

To effective handle image quality assessment (IQA) where the images might be with sophisticated characteristics, we proposed a deep clustering-based ensemble approach for image quality assessment toward diverse images. Our approach is based on a convolutional DAE-aware deep architecture. By leveraging a layer-by-layer pre-training, our proposed deep feature clustering architecture extracted a fixed number of high-level features at first. Then, it optimally splits image samples into different clusters by using the fuzzy C-means algorithm based on the engineered deep features. For each cluster, we simulated a particular fitting function of differential mean opinion scores with each assessed image’s PSNR, SIMM, and VIF scores. Comprehensive experimental results on TID2008, TID2013 and LIVE databases have demonstrated that compared to the state-of-the-art counterparts, our proposed IQA method can reflect the subjective quality of images more accurately by seamlessly integrating the advantages of three existed IQA methods.     

No-reference video quality evaluation by a deep transfer CNN architecture

The standard no-reference video quality assessment (NR-VQA) is designed for a specific type of distortion. It quantifies the visual quality of a distorted video without the reference one. Practically, there is a deviation between the result of NR-VQA and human subjective perception. To tackle this problem, we propose a 3D deep convolutional neural network (3D CNN) to evaluate video quality without reference by generating spatial/temporal deep features within different video clips 3D CNN is designed by collaboratively and seamlessly integrating the features output from VGG-Net on video frames. To prevent our adopted VGG-Net from overfitting, the parameters are transferred from the deep architecture learned from the ImageNet dataset. Extensive IQA/VQA experimental results based on the LIVE, TID, and the CSIQ video quality databases have demonstrated that the proposed IQA/VQA model performs competitively the conventional methods.     

No-reference image quality assessment based on spatial and spectral entropies

We develop an efficient general-purpose no-reference (NR) image quality assessment (IQA) model that utilizes local spatial and spectral entropy features on distorted images. Using a 2-stage framework of distortion classification followed by quality assessment, we utilize a support vector machine (SVM) to train an image distortion and quality prediction engine. The resulting algorithm, dubbed Spatial–Spectral Entropy-based Quality (SSEQ) index, is capable of assessing the quality of a distorted image across multiple distortion categories. We explain the entropy features used and their relevance to perception and thoroughly evaluate the algorithm on the LIVE IQA database. We find that SSEQ matches well with human subjective opinions of image quality, and is statistically superior to the full-reference (FR) IQA algorithm SSIM and several top-performing NR IQA methods: BIQI, DIIVINE, and BLIINDS-II. SSEQ has a considerably low complexity. We also tested SSEQ on the TID2008 database to ascertain whether it has performance that is database independent.     

Reduced-reference image quality assessment using moment method

Reduced-reference image quality assessment (RR IQA) aims to evaluate the perceptual quality of a distorted image through partial information of the corresponding reference image. In this paper, a novel RR IQA metric is proposed by using the moment method. We claim that the first and second moments of wavelet coefficients of natural images can have approximate and regular change that are disturbed by different types of distortions, and that this disturbance can be relevant to human perceptions of quality. We measure the difference of these statistical parameters between reference and distorted image to predict the visual quality degradation. The introduced IQA metric is suitable for implementation and has relatively low computational complexity. The experimental results on Laboratory for Image and Video Engineering (LIVE) and Tampere Image Database (TID) image databases indicate that the proposed metric has a good predictive performance.     

Content-weighted mean-squared error for quality assessment of compressed images

Image quality assessment (IQA) has been intensively studied, especially for the full-reference (FR) scenario. However, only the mean-squared error (MSE) is widely employed in compression. Why other IQA metrics work ineffectively? We first sum up three main limitations including the computational time, portability, and working manner. To address these problems, we then in this paper propose a new content-weighted MSE (CW-MSE) method to assess the quality of compressed images. The design principle of our model is to use adaptive Gaussian convolution to estimate the influence of image content in a block-based manner, thereby to approximate the human visual perception to image quality. Results of experiments on six popular subjective image quality databases (including LIVE, TID2008, CSIQ, IVC, Toyama and TID2013) confirm the superiority of our CW-MSE over state-of-the-art FR IQA approaches.     

No-reference image quality assessment based on non-subsample shearlet transform and natural scene statistics

A novel no-reference (NR) image quality assessment (IQA) method is proposed for assessing image quality across multifarious distortion categories. The new method transforms distorted images into the shearlet domain using a non-subsample shearlet transform (NSST), and designs the image quality feature vector to describe images utilizing natural scenes statistical features:coefficient distribution, energy distribution and structural correlation (SC) across orientations and scales. The final image quality is achieved from distortion classification and regression models trained by a support vector machine (SVM). The experimental results on the LIVE2 IQA database indicate that the method can assess image quality effectively, and the extracted features are susceptive to the category and severity of distortion. Furthermore, our proposed method is database independent and has a higher correlation rate and lower root mean squared error (RMSE) with human perception than other high performance NR IQA methods.     

A weighted full-reference image quality assessment based on visual saliency

In full reference image quality assessment (IQA), the images without distortion are usually employed as reference, while the structures in both reference images and distorted images are ignored and all pixels are equally treated. In addition, the role of human visual system (HVS) is not taken account into subjective IQA metric. In this paper, a weighted full-reference image quality metric is proposed, where a weight imposed on each pixel indicates its importance in IQA. Furthermore, the weights can be estimated via visual saliency computation, which can approximate the subjective IQA via exploiting the HVS. In the experiments, the proposed metric is compared with several objective IQA metrics on LIVE release 2 and TID 2008 database. The results demonstrate that SROCC and PLCC of the proposed metric are 0.9647 and 0.9721, respectively,which are higher than other methods and it only takes 427.5 s, which is lower than that of most other methods.     

AdaBoost neural network and cyclopean view for no-reference stereoscopic image quality assessment

Stereoscopic imaging is widely used in many fields. In many scenarios, stereo images quality could be affected by various degradations, such as asymmetric distortion. Accordingly, to guarantee the best quality of experience, robust and accurate reference-less metrics are required for quality assessment of stereoscopic content. Most existing stereo no-reference Image Quality Assessment (IQA) models are not consistent with asymmetrical distortions. This paper presents a new no-reference stereoscopic image quality assessment metric using a human visual system (HVS) modeling and an advanced machine-learning algorithm. The proposed approach consists of two stages. In the first stage, cyclopean image is constructed considering the presence of binocular rivalry in order to cover the asymmetrically distorted part. In the second stage, gradient magnitude, relative gradient magnitude, and gradient orientation are extracted. These are used as a predictive source of information for the quality. In order to obtain the best overall performance against different databases, Adaptive Boosting (AdaBoost) idea of machine learning combined with artificial neural network model has been adopted. The benchmark LIVE 3D phase-I, phase-II, and IRCCyN/IVC 3D databases have been used to evaluate the performance of the proposed approach. Experimental results have demonstrated that the proposed metric performance achieves high consistency with subjective assessment and outperforms the blind stereo IQA over various types of distortion.     

Toward an unsupervised blind stereoscopic 3D image quality assessment using joint spatial and frequency representations

Existing blind stereoscopic 3D (S3D) image quality assessment (IQA) metrics usually require supervised learning methods to predict S3D image quality, which limits their applicability in practice. In this paper, we propose an unsupervised blind S3D IQA metric that utilizes the joint spatial and frequency representations of visual perception. The metric proposed in this work was inspired by the binocular visual mechanism; furthermore, it is unsupervised and does not require subject-rated samples for training. To be more specific, first, the various binocular quality-aware features in spatial and frequency domains are extracted from the monocular and cyclopean views of natural S3D image patches. Subsequently, these features are utilized to establish a pristine multivariate Gaussian (MVG) model to characterize natural S3D image regularities. Finally, with the learned MVG model, the final quality score for a distorted S3D image can be yielded using a Bhattacharyya-like distance. Our experimental results illustrate that, compared to related existing metrics, the devised metric achieves competitive prediction performance.     

一个图像质量盲评估的统计测度

本文提出了一个图像质量盲评估的统计测度.该测度首先根据自然图像的统计性质与失真图像的模型,实现对图像小波系数分布参数的盲估计;再利用估计的分布参数来计算失真图像与参考图像之间的互信息,以量化失真图像对参考图像的保真度,进而实现对图像质量的评估.本文提出的测度避免了对参考图像的依赖,且克服了现有图像质量盲评估对特征选择与提取、机器学习等过程的依赖.LIVE图像质量评估数据库的总体评估结果表明：本文提出的盲评估统计测度对图像质量评估结果与数据库的主观评估结果高度一致,且优于文献中报道的盲评估测度.     

基于支持向量回归的无参考模糊和噪声图像质量评价方法

基于支持向量回归(SVR)和图像奇异值分解,提出了一种新的无参考(NR,no-reference)模糊和噪声图像质量评价(IQA)方法。首先通过对待评价图像进行高斯低通滤波生成再模糊图像,然后分别对它们进行奇异值分解并计算奇异值的改变量,最后使用奇异值的改变量作为SVR的输入,训练预并测得到图像的质量评分。在3个公开的模糊和噪声数据库上的实验结果表明,新方法预测得分与主观得分有较好的一致性,获得了较好的评价指标;对于模糊失真类型和噪声失真类型,在LIVE2数据库上的性能评价指标斯皮尔曼等级相关系数(SROCC)分别达到0.961 3和0.965 9。     

No-reference stereoscopic image quality evaluator based on human visual characteristics and relative gradient orientation

Stereoscopic image quality assessment (SIQA) is of great significance to the development of modern three-dimensional (3D) display technology. In this work, by further mining the relationship between visual features and stereoscopic image quality perception, we build a new no-reference SIQA model, which combines the monocular and binocular features. Statistical quality-aware structural features from relative gradient orientation (RGO) map and texture features from the histogram of the weighted local binary pattern (LBP) in the texture image (TLBP) are not only extracted from both monocular view, but also extracted from binocular views to predict binocular quality perception. Meanwhile, the color statistical features ignored by most models and the binocularity feature is extracted to complement the monocular features and the above binocular features, respectively. Finally, all the extracted features and subjective scores are used to predict the objective quality score through the support vector regression (SVR) model. Experiments on four popular stereoscopic image databases show that the proposed model achieves high consistency with subjective assessment, and the performance of the model is very competitive with the latest models.     

Non-reference image quality assessment based on deep clustering

Image quality assessment (IQA) is an indispensable technique in computer vision and pattern recognition Existing deep IQA methods have achieved remarkable performance. As far as we know, these deep learning-based IQA algorithms lack an adaptive features extraction mechanism toward input images with varying sizes and the stability in avoiding disturbance from data noises and model deviation. To solve these problems, we propose a non-reference IQA method by designing a novel unsupervised deep clustering framework, where a 13-layer network structure is proposed that upgrades the fully-connected layers to produce high-level features with adaptive sizes. Moreover, we add a contracted regular term with a contracted autoencoder into the clustering loss function to form a quality model reflecting the clustering structure. Compared to the other IQA algorithms, our model with simple structure exhibits more stable and robust performance by the initial configuration of network parameters during end-to-end training. The experimental results on the LIVE and CSIP databases have shown that our method not only performs better than the state-of-the-art IQA algorithms, but also has a simpler structure and better adaptability.     

基于非局部梯度的图像质量评价算法

图像质量评价研究的目标在于模拟人类视觉系统对图像质量的感知过程,构建与主观评价结果尽可能一致的客观评价算法。现有的很多算法都是基于局部结构相似设计的,但人对图像的主观感知是高级的、语义的过程,而语义信息本质上是非局部的,因此图像质量评价应该考虑图像的非局部信息。该文突破了经典的基于局部信息的算法框架,提出一种基于非局部信息的框架,并在此框架内构建了一种基于非局部梯度的图像质量评价算法,该算法通过度量参考图像与失真图像的非局部梯度之间的相似性来预测图像质量。在公开测试数据库TID2008, LIVE, CSIQ上的数值实验结果表明,该算法能获得较好的评价效果。     

No-reference image quality assessment using statistical characterization in the shearlet domain

Image and video quality measurements are crucial for many applications, such as acquisition, compression, transmission, enhancement, and reproduction. Nowadays, no-reference (NR) image quality assessment (IQA) methods have drawn extensive attention because it does not rely on any information of original images. However, most of the conventional NR-IQA methods are designed only for one or a set of predefined specific image distortion types, which are unlikely to generalize for evaluating image/video distorted with other types of distortions. In order to estimate a wide range of image distortions, in this paper, we present an efficient general-purpose NR-IQA algorithm which is based on a new multiscale directional transform (shearlet transform) with a strong ability to localize distributed discontinuities. This is mainly based on distorted natural image that leads to significant variation in the spread discontinuities in all directions. Thus, the statistical property of the distorted image is significantly different from that of natural images in fine scale shearlet coefficients, which are referred to as ‘distorted parts’. However, some ‘natural parts’ are reserved in coarse scale shearlet coefficients. The algorithm relies on utilizing the natural parts to predict the natural behavior of distorted parts. The predicted parts act as ‘reference’ and the difference between the reference and distorted parts is used as an indicator to predict the image quality. In order to achieve this goal, we modify the general sparse autoencoder to serve as a predictor to get the predicted parts from natural parts. By translating the NR-IQA problem into classification problem, the predicted parts and distorted parts are utilized to form features and the differences between them are identified by softmax classifier. The resulting algorithm, which we name SHeArlet based No-reference Image quality Assessment (SHANIA), is tested on several database (LIVE, Multiply Distorted LIVE and TID2008) and shown to be suitable for many common distortions, consistent with subjective assessment and comparable to full-reference IQA methods and state-of-the-art general purpose NR-IQA algorithms.     

BGT: A blind image quality evaluator via gradient and texture statistical features

Blind image quality assessment (BIQA) aims to design a model that can accurately evaluate the quality of the distorted image without any information about its reference image. Previous studies have shown that gradients and textures of image is widely used in image quality evaluation tasks. However, few studies used the joint statistics of gradient and texture information to evaluate image quality. Considering the visual perception characteristics of the human visual system, we develop a novel general-purpose BIQA model via two sets of complementary perception features. Specifically, the joint statistical histograms of gradient and texture are extracted as the first set of features, and the second set of features is extracted using the local binary pattern (LBP) operator. After extracting two groups of complementary quality-aware features, the feature vectors are sent to the support vector regression machine to establish the nonlinear relationship between quality-aware features and quality scores. A large number of experiments on seven large benchmark databases show that the proposed BIQA model has higher accuracy, better generalization properties and lower computational complexity than the relevant state-of-the-art BIQA metrics.     

基于LBP特征和贝叶斯模型的单样本人脸识别

针对单样本人脸识别这一人脸识别中的难点问题,提出了一种基于局部二元模式(LBP)直方图特征和贝叶斯(Bayes)模型的人脸识别方法.首先在独立的训练集上学习同类样本和异类样本的LBP直方图特征的相似度先验信息,估计同类和异类的类条件概率密度函数,在识别过程中利用一对图像的LBP直方图相似度计算该对图像属于同一类的后验概...