基于深层特征学习的无参考立体图像质量评价

提出了一种基于深层特征学习的无参考(NR)立体图 像质量评价方 法。与传统人工提取图像特征不同,采用卷积神经网络(CNN)自动提取图像特征,评价过程 分为训练和 测试两阶段。在训练阶段,将图像分块训练CNN网络,利用CNN提取图像块特征,并结合不同 的整合方式 得到图像的全局特征,通过支持向量回归(SVR)建立主观质量与全局特征的回归模型;在测 试阶段,由已训练的CNN网 络和回归模型,得到左右图像和独眼图的质量。最后,根据人眼双目视觉特性融合左图像、 右图像和独眼 图的质量,得到立体图像质量。本文方法在LIVE-I和LIVE-II数据库上的Spearman等级系 数(SROCC)分别达 到了0.94,评价结果准确,与人眼的主 观感受一致。     

MuRNet: A deep recursive network for super resolution of bicubically interpolated images

In many real-world cases such as printer devices and in-camera interpolation, only the interpolated versions of the low-resolution images are available. In this paper, a new low-complexity high-performance image super resolution network is proposed that starting from the bicubic interpolated version of the low resolution image produces a high quality super resolved image. The main idea in the proposed scheme is the development of a feature generating block that is capable of producing features using multiple local spatial ranges and multiple resolution levels, fusing them in order to provide a rich set of feature maps, and using them in a recursive framework. The objective in designing such a recursive block is not simply to provide a light-weight network, as is traditionally done in the design of such a network, but also to provide a low count on the number of multiply-accumulate operations with high performance. The experimental results are provided to show that the proposed network outperforms other recursive super resolution networks when their super resolution capability, the number of parameters and number of multiply-accumulate operations are simultaneously taken into consideration.     

基于广义回归神经网络无参考模糊图像质量评价

提出一种基于广义回归神经网络的无参考模糊图像质量评价方法.该方法首先通过相位一致变换生成待评测图像的相位一致图像,然后利用灰度共生矩阵计算相位一致图像的信息熵、能量、对比度、相关性和同质性5个特征,最后利用广义回归神经网络模型训练学习,预测得到无参考模糊图像质量得分.在3个公开数据库模糊图像上的实验结果表明,新方法预测得分与主观得分有较好的一致性,更加符合人类视觉特性.     

基于深度学习网络的可见光图像重构质量评价研究

可见光图像重构质量评价一直是一个难点,因此,设计了基于深度学习网络的可见光图像重构质量评价方法.通过卷积神经网络(CNN)与图像质量评价方法(IQA)相结合,构成IQA-CNN模型,引入信息熵构建改进IQA-CNN模型,向该模型内输入重构可见光图像,归一化预处理后划分成数个分块,经有监督学习法训练该模型后,获取到该模型...     

Screen content image quality assessment based on convolutional neural networks

Screen content image (SCI) is a composite image including textual and pictorial regions resulting in many difficulties in image quality assessment (IQA). Large SCIs are divided into image patches to increase training samples for CNN training of IQA model, and this brings two problems: (1) local quality of each image patch is not equal to subjective differential mean opinion score (DMOS) of an entire image; (2) importance of different image patches is not same for quality assessment. In this paper, we propose a novel no-reference (NR) IQA model based on the convolutional neural network (CNN) for assessing the perceptual quality of SCIs. Our model conducts two designs solving problems which benefits from two strategies. For the first strategy, to imitate full-reference (FR) CNN-based model behavior, a CNN-based model is designed for both FR and NR IQA, and performance of NR-IQA part improves when the image patch scores predicted by FR-IQA part are adopted as the ground-truth to train NR-IQA part. For the second strategy, image patch qualities of one entire SCI are fused to obtain the SCI quality with an adaptive weighting method taking account the effect of the different image patch contents. Experimental results verify that our model outperforms all test NR IQA methods and most FR IQA methods on the screen content image quality assessment database (SIQAD). On the cross-database evaluation, the proposed method outperforms the existing NR IQA method in terms of at least 2.4 percent in PLCC and 2.8 percent in SRCC, which shows high generalization ability and high effectiveness of our model.     

Image quality assessment based on deep learning with FPGA implementation

To improve image quality assessment (IQA) methods, it is believable that we have to extract image features that are highly representative to human visual perception. In this paper, we propose a novel IQA algorithm by leveraging an optimized convolutional neural network architecture that is designed to automatically extract discriminative image quality features. And the IQA algorithm uses local luminance coefficient normalization, dropout and the other advanced techniques to further improve the network learning ability. At the same time the proposed IQA algorithm is implemented based on Field Programmable Gate Array (FPGA) and further evaluated on two public databases. Extensive experimental results have shown that our method outperforms many existing IQA algorithms in terms of accuracy and speed.     

Learning to predict the quality of distorted-then-compressed images via a deep neural network

Being captured by amateur photographers, reciprocally propagated through multimedia pipelines, and compressed with different levels, real-world images usually suffer from a wide variety of hybrid distortions. Faced with this scenario, full-reference (FR) image quality assessment (IQA) algorithms can not deliver promising predictions due to the inferior references. Meanwhile, existing no-reference (NR) IQA algorithms remain limited in their efficacy to deal with different distortion types. To address this obstacle, we explore a NR-IQA metric by predicting the perceptual quality of distorted-then-compressed images using a deep neural network (DNN). First, we propose a novel two-stream DNN to handle both authentic distortions and synthetic compressions and adopt effective strategies to pre-train the two branches of the network. Specifically, we transfer the knowledge learned from in-the-wild images to account for authentic distortions by utilizing a pre-trained deep convolutional neural network (CNN) to provide meaningful initializations. Meanwhile, we build a CNN for synthetic compressions and pre-train it on a dataset including synthetic compressed images. Subsequently, we bilinearly pool these two sets of features as the image representation. The overall network is fine-tuned on an elaborately-designed auxiliary dataset, which is annotated by a reliable objective quality metric. Furthermore, we integrate the output of the authentic-distortion-aware branch with that of the overall network following a two-step prediction manner to boost the prediction performance, which can be applied in the distorted-then-compressed scenario when the reference image is available. Extensive experimental results on several databases especially on the LIVE Wild Compressed Picture Quality Database show that the proposed method achieves state-of-the-art performance with good generalizability and moderate computational complexity.     

Research on image feature extraction and retrieval algorithms based on convolutional neural network

With the rapid development of mobile Internet and digital technology, people are more and more keen to share pictures on social networks, and online pictures have exploded. How to retrieve similar images from large-scale images has always been a hot issue in the field of image retrieval, and the selection of image features largely affects the performance of image retrieval. The Convolutional Neural Networks (CNN), which contains more hidden layers, has more complex network structure and stronger ability of feature learning and expression compared with traditional feature extraction methods. By analyzing the disadvantage that global CNN features cannot effectively describe local details when they act on image retrieval tasks, a strategy of aggregating low-level CNN feature maps to generate local features is proposed. The high-level features of CNN model pay more attention to semantic information, but the low-level features pay more attention to local details. Using the increasingly abstract characteristics of CNN model from low to high. This paper presents a probabilistic semantic retrieval algorithm, proposes a probabilistic semantic hash retrieval method based on CNN, and designs a new end-to-end supervised learning framework, which can simultaneously learn semantic features and hash features to achieve fast image retrieval. Using convolution network, the error rate is reduced to 14.41% in this test set. In three open image libraries, namely Oxford, Holidays and ImageNet, the performance of traditional SIFT-based retrieval algorithms and other CNN-based image retrieval algorithms in tasks are compared and analyzed. The experimental results show that the proposed algorithm is superior to other contrast algorithms in terms of comprehensive retrieval effect and retrieval time.     

Deep network based stereoscopic image quality assessment via binocular summing and differencing

With the development of deep networks in dealing with various visual tasks, the deep network based on binocular vision is expected to tackle the issue of stereoscopic image quality assessment. Here, we present a stereoscopic image quality assessment method using the deep network with four channels together, which takes the left view, right view, binocular summing view, and binocular differencing view as the inputs of the network. The visual features are enhanced through the concatenation in a weighted way, so that the binocular vision can be adequately included in the binocular addition and subtraction information. Compared with the state-of-the-art metrics, the proposed method exhibits relatively high performances on four benchmark databases.     

Low-light image enhancement via deep Retinex decomposition and bilateral learning

Low-light images enhancement is a challenging task because enhancing image brightness and reducing image degradation should be considered simultaneously. Although existing deep learning-based methods improve the visibility of low-light images, many of them tend to lose details or sacrifice naturalness. To address these issues, we present a multi-stage network for low-light image enhancement, which consists of three sub-networks. More specifically, inspired by the Retinex theory and the bilateral grid technique, we first design a reflectance and illumination decomposition network to decompose an image into reflectance and illumination maps efficiently. To increase the brightness while preserving edge information, we then devise an attention-guided illumination adjustment network. The reflectance and the adjusted illumination maps are fused and refined by adversarial learning to reduce image degradation and improve image naturalness. Experiments are conducted on our rebuilt SICE low-light image dataset, which consists of 1380 real paired images and a public dataset LOL, which has 500 real paired images and 1000 synthetic paired images. Experimental results show that the proposed method outperforms state-of-the-art methods quantitatively and qualitatively.     

No-reference image quality assessment using structural activity

Presuming that human visual perception is highly sensitive to the structural information in a scene, we propose the concept of structural activity (SA) together with a model of SA indicator in a new framework for no-reference (NR) image quality assessment (QA) in this study. The proposed framework estimates image quality based on the quantification of the SA information of different visual significance. We propose some alternative implementations of SA indicator in this paper as examples to demonstrate the effectiveness of the SA-motivated framework. Comprehensive testing demonstrates that the model of SA indicator exhibits satisfactory performance in comparison with subjective quality scores as well as representative full-reference (FR) image quality measures.     

一种使用边缘增强技术提高相似图片检索召回率的方法

针对大规模图像分类处理中图像旋转或背景变换导致的配准度较低问题,提出一种基于边缘增强的卷积神经网络图像分类方法。该方法通过VGG19网络模型提取图像特征,并使用余弦相似度进行图像分类判定,利用边缘增强突出图像主体的边缘特征,降低图像旋转或背景变换对VGG19网络分类性能带来的影响。实验证明,该方法可以有效地提高同一主体旋转图像和背景变换图像与原始图像的相似度,适用于各类图像的分类。     

SRNHARB: A deep light-weight image super resolution network using hybrid activation residual blocks

The quality of the images in all image-based applications and specially in computer vision applications is very crucial. Hence, design of a light-weight high-performance image super resolution scheme that enhances the quality of the acquired images is crucial for satisfactory functioning of such applications. Design of most of image super resolution schemes ignore extracting and processing of the negative-valued features of the images. In this paper, a novel light-weight residual block, which efficiently extracts and processes both the positive and negative-valued features, is proposed. This new residual block is capable of producing a richer set of features in order to improve the super resolution performance of the network using a set of such blocks. The network using the new residual blocks is shown to yield a performance superior to those of the existing light-weight super resolution networks using other types of residual blocks.     

An ensemble image quality assessment algorithm based on deep feature clustering

To effective handle image quality assessment (IQA) where the images might be with sophisticated characteristics, we proposed a deep clustering-based ensemble approach for image quality assessment toward diverse images. Our approach is based on a convolutional DAE-aware deep architecture. By leveraging a layer-by-layer pre-training, our proposed deep feature clustering architecture extracted a fixed number of high-level features at first. Then, it optimally splits image samples into different clusters by using the fuzzy C-means algorithm based on the engineered deep features. For each cluster, we simulated a particular fitting function of differential mean opinion scores with each assessed image’s PSNR, SIMM, and VIF scores. Comprehensive experimental results on TID2008, TID2013 and LIVE databases have demonstrated that compared to the state-of-the-art counterparts, our proposed IQA method can reflect the subjective quality of images more accurately by seamlessly integrating the advantages of three existed IQA methods.     

基于卷积神经网络和注意力机制的图像检索

当前先进的图像检索方法中,存在着不能很好地分辨图像中不同区域和内容的重要性的问题,导致计算资源分配不合理、检索正确率较低等一系列结果.为了解决这些问题,提出了一种基于卷积神经网络(Convolutional Neural Network,CNN)和注意力机制的图像检索方法.首先使用卷积神经网络提取特征,然后使用注意力机...     

Massive-scale image retrieval based on deep visual feature representation

This paper proposes an image retrieval algorithm towards massive-scale multimedia data. In order to be consistent with human visual system, we first design a color attention function to describe the important of different image patches. Subsequently, we combine color and texture to construct candidate regions, which will be fed into a deep neural network (DNN) for deep representation extraction. Then, we design a similarity function to calculate the distance among different images, where top-ranking images are considered as the required images. Experimental results show the effectiveness and robustness of our proposed method.     

A brief survey on adaptive video streaming quality assessment

Quality of experience (QoE) assessment for adaptive video streaming plays a significant role in advanced network management systems. It is especially challenging in case of dynamic adaptive streaming schemes over HTTP (DASH) which has increasingly complex characteristics including additional playback issues. In this paper, we provide a brief overview of adaptive video streaming quality assessment. Upon our review of related works, we analyze and compare different variations of objective QoE assessment models with or without using machine learning techniques for adaptive video streaming. Through the performance analysis, we observe that hybrid models perform better than both quality-of-service (QoS) driven QoE approaches and signal fidelity measurement. Moreover, the machine learning-based model slightly outperforms the model without using machine learning for the same setting. In addition, we find that existing video streaming QoE assessment models still have limited performance, which makes it difficult to be applied in practical communication systems. Therefore, based on the success of deep learned feature representations for traditional video quality prediction, we also apply the off-the-shelf deep convolutional neural network (DCNN) to evaluate the perceptual quality of streaming videos, where the spatio-temporal properties of streaming videos are taken into consideration. Experiments demonstrate its superiority, which sheds light on the future development of specifically designed deep learning frameworks for adaptive video streaming quality assessment. We believe this survey can serve as a guideline for QoE assessment of adaptive video streaming.     

Research on the parallelization of image quality analysis algorithm based on deep learning

Image quality assessment is an indispensable in computer vision applications, such as image classification, image parsing. With the development of Internet, image data acquisition becomes more conveniently. However, image distortion is inevitable due to imperfect image acquisition system, image transmission medium and image recording equipment. Traditional image quality assessment algorithms only focus on low-level visual features such as color or texture, which could not encode high-level features effectively. CNN-based methods have shown satisfactory results in image quality assessment. However, existing methods have problems such as incomplete feature extraction, partial image block distortion, and inability to determine scores. So in this paper, we propose a novel framework for image quality assessment based on deep learning. We incorporate both low-level visual features and high-level semantic features to better describe images. And image quality is analyzed in a parallel processing mode. Experiments are conducted on LIVE and TID2008 datasets demonstrate the proposed model can predict the quality of the distorted image well, and both SROCC and PLCC can reach 0.92 or higher.     

An optimized CNN-based quality assessment model for screen content image

Most existing convolutional neural network (CNN) based models designed for natural image quality assessment (IQA) employ image patches as training samples for data augmentation, and obtain final quality score by averaging all predicted scores of image patches. This brings two problems when applying these methods for screen content image (SCI) quality assessment. Firstly, SCI contains more complex content compared to natural image. As a result, qualities of SCI patches are different, and the subjective differential mean opinion score (DMOS) is not appropriate as qualities of all image patches. Secondly, the average score of image patches does not represent the quality of entire SCI since the human visual system (HVS) is sensitive to image patches containing texture and edge information. In this paper, we propose a novel quadratic optimized model based on the deep convolutional neural network (QODCNN) for full-reference (FR) and no-reference (NR) SCI quality assessment to overcome these two problems. The contribution of our algorithm can be concluded as follows: 1) Considering the characteristics of SCIs, a valid network architecture is designed for both NR and FR visual quality evaluation of SCIs, which makes the networks learn the feature differences for FR-IQA; 2) with the consideration of correlation between local quality and DMOS, a training data selection method is proposed to fine-tune the pre-trained model with valid SCI patches; 3) an adaptive pooling approach is employed to fuse patch quality to obtain image quality, owns strong noise robust and effects on both FR and NR IQA. Experimental results verify that our model outperforms both current no-reference and full-reference image quality assessment methods on the benchmark screen content image quality assessment database (SIQAD). Cross-database evaluation shows high generalization ability and high effectiveness of our model.     

Sparse representation-based image quality assessment

A highly promising approach to assess the quality of an image involves comparing the perceptually important structural information in this image with that in its reference image. The extraction of the perceptually important structural information is however a challenging task. This paper employs a sparse representation-based approach to extract such structural information. It proposes a new metric called the sparse representation-based quality (SPARQ) index that measures the visual quality of an image. The proposed approach learns the inherent structures of the reference image as a set of basis vectors. These vectors are obtained such that any structure in the image can be efficiently represented by a linear combination of only a few of these basis vectors. Such a sparse strategy is known to generate basis vectors that are qualitatively similar to the receptive field of the simple cells present in the mammalian primary visual cortex. To estimate the visual quality of the distorted image, structures in the visually important areas in this image are compared with those in the reference image, in terms of the learnt basis vectors. Our approach is evaluated on six publicly available subject-rated image quality assessment datasets. The proposed SPARQ index consistently exhibits high correlation with the subjective ratings of all datasets and overall, performs better than a number of popular image quality metrics.