Underwater image super-resolution using multi-stage information distillation networks

Recently, single image super-resolution (SISR) has been widely applied in the fields of underwater robot vision and obtained remarkable performance. However, most current methods generally suffered from the problem of a heavy burden on computational resources with large model sizes, which limited their real-world underwater robotic applications. In this paper, we introduce and tackle the super resolution (SR) problem for underwater robot vision and provide an efficient solution for near real-time applications. We present a novel lightweight multi-stage information distillation network, named MSIDN, for better balancing performance against applicability, which aggregates the local distilled features from different stages for more powerful feature representation. Moreover, a novel recursive residual feature distillation (RRFD) module is constructed to progressively extract useful features with a modest number of parameters in each stage. We also propose a channel interaction & distillation (CI&D) module that employs channel split operation on the preceding features to produce two-part features and utilizes the inter channel-wise interaction information between them to generate the distilled features, which can effectively extract the useful information of current stage without extra parameters. Besides, we present USR-2K dataset, a collection of over 1.6K samples for large-scale underwater image SR training, and a testset with an additional 400 samples for benchmark evaluation. Extensive experiments on several standard benchmark datasets show that the proposed MSIDN can provide state-of-the-art or even better performance in both quantitative and qualitative measurements.     

Reference guided image super-resolution via efficient dense warping and adaptive fusion

Due to the limited improvement of single-image based super-resolution (SR) methods in recent years, the reference based image SR (RefSR) methods, which super-resolve the low-resolution (LR) input with the guidance of similar high-resolution (HR) reference images are emerging. There are two main challenges in RefSR, i.e. reference image warping and exploring the guidance information from the warped references. For reference warping, we propose an efficient dense warping method to deal with large displacements, which is much faster than traditional patch (or texture) matching strategy. For the SR process, since different reference images complement each other, and have different similarities with the LR image, we further propose a similarity based feature fusion strategy to take advantage of the most similar reference regions. The SR process is realized by an encoder–decoder network and trained with pixel-level reconstruction loss, degradation loss and feature-level perceptual loss. Extensive experiments on three benchmark datasets demonstrate that the proposed method outperforms state-of-the-art SR methods in both subjective and objective measurements.     

Real-world single image super-resolution: A brief review

Single image super-resolution (SISR), which aims to reconstruct a high-resolution (HR) image from a low-resolution (LR) observation, has been an active research topic in the area of image processing in recent decades. Particularly, deep learning-based super-resolution (SR) approaches have drawn much attention and have greatly improved the reconstruction performance on synthetic data. However, recent studies show that simulation results on synthetic data usually overestimate the capacity to super-resolve real-world images. In this context, more and more researchers devote themselves to develop SR approaches for realistic images. This article aims to make a comprehensive review on real-world single image super-resolution (RSISR). More specifically, this review covers the critical publicly available datasets and assessment metrics for RSISR, and four major categories of RSISR methods, namely the degradation modeling-based RSISR, image pairs-based RSISR, domain translation-based RSISR, and self-learning-based RSISR. Comparisons are also made among representative RSISR methods on benchmark datasets, in terms of both reconstruction quality and computational efficiency. Besides, we discuss challenges and promising research topics on RSISR.     

Super-Resolution Reconstruction of Images Based on Microarray Camera

In the field of images and imaging, super-resolution (SR) reconstruction of images is a technique that converts one or more low-resolution (LR) images into a highresolution (HR) image. The classical two types of SR methods are mainly based on applying a single image or multiple images captured by a single camera. Microarray camera has the characteristics of small size, multi views, and the possibility of applying to portable devices. It has become a research hotspot in image processing. In this paper, we propose a SR reconstruction of images based on a microarray camera for sharpening and registration processing of array images. The array images are interpolated to obtain a HR image initially followed by a convolution neural network (CNN) procedure for enhancement. The convolution layers of our convolution neural network are 3×3 or 1×1 layers, of which the 1×1 layers are used to improve the network performance particularly. A bottleneck structure is applied to reduce the parameter numbers of the nonlinear mapping and to improve the nonlinear capability of the whole network. Finally, we use a 3×3 deconvolution layer to significantly reduce the number of parameters compared to the deconvolution layer of FSRCNN-s. The experiments show that the proposed method can not only ameliorate effectively the texture quality of the target image based on the array images information, but also further enhance the quality of the initial high resolution image by the improved CNN.     

基于洛伦兹范数的医学图像超分辨率重建研究

在超分辨率图像重建(SR)模型中,为了达到良好的重建效果,选择一个合适的代价函数是研究的重点。采用SR重建模型中的差错项选择了洛伦兹范数,正则化项选择了吉洪诺夫正则化,重建过程采用了迭代方法。提出的算法可以有效地解决医学图像SR重建过程中的去异值点和图像边缘保持的两大关键问题,达到良好的重建效果。为了验证上述算法的有效性,就一系列添加了运动模糊和不同噪声的低分辨率MRI医学图像进行了SR重建,并且与基于L2范数的重建算法的重建效果进行了比较分析。实验结果显示,所提算法具有良好的实用性和有效性。     

Fast super-resolution algorithm using rotation-invariant ELBP classifier and hierarchical pattern matching

This paper proposes a fast super-resolution (SR) algorithm using content-adaptive two-dimensional (2D) finite impulse response (FIR) filters based on a rotation-invariant classifier. The proposed algorithm consists of a learning stage and an inference stage. In the learning stage, we cluster a sufficient number of low-resolution (LR) and high-resolution (HR) patch pairs into a specific number of groups using the rotation-invariant classifier, and choose a specific number of dominant clusters. Then, we compute the optimal 2D FIR filter(s) to synthesize a high-quality HR patch from an LR patch per cluster, and finally store the patch-adaptive 2D FIR filters in a dictionary. Also, we present a smart hierarchical addressing method for effective dictionary exploration in the inference stage. In the inference stage, the ELBP of each input LR patch is extracted in the same way as the learning stage, and the best matched FIR filter(s) to the input LR patch is found from the dictionary by the hierarchical addressing. Finally, we synthesize the HR patch by using the optimal 2D FIR filter. The experimental results show that the proposed algorithm produces better HR images than the existing SR methods, while providing fast running time.     

用于细胞三维结构观察的现代显微影像仪器及相关技术

细胞三维结构的观察分析能提供更多细胞显微水平结构和功能的信息。现代显微影像仪器及相关的三维重建等技术是研究细胞三维结构的有力工具。本文概述多种可用于细胞三维结构观察的显微工具,简单介绍其原理、优势、最新技术动态及应用领域等。     

非局部联合稀疏近似的超分辨率重建算法

该文结合联合稀疏近似和非局部自相似的概念,提出非局部联合稀疏近似的超分辨率重建方法。该方法将输入图像的跨尺度高、低分辨率图像块统一进行联合稀疏编码,建立它们之间的稀疏关联,并将这种关联作为先验知识来指导图像的超分辨率重建。该文方法保证跨尺度自相似集具有相同的稀疏性模式,能更有效地利用图像的自相似性先验信息,提高算法的自适应性。通过自然图像实验,与其它几种基于学习的超分辨率算法对比,超分辨率效果有较好改善。     

A modular neural network for super-resolution of human faces

This paper presents the original and versatile architecture of a modular neural network and its application to super-resolution. Each module is a small multilayer perceptron, trained with the Levenberg-Marquardt method, and is used as a generic building block. By connecting the modules together to establish a composition of their individual mappings, we elaborate a lattice of modules that implements full connectivity between the pixels of the low-resolution input image and those of the higher-resolution output image. After the network is trained with patterns made up of low and high-resolution images of objects or scenes of the same kind, it will be able to enhance dramatically the resolution of a similar object’s representation. The modular nature of the architecture allows the training phase to be readily parallelized on a network of PCs. Finally, it is shown that the network performs global-scale reconstruction of human faces from very low resolution input images.