基于改进Keren配准方法的超分辨率算法

提出一种基于边缘检测和Keren配准方法的自适应归一化卷积超分辨率重建算法。为了进一步提高低分辨率序列图像间的配准精度,该算法将边缘检测与Keren配准算法相结合。首先利用Roberts算子对图像序列进行边缘检测,然后利用基于简化四参数仿射变换模型的Keren改进算法求出边缘图像间的平移和旋转参数。仿真实验结果表明即使在含有噪声及大角度旋转情况下,相比Keren改进算法该算法配准精度得到了显著提高;其中采用Roberts算子相比其他传统算子可获得更高的配准精度。最后采用自适应归一化卷积超分辨率融合算法进行超分辨率重建,真实混叠图像序列的实验表明,基于提出的这种配准方法的超分辨率重建图像获得了很好的视觉效果和更高的分辨能力,具有良好的应用价值。     

基于核回归正则项的图像超分辨率重建

讨论了图像成像的基本模型,并提出了一种基于调整核回归函数作为正则项的序列图像重建算法。该算法是对已经提出的核回归算法的改进,减少其在超分辨率图像重建时的运算量。而且在图像配准过程中针对图像间只存在平移和旋转变换,采用了基于矩形像素值的亚像素配准方法,以提高配准的速度和精度。利用此算法对序列图像进行重建仿真,并通过结论得出其在噪声严重的情况下具有更好的边缘保留特性。     

多级FFD融合超分辨率重建的视频人脸识别

针对视频人脸识别中由于人脸畸变、表情变化等非刚性变化导致无法精确配准和重建的问题,提出一种基于多级自由变形配准的超分辨率重建算法。首先,利用低分辨率FFD网格全局配准,引入边缘配准度量到差平方总和准则;然后,将全局配准后的图像和基准图像划分成一系列对应子图对,使用高分辨率FFD网格对相关系数小的子图对进行局部配准;最后,采用凸集投影算法对多帧低分辨率图像重建SR人脸图像,并利用支持向量机分类器完成人脸识别。在标准视频库Choke Point和自己搜集的人脸视频库上的实验结果表明,在人脸畸变和表情变化很大的情况下,本文算法也能够精确配准和重建人脸图像,相比其它几种视频人脸识别算法,本文算法取得了更好的识别效果。     

基于振铃抑制的多视频超分辨率重建

为了利用较少低分辨率视频序列实现多视频超分辨率重建,本文提出一种将时空分别进行重建的算法。首先利用已有方法进行时间重建,再以得到的高时间分辨率的视频序列帧为参考帧,结合输入低分辨率视频序列帧进行空间重建。此外,针对传统重建方法在配准不精确的情况下会产生振铃现象这一问题,提出一种加入自适应惩罚项的改进迭代反投影(IBP)算法。实验结果表明,本文算法在输入低分辨率序列较少的情况下,能较好地实现多视频超分辨率重建,且能有效抑制振铃现象;重建出的高分辨率视频序列的结构相似度较对比算法提高3.4%~6.1%;在主观感受上,图像边缘锐利、人工效应少。     

一种用于动态视频超分辨率的多尺度最小二乘仿射块匹配图像配准方法

动态视频的超分辨率复原中,连续各帧图像间的精确匹配具有非常重要的意义。该文提出一种基于多尺度最小二乘仿射块匹配的图像配准方法。首先定义了一个指标Dmv来衡量图像的整体和局部匹配效果,并以此为基础设计了一种多尺度块选择机制,根据图像的运动情况选择匹配块大小,以兼顾图像中运动平坦和非平坦区域的匹配效果。与传统的块匹配方法不同,该文采用基于仿射模型的最小二乘配准方法实现各图像块的匹配,并通过修正步长的归一化处理解决了不同大小图像块在匹配时的收敛问题,从而在提高参数估计精度的同时降低了算法的运算量。最后,通过实验对算法的匹配性能及其对超分辨率复原算法整体性能的影响进行了测试。实验结果表明,该方法不仅可以实现更为准确的运动估计,当用于最大后验概率MAP超分辨率复原算法时,能够进一步有效提高算法的复原性能和实现速度。     

一种改进的超分辨率算法

利用多帧低分辨率图像重建一幅高分辨率图像成为迫切需要解决的难题,传统基于插值的超分辨率算法的发展受到了限制。本文基于重建方法,根据低分辨率图像帧间运动参数,提出了合理的权重分配算法。实验结果表明,图像超分辨率重建取得了良好效果。     

基于自相似性约束的视频稀疏超分辨率重建

通过研究帧间自相似性对图像重建的影响,提出一种自相似性约束的单视频稀疏超分辨率重建算法,以达到保持图像局部结构完整性的同时有效去噪的目的。该算法运用主成分分析PCA训练出适应图像不同局部结构的分类词典;通过帧间光流场的粗略运动估计和帧内帧间的精确块匹配,搜索自相似信息,运用非局部均值NLM滤波,并以此约束稀疏模型。仿真实验表明,提出的算法无论是客观指标,还是主观视觉上都超过了进行比较的几种分辨率提高算法。     

一种视频序列的超分辨率重建算法

本文构建了一种视频序列超分辨率重建框架.在此框架下,讨论了基于最小二乘规整化泛函的单帧图像的超分辨率重建算法及其收敛性、凸性和参数选择等;还提出了基于加权矩阵的运动配准融合,并研究了运动补偿阵和加权阵的构成和特点.仿真结果表明方法的有效性和实用性.     

一种视频序列的超分辨率重建算法

本文构建了一种视频序列超分辨率重建框架.在此框架下,讨论了基于最小二乘规整化泛函的单帧图像的超分辨率重建算法及其收敛性、凸性和参数选择等;还提出了基于加权矩阵的运动配准融合,并研究了运动补偿阵和加权阵的构成和特点.仿真结果表明方法的有效性和实用性.     

基于场景复杂度与不变特征的航拍视频实时配准算法

实时、鲁棒的图像配准是航拍视频电子稳像、全景图拼接和地面运动目标自动检测与跟踪的前提和关键技术.本文以航拍视频序列为处理对象,提出了一种新的基于场景复杂度与不变特征的实时配准算法,其主要特点包括：(1)在对航拍视频配准难点进行详细分析的基础上,有针对性的提出基于积分图的快速图像尺度空间构建、依据场景复杂度的检测特征点数量在线精确控制、基于描述子误差分布统计特性级的联分类器构造等新方法,使得算法配准性能不随场景的复杂度发生改变,能够在各种地貌条件下实时、稳定的进行图像配准;(2)将多尺度Harris角点和SIFT描述子相结合,并通过对帧间变换模型参数进行鲁棒估计,保证了算法具有良好的旋转、尺度、亮度不变性和配准精度.实验结果表明,算法可在场景变化、图像大幅度平移、尺度缩放和任意角度旋转等复杂条件下实时、精确的进行图像配准,对分辨率为320×240的航拍序列的平均处理速度达到20.7帧/秒.     

基于L曲线的自适应时空正则化视频超分辨率重建

为了解决视频超分辨率重建的病态问题,以得到良好的重建效果,提出了一种新颖的视频超分辨率重建算法。在算法中引入了时空联合正则化算子,通过视频帧本身的空间平滑信息和视频相邻帧的帧间相关先验信息的引入,提高了解的质量;同时,为了选择合适的时空正则化系数,提出了基于L曲线的自适应时空正则化系数计算方法,可以自适应地计算合适的正则化系数。通过对模拟图像序列和真实视频序列的实验结果表明,算法能得到较为精确的解,重建出具有良好视觉效果的高分辨率视频。     

基于累积直方图的视频镜头边界检测方法

提出了一种基于累积直方图的视频镜头边界检测方法,以累积直方图来代表视频帧图像的特征,其帧差充分反映了视频帧图像间的差异性;结合滑动窗的局部阈值分割处理,获得镜头边界检测对物体/摄像机的运动和光线变化的不敏感性。实验结果表明,本方法在镜头突变边界检测中达到95.97%的查全率和96.75%的查准率。     

生成式对抗神经网络的多帧红外图像超分辨率重建

生成式对抗神经网络在约束图像生成表现出了巨大潜力,使得其适合运用于图像超分辨率重建。但是使用生成式对抗神经网络重建后的超分辨率图像存在过度平滑,缺少高频细节信息的缺点。针对单帧图像超分辨率重建方法不能有效利用图像序列间的时间-空间相关性的问题,提出了一种基于生成式对抗神经网络的多帧红外图像超分辨率重建方法（M-GANs）。首先,对低分辨率图像序列进行运动补偿;其次,使用权值表示卷积层对运动补偿后的图像序列进行权值转换计算;最后,将其输入生成式对抗重建网络,输出重建后的高分辨率图像。实验结果表明:文中方法在主观及客观评价中均优于当前代表性的超分辨率重建方法。     

Rate control for consistent visual quality of H.264/AVC encoding

In this paper, a novel rate control scheme with sliding window basic unit is proposed to achieve consistent or smooth visual quality for H.264/AVC based video streaming. A sliding window consists of a group of successive frames and moves forward by one frame each time. To make the sliding window scheme possible for real-time video streaming, the initial encoder delay inherently in a video streaming system is utilized to generate all the bits of a window in advance, so that these bits for transmission are ready before their due time. The use of initial encoder delay does not introduce any additional delay in video streaming but benefits visual quality as compared to traditional one-pass rate control algorithms of H.264/AVC. Then, a Sliding Window Buffer Checking (SWBC) algorithm is proposed for buffer control at sliding window level and it accords with traditional buffer measurement of H.264/AVC. Extensive experimental results exhibit that higher coding performance, consistent visual quality and compliant buffer constraint can be achieved by the proposed algorithm.     

Choice of low resolution sample sets for efficient super-resolution signal reconstruction

In applications such as super-resolution imaging and mosaicking, multiple video sequences are registered to reconstruct video with enhanced resolution. However, not all computed registration is reliable. In addition, not all sequences contribute useful information towards reconstruction from multiple non-uniformly distributed sample sets. In this paper we present two algorithms that can help determine which low resolution sample sets should be combined in order to maximize reconstruction accuracy while minimizing the number of sample sets. The first algorithm computes a confidence measure which is derived as a combination of two objective functions. The second algorithm is an iterative ranked-based method for reconstruction which uses confidence measures to assign priority to sample sets that maximize information gain while minimizing reconstruction error. Experimental results with real and synthetic sequences validate the effectiveness of the proposed algorithms. Application of our work in medical visualization and super-resolution reconstruction of MRI data are also presented.     

Video super-resolution reconstruction based on deep convolutional neural network and spatio-temporal similarity

Existing learning-based super-resolution (SR) reconstruction algorithms are mainly designed for single image, which ignore the spatio-temporal relationship between video frames. Aiming at applying the advantages of learning-based algorithms to video SR field, a novel video SR reconstruction algorithm based on deep convolutional neural network (CNN) and spatio-temporal similarity (STCNN-SR) was proposed in this paper. It is a deep learning method for video SR reconstruction, which considers not only the mapping relationship among associated low-resolution (LR) and high-resolution (HR) image blocks, but also the spatio-temporal non-local complementary and redundant information between adjacent low-resolution video frames. The reconstruction speed can be improved obviously with the pre-trained end-to-end reconstructed coefficients. Moreover, the performance of video SR will be further improved by the optimization process with spatio-temporal similarity. Experimental results demonstrated that the proposed algorithm achieves a competitive SR quality on both subjective and objective evaluations, when compared to other state-of-the-art algorithms.     

Arbitrary Frame Rate Transcoding Through Temporal and Spatial Complexity

In this paper, an arbitrary frame rate transcoding joint considering temporal and spatial complexity of frames in the adaptive length sliding window is proposed. The length of a sliding window can be adjusted according to bandwidth variation in order to decide the number of skipped frames. The proposed method preserves significant frames and drops non-significant ones using the complexity measurements. Moreover, the motion vector composition algorithm is proposed to reduce the computations of motion estimation process by adopting the coding feature of variable block sizes in H.264/AVC video transcoder. Experimental results show that the proposed method achieves higher visual quality compared to other existing methods. After combining with the proposed fast motion composition algorithm, our proposed algorithm reduces encoding time significantly with slight visual quality degradation.      

Sliding‐window forward error correction using Reed‐Solomon code and unequal error protection for real‐time streaming video

Forward error correction (FEC) techniques are widely used to recover packet losses over unreliable networks in real‐time video streaming applications. Traditional frame‐level FEC encodes 1 video frame in each FEC coding window. By contrast, in the expanding‐window FEC scheme, high‐priority frames are included in the FEC processing of the following frames, so as to construct a larger coding window. In general, expanding‐window FEC improves the recovery performance of FEC, because the high‐priority frame can be protected by multiple windows and the use of a larger coding window increases the efficiency. However, the larger window size also increases the complexity of the coding and the memory space requirements. Consequently, expanding‐window FEC is limited in terms of practical applications. Sliding‐window FEC adopts a fixed window size in order to approximate the performance of the expanding‐window FEC method, but with a reduced complexity. Previous studies on sliding‐window FEC have generally adopted an equal error protection (EEP) mechanism to simplify the analysis. This paper considers the more practical case of an unequal error protection (UEP) strategy. An analytical model is derived for estimating the playable frame rate (PFR) of the proposed sliding‐window FEC scheme with a Reed‐Solomon erasure code for real‐time non‐scalable streaming applications. The analytical model is used to determine the optimal FEC configuration which maximizes the PFR value under given transmission rate constraints. The simulation results show that the proposed sliding‐window scheme achieves almost the same performance as the expanding‐window scheme, but with a significantly lower computational complexity.     

超分辨率重建算法综述

随着高分辨率移动设备和超高清电视的发展,对已有的低分辨率视频进行超分辨率上采样成为最近的一个研究热点.对已有的超分辨率重建算法根据输入输出方式的不同,分为多图像超分辨率重建、单图像超分辨率重建、视频超分辨率重建三大类,综述了其中每类算法的发展情况及常用算法,并对不同算法的特点分析比较.随后讨论了多图像超分辨率重建和单图像超分辨率重建方法对视频超分辨率重建方法的影响,最后展望了超分辨率重建算法的进一步发展.